Find answers from the community

Updated 9 months ago

I am trying to use `IngestionPipeline`

I am trying to use IngestionPipeline to be executed locally but it has a dependency on llama_index_client which comes from a poetry module mapping:

Plain Text

poetry_requirements(
    name="poetry",
    module_mapping={"llamaindex-py-client": ["llama_index_client"]},
)

which is not installed by default, and even when installing llamaindex-py-client with pip, there is no way to get this mapping.

Therefore, I am wondering if we are able to use IngestionPipeline standalone ?

8 comments

LLogan M

hmm, this mapping is only used in our CICD, specifically by a tool called pants -- curious how this is causing issues?

llamaindex-py-client is included as a direct dependency of llama-index-core, which the ingestion pipeline comes from

GGeoloeG

https://github.com/run-llama/llama_index/blob/37c95965426bddae82cec1ad49d3aa82e8bfe819/llama-index-core/llama_index/core/ingestion/pipeline.py#L15

and the corresponding trace in message.txt

LLogan M

@GeoloeG I've never seen anyone have this issue before 😅 does pip show llamaindex-py-client show anything? pip install llamaindex-py-client should install it if not

GGeoloeG

indeed, it was not properly installed... but missed it in the Dockerfile which had a --no-deps for the pip install, let's see what are the conflicts...

GGeoloeG

fixed the dep issue! Thanks @Logan M

GGeoloeG

playing around with the RedisVectorStore, it seems that redisearch is a dep, but again is not installed with redis (5.0.1), will investigate and add this as dep if really needed for the integration

LLogan M

Actually, there's a large PR to refactor the redis vector store to use redisVL lol

LLogan M

Should be merged in a few days

Add a reply