Find answers from the community

Updated last year

Logan M When I m using Multi hop Agent

@Logan M When I'm using Multi-hop Agent calls, there's increased response times for queries especially when calling multiple tools across agents. Is there a caching solution that I can use to help reduce response times?

8 comments

LLogan M

We've been meaning to integrate something like gpt cache, but we haven't gotten around to it yet lol

LLogan M

I agree it would probably be helpful

VVish

I see

VVish

What about implementations like Langchain's InMemoryCache or something

LLogan M

I think you'd have to use a langchain LLM class (which then should have caching)

LLogan M

And that should be compatible with llama index 🫰

VVish

Well Langchain LLMs aren't compatible with OpenAI Agents iirc

VVish

What I meant was more so porting what they are doing for caching, natively to llama index somehow

Add a reply