Find answers from the community

Updated 4 months ago

hey guys just curious - reranker seems

At a glance

hey guys just curious - reranker seems to improve performance for my RAG, but makes it waaay longer to generate responses, any tips for reducing this?

5 comments

LLogan M

Use a reranker that doesn't run locally? (i.e. cohere?)

LLogan M

Or run your reranking on a GPU

RRZ

yeah makes sense

RRZ

just thinking maybe its possible to finetune a smaller reranker

RRZ

but maybe thats not great performance

Add a reply