You ll need to start with a fresh index

LLogan M

You'll need to start with a fresh index if you switch embeddings, the dimensions of every embedding vector need to be the same 👍

21 comments

PPocketColin

hmm I'm generating a GPTVectorStoreIndex from_documents and then persisting it. I re-run that and a new /storage directory is created with the new embeddings (afaik). Is there something else I need to do to ensure that the index is new?

LLogan M

When you initially create the index using from documents, are you using the modified embeddings there?

LLogan M

Whichever embeddings model you use to create the index, you should use for the query as well 🤔

PPocketColin

mmm I don't think that's the case? The default model for embeddings is ADA-2 but it's GPT-3.5-turbo for the query

PPocketColin

and generating embeddings with ada-2 and then querying with gpt-3.5-turbo works fine

PPocketColin

I ran into the same issue generating embeddings with Curie

LLogan M

Right. Ada is the default for embeddings (mostly because it's fast, cheap, and works well)

Maybe I should make a quick example of what I mean to make this work haha one sec

PPocketColin

this might also help: I've abstracted my service_context and return basically this to do both the embeddings creation and querying:

Plain Text

prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit)
llm_predictor = LLMPredictor(llm=ChatOpenAI(temperature=1, model_name="gpt-3.5-turbo"))
embed_model = OpenAIEmbedding(model=OpenAIEmbeddingModelType.ADA)

service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper, embed_model=embed_model)

PPocketColin

(that doesn't work though because ADA-1 also appears to have the same issue)

LLogan M

Right, so when you call from_documents(), you'll need to pass in that service context

.from_documents(documents, service_context=service_context)

Then, maybe you persist it

index.storage_context.persist()

When you load it again, you need to pass in the same service context

index = load_index_from_storage(storage_context, service_context=service_context)

As long as the service context is the same for both steps, it should be working 🤔

PPocketColin

yep that's exactly what I'm doing!

LLogan M

Hmmmm

LLogan M

I've definitely managed to change the embeddings fine before (using huggingface embeddings) so something weird is going on lol

LLogan M

Two things that might work? Lol