Find answers from the community

Updated 3 months ago

How would you save this to disk with the

How would you save this to disk with the newest update?

Plain Text

# Load documents
documents = SimpleDirectoryReader('data').load_data()

# Build the GPTVectorStoreIndex
index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context)

6 comments

LLogan M

Plain Text

index.storage_context.persist(persist_dir="./storage")

from llama_index import load_index_from_storage, ServiceContext

storage_context = StorageContext.from_defaults(persist_dir="./storage")
index = load_index_from_storage(storage_context, service_context=service_context)

TTeemu

Thank you! I see that now they're stored in 3 different files? Is there a reason for that?

LLogan M

It's to allow for more flexible storage in the future. There's the docstore (your nodes), index struct (index information) and vector store (any vectors)

LLogan M

At least thats my take lol I also wasn't the one who designed it, so I can't say for sure

TTeemu

Ah I see, I guess I'll just go with it then. Thank you again, you're a lifesaver ❤️

LLogan M

I need to play around with it more, but I think if you give each index an ID, you can save/load multiple to the same folder. At least that's what the docs make me think haha

https://gpt-index.readthedocs.io/en/latest/how_to/storage/save_load.html

Add a reply