Find answers from the community

Updated 3 months ago

Hey How does Llama Index split long

Hey. How does Llama Index split long texts? Eg langchain provides some features (textsplitter, recursiveTextSplitter, SpacySplitter and so on). What about Llama Index?
j
1 comment
Hi, we split the text under the hood by default, but you can choose to use a langchain text splitter too! (e.g. pass in a text_splitter arg when building an index https://gpt-index.readthedocs.io/en/latest/reference/indices.html#module-gpt_index.indices.base)

You could also choose to "pre-split" the text into separate Document objects before feeding into gpt index
Add a reply
Sign up and join the conversation on Discord