👋 Is there a way to change the default templates in schema.py (specifically DEFAULT_TEXT_NODE_TMPL)? The template is used by the get_content method on the TextNode.

Are there any tools in Llama_Index or Langchain that can be used for cleaning up nodes before embedding? I'm looking for something to help remove white space, special characters, etc.

Hello 👋 ! I'm trying to persist a Weaviate vector store to an S3 bucket, but I don't seem to be able to store the actual vector_store.json file. It's generating the docstore.json, graph_store.json, and index_store.json files, but no vector store. Any idea why that's happening? I've used this with the standard llama_index vector store without any issues. The code is pretty simple:

Err weird question hoping anyone else is/has seeing this: Has anyone else's retriever suddenly become very dumb over night? I'm using OpenAI's Davinci model for embedding and I swear yesterday it was doing great. Today, I query something I've queried before and it returns the worst nodes. I'm so confused.

Why would setting query_str="", custom_embedding_strs=[ "whatever" ] in the QueryBundle that is passed to a VectorIndexRetriever.retrieve result in 0 nodes being found but giving query_str="" a value suddenly result in nodes found? Based on the docs, I thought that query_str isn't used for embedding-based queries?

Hello! I have a question about VectorIndexRetriever . When I try to set the vector_store_query_mode to VectorStoreQueryMode.TEXT_SEARCH I get the following error: Invalid query mode: VectorStoreQueryMode.TEXT_SEARCH

Looking through the code, I don't see how you could ever use that query mode as it seems to only accept learning modes, MMR, and default? https://github.com/jerryjliu/llama_index/blob/main/llama_index/vector_stores/simple.py#L228

👋 long time no see! Quick question: Is there a data connector similar to SimpleDirectoryReader that would allow me to specify the filesystem (in the same way that persist takes an optional fs param)? Specifically, I'm trying to pass it a s3fs.S3FileSystem so it can grab a PDF from my S3 bucket.

Quick question - Can someone clarify what num_outputs refers to in the PromptHelper class? Is that the number of tokens allowed in the output? The name is throwing me off

Hey ya'll - is there a way to get the query engine to submit multiple nodes at once with queries? Is that what embedding_limit is used for? I'm running into an issue where my Vector Store Index (when using the Curie model on OpenAI) is creating small chunks because it's creating a chunk per page in the documents it's indexing. I like that it's keeping track of pages, but the chunk sizes are so small that the query doesn't have enough context to create a decent response. Does that make sense? Alternatively, a chunk minimum size limit could also work I guess if that's an option available (I don't see anything like that in the docs)?

Hey ya'll! I'm super excited to be diving into in-context training and am loving llama_index. I'm wondering if anyone can help point me in the right direction for a response synthesis that is additive as opposed to refining. For example: I have a retriever with similarity_top_k=5 and I would like to generate 5 items for each set of nodes retrieved. I was trying to get this to work using Refine and modifying the refine_template but it really doesn't seem to want to attach a new answer to the old one in the response. Should I try a different response synthesizer (like maybe Tree Summarize?) or should I keep working with my prompts?

Is there a quick way to grab nodes by page number? I have an array of page numbers and I'd like to grab all of the nodes associated with those pages.

Feedback request post: I'm working on a project to pull relevant page references from books based on a question. For example: if I asked "what spells has Harry Potter used?", the AI could respond "check pages 50, 100, and 120 for information on that". I'm using OpenAI to do Text Search Embeddings to create the Vectors for my index and then passing the full questions to the retriever and it works pretty well but I would love feedback on what I could tweak/test to get better results. I've managed to get the retriever to be accurate about 88% of the time (i.e., 12% of the time it misses a key page I would have expected it to return).

Happy Monday, all! Has anyone ever run into OpenAI GPT refusing to generate JSON? I'm ending my prompt with "Format response as JSON" and it's saying "I'm sorry, as a language model AI, I am not capable of generating JSON format" which is silly because I've seen it generate JSON before.

Does anyone have any experience getting OpenAI to increase their usage limit? We hit our hard limit yesterday and I requested an increase, but I couldn't find any information on how they approve requests or expectations around what you can request. For example, is it insane to request a limit of $1000? Or is that low?

Would people find it useful if the RetrieverQueryEngine query method took an optional retriever_query_bundle param in case you want your retriever's query to be different from the one the response synthesizer uses? I'm feeling like I'd like to be able to fine tune the retriever with the query bundle but I don't want to change what the synthesizer gets.