Oooo that prompt is going to cause issues. Had a chat with another guy who had a similar issue
The curly brackets are also used in the template to mark variables, and the backets in your example are causing issues 🙃
My solution for the other guy was just be verbose in describing what you want instead of giving an example
I.e. "Return your response as valid json..."
this was working before hmm
usually the double {{ works.
I also had the same prompt but without response_mode="tree_summarize"
Yea the response mode does something finicky with the prompt 🫠
i always find it so hard to read the source code idk why
maybe because i am a javascript dev by trade
ok this is silly and probably a lesson on python more than anything, but, I'm trying to follow the GPTSimpleVectorIndex query method to understand what response_mode does
so I follow the constructor into SimpleVectorStore, and get to the query method here:
def query(
self,
query_embedding: List[float],
similarity_top_k: int,
doc_ids: Optional[List[str]] = None,
) -> VectorStoreQueryResult:there isn't even a response_mode!!
but not super intuitive lol
it's like 3 abstractions up
Yea, the code is kind of like a Russian doll situation lol
For myself, the only way I understood it all was to use a debugger and step through and into every function call lol
and since we call super().init
what is the debugger called in python?
that's a good idea. i'll try to figure it out with vscode
You can use pdb if you are familiar with gdb, otherwise an IDE like vscode or pycharm can provide a gui debugger (I always use pdb personally lol)
thanks, added you on linkedin.
Okay I dug in and really couldn’t figure out what’s going on. It adds context str input variable but no context? Maybe I need to look at how another index handles it
Hmm, yea that's what it looks like, somehow the context_str is missing, or it's pulling in more variables?
also it gets expensive debugging this stuff
because sometimes you need to send a bunch of tokens to get a correct result., have you figured out a way to combat that?
Usually I just set breakpoints before calling the LLM, so I can inspect issues before making lllm calls and then kill it
or if i need the llm calls, I just use gpt3.5 so its cheap lol
Unless you mean debugging the model and not the code lol then not really a work around for that 🙃
yeah the simplevectorindex seems to be missing somerthing like
summaries = [
self._llm_predictor.predict(
self.summary_prompt, context_str=text_chunk
)[0]
for text_chunk in text_chunks
]which is in a GPTTreeIndex
why am I use a simplevectorindex instead of tree?
Why does simple vector index need summaries? 👀
Reminder: I am generating titles, summaries, linkedin threads, twitter threads
from a transcription of a podcast.
I think my issue is I'm trying to get away with one index for all of those.. but, all of them are technically summaries just worded differently.
so I should be using a TreeIndex yeah?
Yea i kind of agree.
I think a tree index might be the way to go (costs to build, but cheaper to query compared to a list index)
and I can store the the tree index as like a json blob in a postgres if i wanted yeah
response_mode="tree_summarize" doesn't work on a list index either
which i guess makes sense
Well, it should work 🙃 but again, your prompt and how llama index/langchain/python does string formatting is causing issues :PepeHands:
okay, so, treeindex w/ response_mode=tree_summarize and no top_k_similarity needs.
Actually for tree index, use mode="summarize" i think
I really need to setup debugger to fix this
Can you send your prompt again? Imma debug this now too haha seems too silly
python
prompt = f"""You are a copywriter. Write three concise summaries, make sure each of them are unique. {self.extra_prompt} \n {self.host_names}. {self.guest_names}. \n Write it as an exciting and captivating manner. Add 'On this episode of' to the beginning. Try to include all topics. No longer than 200 tokens \n Make sure the length of each summary is no longer than 4 sentences. Return the format in a JSON Object {{"summaries": ["Summary 1", "Summary 2", "Summary 3"]}}:"""which comes out to:
prompt = f"""You are a copywriter. Write three concise summaries, make sure each of them are unique. Write it as an exciting and captivating manner. Add 'On this episode of' to the beginning. Try to include all topics. No longer than 200 tokens \n Make sure the length of each summary is no longer than 4 sentences. Return the format in a JSON Object {{"summaries": ["Summary 1", "Summary 2", "Summary 3"]}}:"""but in the error response it looks like it's stripping the {{ into {
so maybe it's doing that prematurely
Maybe you need 3 curly brackets lol
If I find the issue, I'll let you have the PR honors
I need to figure out how to setup debugging lol through this convoluted docker compose i have
now I'm getting a different error with
{{{ langchainapi-langchain-1 | File "/srv/src/gpt.py", line 58, in summary
langchainapi-langchain-1 | prompt = f"""You are a copywriter. Write three concise summaries, make sure each of them are unique. {self.extra_prompt} \n {self.host_names}. {self.guest_names}. \n Write it as an exciting and captivating manner. Add 'On this episode of' to the beginning. Try to include all topics. No longer than 200 tokens \n Make sure the length of each summary is no longer than 4 sentences. Return the format in a JSON Object {{{"summaries": ["Summary 1", "Summary 2", "Summary 3"]}}}:"""
langchainapi-langchain-1 | ValueError: Invalid format specifieroof, debugging inside docker can be a pain
No PR needed I think, just need to change how build your prompt
@bmax Removing the
f from the start of the string fixed it (I think then it keeps the double brackets)
So, solution is to do something like this
extra_prompt = """test test"""
query_str = f"""Write three unique titles of the text. {extra_prompt} \n Write it as an exciting podcast description. Act as an Copywriter. Try to include all topics. No longer than 200 tokens \n"""
query_str += """Return the format in a JSON Object {{"titles": ["Title 1", "Title 2", "Title 3"]}}\n return only valid JSON:"""Basically append the JSON example at the end, instead of doing it all in one go
Wait the f causes the issue
Like the formatted python string?!
I think by using it it gets rid of the double brackets too early?
[mycolabs-langchain] [2023-04-18 22:33:06] INFO:openai:error_code=None error_message='That model is currently overloaded with other requests. You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID 4e12db8e33737809e93b65ae0e5d5939 in your message.)' error_param=None error_type=server_error message='OpenAI API error received' stream_error=False
Lol I saw that yesterday too.
Guess the servers are busy busy
