Chatbot context sharing issue across multiple users

PPragyan Mohapatra

I am currently building a chatbot that reads data from different pdfs. I use chatengine to retrieve queries. chat engine keeps the context, but when multiple people are using the bot at the same time, the context is shared in the same memory.

I want every session to have individual memory, currently user 1 is asking about x, user 2 asking about y, then when user 1 is asking about tell me more, it says more about y instead of x because the last context it has is about y. It is not taking into account individual user’s context.
Can I please get some help on this @Logan M

25 comments

LLogan M

Separate the memory per-user

You can pass in the chat history to override at any time

chat_engine.chat(..., chat_history=chat_history) -- where chat history is a list of chat message objects

You can get the current chat history with chat_engine.chat_history

LLogan M

Feel free to manage and store that anywhere

LLogan M

You could also use a chat store
https://docs.llamaindex.ai/en/stable/module_guides/storing/chat_stores/

PPragyan Mohapatra

Ok I tried getting chathistory from chatengine.chat_history and use it inside in chatengine just the way you suggested.
chat_engine.chat(..., chat_history=chat_history) -- where chat history is a list of chat message objects

It still takes latest chat context irrespective of the user

PPragyan Mohapatra

To answer the question tell me more

PPragyan Mohapatra

It is still giving more of latest question irrespective of the user

LLogan M

Yes, but you need to manage that chat history per user (i.e. each user needs to have their own chat_history list)

PPragyan Mohapatra

@Logan M how do I do that

LLogan M

Idk, its up to you lol its general software engineering at this point

You could throw it into redis, some in-memory dictionary, some sql db

And just pull the chat history for each user as needed

PPragyan Mohapatra

Do we have unique user Id in the chat history @Logan M I see the list of ChatMessage Objects, how do we get the userids of the chatmessages, to understand which chatmessage is of whose userid

LLogan M

Ideally only one user is chatting with a given agent/chat history at a given time right? Why would more than one user end up in the chat history of its being managed properly?

PPragyan Mohapatra

Many people can chat at the same time, isnt it? I want each of their memory retained to them separately.
When I pull the chat history to segregate, Can I get any I’d that specifies the unique user @Logan M

LLogan M

I think you should maintain chat history per user

LLogan M

Way easier

LLogan M

I think I've suggested that earlier too 😅

PPragyan Mohapatra

Yes, but for that the chat history that comes back after chat is a list, so the last message should go to which user I’d if new users have started conversing @Logan M
Is there any example?

LLogan M

I think i gave an example right?

It's quite simple

Plain Text

response = chat_engine.chat("hello", chat_history=chat_history)

You just have to maintain that list for each user. It's a list of pydantic objects so it's easy to serialize into any db or whatnot

PPragyan Mohapatra

I understand this part, But chat_history is coming for all the users @Logan M

LLogan M

But it doesn't have to be?

Plain Text

chat_histories = {"user1": [], "user2": []}

response = chat_engine.chat("hello", chat_history=chat_histories["user2"])

chat_histories["user2"].append(ChatMessage(role="assistant", content=respons.response))

LLogan M

If you are using an agent, you'll probably want to include the tool calls in the history. For that, I would just init the agent each time

Plain Text

agent = FunctionCallingAgent.from_tools(...)
response = agent.chat("hello", chat_history=chat_histories["user2"])

chat_histories["user2"] = agent.chat_history

PPragyan Mohapatra

Thanks @Logan M This worked. Also Can we get timestamp from ChatMessage

PPragyan Mohapatra

Since I have userids and corresponding chatmessage objects, I want to delete the userids that were used 1hour back

PPragyan Mohapatra

@Logan M

LLogan M

You'll have to generate the timestamps yourself (this is pretty easy for for most SQL dbs, timestamps are built in)

PPragyan Mohapatra

Sure, Thanks a lot @Logan M
Everything that you suggested worked.

The only thing is response time is very large, how can I reduce it? Any suggestions? @Logan M

Add a reply

Find answers from the community

Chatbot context sharing issue across multiple users