Find answers from the community

Updated last year

Chat history with index not working

How can I use chat history in combination with an index? In this example the AI does not know anything about the chat history and just seems to try and query the index.

Plain Text
@app.route("/history")
def history():
    # Load data
    documents = SimpleDirectoryReader("./src/data/paul_graham").load_data()

    # create index
    index: VectorStoreIndex = VectorStoreIndex.from_documents(documents)

    custom_prompt = PromptTemplate(
        """\
    Given a conversation (between Human and Assistant) and a follow up message from Human, \
    rewrite the message to be a standalone question that captures all relevant context \
    from the conversation.

    <Chat History>
    {chat_history}

    <Follow Up Message>
    {question}

    <Standalone question>
    """
    )

    # list of `ChatMessage` objects
    custom_chat_history = [
        ChatMessage(
            role=MessageRole.USER,
            content="Remember that John Doe is wearing a blue shirt.",
        ),
        ChatMessage(role=MessageRole.ASSISTANT,
                    content=(
                        "Certainly, I'll remember that John Doe is wearing a blue shirt."
                    )
                    ),
    ]

    query_engine = index.as_query_engine()

    chat_engine = CondenseQuestionChatEngine.from_defaults(
        query_engine=query_engine,
        condense_question_prompt=custom_prompt,
        chat_history=custom_chat_history,
        verbose=True,
    )

    chat_response: AgentChatResponse = chat_engine.chat(
        "What color shirt is John Doe wearing?",
        # tool_choice="query_engine_tool"
    )

    pprint("answer:")
    pprint(chat_response.response)

    return "Done."

Response:

Plain Text
I'm sorry, but I cannot answer that question based on the given context information.
M
L
3 comments
Also, can I / should I use a service context with this? Because It seems like I can't use tool_choice even when a service context was created.
This is the wrong kind of chat engine to use for this purpose I think

It doesn't use tool_choice, or even pick any tools.

It takes the chat history, rephrases it into a query, queries the index, and returns that response from the index.
You probably want to use CondensePlusContextChatEngine, ContextChatEngine, or an agent
Add a reply
Sign up and join the conversation on Discord