Find answers from the community

Updated 2 weeks ago

Tools for Agent: Efficient Utilization and Prompt Guidance

hello there, I have written 2 sets of tools for my agent using FunctionCallingAgentWorker. The first set of tool includes 3 QueryEngineTool and the second set includes 1 FunctionTool (pydantic base model, does simple addition on any numbers of input floats).

I put them all into a list and pass it to my agent. During query time, the agent is always using the first set of tools and not using the second set at all. In my prompt, do I need to include statements like "use this tool when doing this task" sort of sentence?
L
g
11 comments
when you saw you put them into a list, you mean a list like [too1, tool2, tool3, tool4] right?

The biggest impacts to tool usage
  • descriptions and tool names
  • the llm you are using (not every llm is good at being an agent(
Yes exactly as you said, a single list containing all tools. As for LLM, gpt-4o.
Are your tool names/descriptions just very vague or unrelated to the input message?
Plain Text
# define tools for query engine
    vector_tool = QueryEngineTool.from_defaults(
        query_engine=mm_query_engine,
        name="vector_tool",
        description=(
            """Useful for retrieving specific context from the data."""
        ),
    )

    page_amounts_list_tool = QueryEngineTool.from_defaults(
        query_engine=mm_query_engine,
        name="page_amounts_list_tool",
        description=(
            """Useful for retrieving total amount excluding tax from all line 
            items in a given page and putting them in a list."""
        ),
    )
    
    invoice_amount_validation_tool = QueryEngineTool.from_defaults(
        query_engine=mm_query_engine,
        name="invoice_amount_validation_tool",
        description=(
            """Useful for validating the sum of all amounts of all line items
            in the invoice against the total amount excluding tax of the
            invoice. Validation is successful only when the two amounts are 
            equal."""
        ),
    )

# define function tools with pydantic basemodels
    class SumAmounts(BaseModel):
        """Data class for summing amounts of line items in a list."""
        amounts: List[float] = Field(..., title="List of total amounts excluding tax of line items to sum.")
        
    def sum_amounts(input_amounts: SumAmounts) -> float:
        """Sums a list of amounts provided in input_amounts."""
        return round(sum(input_amounts.amounts), 2)
    
    sum_tool = FunctionTool.from_defaults(
        fn=sum_amounts,
        name='sum_amounts',
        description='Sums a list of amounts provided in the "amounts" field.'
    )
i think they are quite specific?
Useful for retrieving specific context from the data. -- context and data about what?

sum_amounts -- why not just have a list[float] as input to the function to simplify the prompt?

It seems like to me these tools kind of suggest a specific order of actions to take. Rather than just letting them LLM decide the order, you could use some lower level functions and a workflow to put the system more on rails. Stuff like limiting the tools available depending on some global state, or executing tools in a specific order.

Hard to say how well these tools will work in practice without seeing the input prompt to the agent.
Those are all my tips πŸ™‚
my reason for introducing agents into my traditional rag application is to try to see if i can have an automated validation layer (sum of table line item's amount in the invoice equals to its totals). llama-parse has already helped me a lot in the parsing so that's awesome.

i've watched some of your workflow introduction on youtube. it seems to be a little too complicated for me now. and its quite verbose. for my use case i'm not sure if i will be needing workflow. i might give it a try, for now will like to see if agents alone can solve my problem.
can i share my inputs with you to have a look?
It can be verbose, but tbh hoping that 5 lines of code will solve all your problems probably isn't a sustainable approach for production apps (not yet, anyways lol)

Even if you shared the initial agent.chat(), and what you expect to happen, that would help a lot
fair point, i'll give it a go later. i want to try out agents alone for now. i have some progress yesterday and reading the outputs from the llm i can see the agent is already using my tools from the first and second set (changing the descriptions really help!). i will prepare some screenshots and send it here
Add a reply
Sign up and join the conversation on Discord