Counting tokens in llm instances without a query engine
Counting tokens in llm instances without a query engine
At a glance
The community member is asking how to count tokens when using LLM instances in a workflow without using a query engine, and whether the TokenCountingHandler needs to be set globally in a callback manager. In the comments, another community member suggests skipping the token counting handler and creating a custom counter using newer instrumentation, providing a link to a Colab notebook. The second comment indicates that this will track every LLM call made. There is no explicitly marked answer in the post and comments.