You can add a custom LLM Token Tracker panel to a tab on your Admin Hub from the available widget presets. This panel shows the tokens that have been sent and received by the Chatbot while it is in operation, organized by LLM provider.
Understanding the Tile
The LLM Token Tracker Mini diagnostic tiles allow you to monitor token creation for requests and responses handled by your LLMs:
Each Mini tile shows the following details:
- Name: The LLM Provider name.
- Outbound: Tokens sent to the LLM.
- Inbound: Tokens received by Pyramid from the LLM.
Filtering
The following options filter your view:
- Duration: Count the LLM tokens for the selected time period. This can be any time from the Last Day to All Time. It is important to note that your Transaction Log Settings affect which tokens are retained. This means that, if you select All Time, Pyramid will return the count of tokens that have been retained given your transaction logging configuration.
- Refresh: Use the drop-down at the top-right to indicate how frequently you would like to refresh this panel.
Where do I see this tile?
This tile is not included on the Admin Hub Dashboard by default, you must add it using the supplied Widget Preset. For details describing how to perform this customization, see Admin Hub.