Overview
The OpenHands SDK provides metrics tracking at two levels: individual LLM metrics and aggregated conversation-level costs:- You can access detailed metrics from each LLM instance using the
llm.metricsobject to track token usage, costs, and latencies per API call. - For a complete view, use
conversation.conversation_statsto get aggregated costs across all LLMs used in a conversation, including the primary agent LLM and any auxiliary LLMs (such as those used by the context condenser).
Getting Metrics from Individual LLMs
A ready-to-run example is available here!Track token usage, costs, and performance metrics from LLM interactions:
Accessing Individual LLM Metrics
Access metrics directly from the LLM object after running the conversation:llm.metrics object is an instance of the Metrics class, which provides detailed information including:
accumulated_cost- Total accumulated cost across all API callsaccumulated_token_usage- Aggregated token usage with fields like:prompt_tokens- Number of input tokens processedcompletion_tokens- Number of output tokens generatedcache_read_tokens- Cache hits (if supported by the model)cache_write_tokens- Cache writes (if supported by the model)reasoning_tokens- Reasoning tokens (for models that support extended thinking)context_window- Context window size used
costs- List of individual cost records per API calltoken_usages- List of detailed token usage records per API callresponse_latencies- List of response latency metrics per API call
Reported costs and custom model names
For Chat Completions and Responses API calls, the SDK selects one charge per response in this order:- A valid provider cost header (
llm_provider-x-litellm-response-costin the assembled response’s additional headers), including a zero charge. - A positive response
usage.cost, unless both custom token rates are configured. This lets custom model aliases report charges without a static pricing entry. - LiteLLM’s cost calculator, using explicit
input_cost_per_tokenandoutput_cost_per_tokenwhen both are configured, or available model pricing.
llm.metrics.costs. If no
reported cost or usable pricing is available, the SDK emits a cost-calculation
warning and adds no cost entry. An accumulated total of zero alone therefore
does not establish that every call was free.
The SDK cannot infer separately billed routing or model-selection calls from the
final model’s token counts. Those charges must be included in an agreed reported
total or recorded separately by the application.
Prompt Cache Support
The SDK enables automatic prompt cache markers whencaching_prompt=True and
model capability detection reports support. It uses LiteLLM model metadata when
available, with a built-in capability list as a fallback. That fallback includes
Claude Haiku 5.5 and Opus 5.5, including Anthropic and LiteLLM-proxy model names.
Set caching_prompt=False when constructing LLM to disable automatic cache
markers. After a provider call, inspect cache_write_tokens and
cache_read_tokens to check reported cache activity. Actual cache use depends on
the provider’s requirements, including its minimum cacheable prompt length.
Ready-to-run Example (LLM metrics)
This example is available on GitHub: examples/01_standalone_sdk/13_get_llm_metrics.py
examples/01_standalone_sdk/13_get_llm_metrics.py
The model name should follow the LiteLLM convention:
provider/model_name (e.g., anthropic/claude-sonnet-4-5-20250929, openai/gpt-4o).
The LLM_API_KEY should be the API key for your chosen provider.Using LLM Registry for Cost Tracking
A ready-to-run example is available here!The LLM Registry allows you to maintain a centralized registry of LLM instances, each identified by a unique
usage_id. This is particularly useful for tracking costs across different LLMs used in your application.
How the LLM Registry Works
Each LLM is created with a uniqueusage_id (e.g., “agent”, “condenser”) that serves as its identifier in the registry. The registry maintains references to all LLM instances, allowing you to:
- Register LLMs: Add LLM instances to the registry with
llm_registry.add(llm) - Retrieve LLMs: Get LLM instances by their usage ID with
llm_registry.get("usage_id") - List Usage IDs: View all registered usage IDs with
llm_registry.list_usage_ids() - Track Costs Separately: Each LLM’s metrics are tracked independently by its usage ID
Ready-to-run Example (LLM Registry)
This example is available on GitHub: examples/01_standalone_sdk/05_use_llm_registry.py
examples/01_standalone_sdk/05_use_llm_registry.py
The model name should follow the LiteLLM convention:
provider/model_name (e.g., anthropic/claude-sonnet-4-5-20250929, openai/gpt-4o).
The LLM_API_KEY should be the API key for your chosen provider.Getting Aggregated Conversation Costs
This example is available on GitHub: examples/01_standalone_sdk/21_generate_extraneous_conversation_costs.py
conversation.conversation_stats. This is particularly useful when your conversation involves multiple LLMs, such as the main agent LLM and auxiliary LLMs for tasks like context condensing.
examples/01_standalone_sdk/21_generate_extraneous_conversation_costs.py
The model name should follow the LiteLLM convention:
provider/model_name (e.g., anthropic/claude-sonnet-4-5-20250929, openai/gpt-4o).
The LLM_API_KEY should be the API key for your chosen provider.Understanding Conversation Stats
Theconversation.conversation_stats object provides cost tracking across all LLMs used in a conversation. It is an instance of the ConversationStats class, which provides the following key features:
Key Methods and Properties
-
usage_to_metrics: A dictionary mapping usage IDs to their respectiveMetricsobjects. This allows you to track costs separately for each LLM used in the conversation. -
get_combined_metrics(): Returns a singleMetricsobject that aggregates costs across all LLMs used in the conversation. This gives you the total cost of the entire conversation. -
get_metrics_for_usage(usage_id: str): Retrieves theMetricsobject for a specific usage ID, allowing you to inspect costs for individual LLMs.
Conversation Budgets
Setmax_budget_per_run when creating a conversation:
max_budget_per_run in
POST /api/conversations. The value is a finite, positive USD amount;
omitting it or sending null leaves the conversation unbounded.
Despite the option’s name, the existing SDK limiter compares the limit with
accumulated conversation cost across all tracked LLMs, including the
condenser. Starting another run does not reset that cost.
The SDK persists the limit in base_state.json, alongside conversation state;
the server’s meta.json does not store it. Both standalone SDK resume and server
restart restore the saved limit. A fork keeps the limit and resets cost by
default; reset_metrics=false preserves cost.
When reopening a local conversation, omitting max_budget_per_run or passing
None keeps its saved limit. Passing a number replaces it. Set
conversation.max_budget_per_run = None on the local conversation to clear and
persist the limit.
The limit is checked after each agent step, except while waiting for tool
confirmation. An LLM call or step can overshoot the budget. If the limit is
reached and the agent still has work to do, the check stops the run with
execution_status="error" and a ConversationErrorEvent whose code is
MaxBudgetReached. A step that finishes the task keeps finished status,
including when it overshoots.
Displaying Budget Usage
Read the limit frommax_budget_per_run in the create response or
GET /api/conversations/{id}. Sum the accumulated_cost values under
stats.usage_to_metrics to get the spend compared with that limit:
ConversationStateUpdateEvent messages with
key="stats" carry updated metrics. Replace the cached stats with the event’s
value and recalculate the sum. A state update with
key="max_budget_per_run" carries a changed limit. REST and base_state.json
reflect completed steps; an in-flight call’s cost is available only after usage
is reported.
Hide the percentage when the limit is absent or null. It can exceed 100%:
keep the actual spend visible even if the progress bar is capped visually.
The legacy per-LLM max_budget_per_task metric is not this conversation limit.
Next Steps
- Context Condenser - Learn about context management and how it uses separate LLMs
- LLM Routing - Optimize costs with smart routing between different models

