Python: [Bug]: Token Count Discrepancy between Microsoft Foundry and Agent Framework - Potential Billing Issue

[vg-utilitas]

Description

Billing and Token Usage Discrepancy in Microsoft Foundry Agent vs. Agent Framework

We have a Microsoft Foundry Agent with tool calls.
We've rebuilt this Agent inside the Agent Framework using the Harness and ran it locally.
Both Agents have the exact same behavior. Same traces, same tool calls, same results, same system prompt.

Microsoft Foundry Agent Call:

• Microsoft Foundry reports an input token count of 35k

Agent Framwork Local Agent Call:

• UsageDetails reports INFO __main__: Token usage (turn): in=144378 out=831 total=145209 reasoning=269 cached_in=105887

In the Corresponding Model Monitor in Foundry:

Before the Calls:

• Total requests: 0
• Total token count: 0
• Avg per request: 0

After the Foundry Agent Call

• Total requests: 1
• Total token count: 35k
• Avg per request: 35

After the Agent Framework Agent Call (after the Foundry Agent Call)

• Total requests: 1 (did not get logged as request should be 2)
• total token count: 180k (145k+35k)
• Avg per request: 180k

Our system prompt has 14k tokens.
It seems that the Agent Framework forwards the entire history to the agent after each tool call.
This seems not to be the case for the Foundry Agent.
Or the tokens are counted differently.
If so, then the Foundry Agent probably only counts the non cached input tokens and reports them to the Model Monitor.
While the Agent Framework reports the entire token count including the cached input tokens to the Model Monitor.

Unsure how this discrepancy affects billing in the agent framework compared to foundry.

If this is intended, a short explanation about how this affects billing would be appreciated.

Thanks in advance.

Code Sample

session_usage: UsageDetails | None = None

agent = create_harness_agent(
        client=client,
        name="productsearch-agent",
        agent_instructions=_load_system_prompt(),
        tools=[get_facets, search_field],
        default_options={"store": True, "allow_multiple_tool_calls": False, "response_format": ProductRecommendations},
        disable_file_memory=True,
        middleware=[UsageLoggingMiddleware()],
        auto_approval_rules=[SkillsProvider.read_only_tools_auto_approval_rule],
    )

session = agent.create_session()
result = await agent.run(user_input, session=session)
session_usage = add_usage_details(session_usage, result.usage_details)

Error Messages / Stack Traces

Package Versions

agent-framework: 1.16.0, agent-framework-core: 1.16.0,

Python Version

Python 3.13

Additional Context

No response

[github-actions]

👋 Hi vg-utilitas — thanks for the report!

Your reported 144,378 input tokens include the 105,887 cached-token subset, leaving 38,491 uncached tokens—close to Foundry's 35k figure. Agent Framework maps these provider fields directly, so cached input should not be added total input and is billed at the provider's discounted cached-input rate. Your Model Monitor's unchanged request count appears to reflect service-side metric semantics or latency; confirming that display requires the raw usage response for each model call and the deployment/model details.

If you still believe there is a framework issue here, please reply with the exact import/class or method you are using, the version, and a minimal snippet that shows the failure. 🙏

[vg-utilitas]

The main issue is probably that Microsoft Foundry and the Agent Framework report two different numbers to the Model Monitor
As long as this does not affect billing the issue is more of an logging issue for the Model Monitor in Foundry.

During the Agent Framework Agent Call with middleware token usage logger:

Start

INFO test_agent.agent: Model call usage: {'input_token_count': 21079, 'output_token_count': 260, 'total_token_count': 21339, 'openai.cache_write_tokens': 0, 'cache_creation_input_token_count': 0, 'openai.cached_input_tokens': 21036, 'cache_read_input_token_count': 
21036, 'openai.reasoning_tokens': 232, 'reasoning_output_token_count': 232}

Tool Call

INFO test_agent.agent: Model call usage: {'input_token_count': 31728, 'output_token_count': 243, 'total_token_count': 31971, 'openai.cache_write_tokens': 10649, 'cache_creation_input_token_count': 10649, 'openai.cached_input_tokens': 21036, 'cache_read_input_token_count': 21036, 'openai.reasoning_tokens': 76, 'reasoning_output_token_count': 76}

Tool Call

INFO test_agent.agent: Model call usage: {'input_token_count': 32425, 'output_token_count': 371, 'total_token_count': 32796, 'openai.cache_write_tokens': 697, 'cache_creation_input_token_count': 697, 'openai.cached_input_tokens': 31685, 'cache_read_input_token_count': 31685, 'openai.reasoning_tokens': 95, 'reasoning_output_token_count': 95}

Tool Call

INFO test_agent.agent: Model call usage: {'input_token_count': 35804, 'output_token_count': 127, 'total_token_count': 35931, 'openai.cache_write_tokens': 3379, 'cache_creation_input_token_count': 3379, 'openai.cached_input_tokens': 32382, 'cache_read_input_token_count': 32382, 'openai.reasoning_tokens': 50, 'reasoning_output_token_count': 50}

End of Agent Call

INFO __main__: Token usage (turn): in=121036 out=1001 total=122037 reasoning=453 cached_in=106139

Agent Framework seems to report the sum of all input tokens (121k), while Foundry seems to report the last context input token count (35k) to the Foundry Model Monitor.

from agent_framework import Agent, ChatMiddleware, ChatResponse, SkillsProvider, create_harness_agent
from agent_framework.foundry import FoundryChatClient
from agent_framework import Content, Message, UsageDetails, add_usage_details

class UsageLoggingMiddleware(ChatMiddleware):
    async def process(self, context, call_next):
        """
        :param context: The chat invocation context for a single model call
        :param call_next: The next middleware or the leaf chat client
        """
        await call_next()
        if isinstance(context.result, ChatResponse):
            logger.info("Model call usage: %s", context.result.usage_details)

session_usage: UsageDetails | None = None

agent = create_harness_agent(
        client=client,
        name="productsearch-agent",
        agent_instructions=_load_system_prompt(),
        tools=[get_facets, search_field],
        default_options={"store": True, "allow_multiple_tool_calls": False, "response_format": ProductRecommendations},
        disable_file_memory=True,
        middleware=[UsageLoggingMiddleware()],
        auto_approval_rules=[SkillsProvider.read_only_tools_auto_approval_rule],
    )

session = agent.create_session()
result = await agent.run(user_input, session=session)
session_usage = add_usage_details(session_usage, result.usage_details)