Skip to content

fix(server-utils): Use cumulative input and cache token counts from streamed message_delta - #25623

Draft
pcbeingused333 wants to merge 2 commits into
getsentry:developfrom
pcbeingused333:fix/anthropic-stream-input-usage
Draft

pcbeingused333 wants to merge 2 commits into
getsentry:developfrom
pcbeingused333:fix/anthropic-stream-input-usage

Conversation

@pcbeingused333

Copy link
Copy Markdown

handleMessageMetadata in the Anthropic streaming instrumentation only took output_tokens from message_delta, while the input and cache counts stayed at their message_start values. The message_delta usage is cumulative and these counts can be larger there, for example with server tools such as web search, where message_delta reports more input and cache-creation tokens than message_start. Spans for those streams under-reported gen_ai.usage.input_tokens and the cache attributes.

Input, cache creation and cache read counts from message_delta now overwrite the stored ones when they are numbers, and the prompt token total is recomputed from them. A null or missing count keeps the message_start value, so streams that don't resend them are unchanged. The message_delta usage type gains the optional input and cache fields.

This change was made with Claude Code.

Fixes #25621

…treamed message_delta

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the external PR from an external contributor label Oct 21, 2026
@RulaKhaled RulaKhaled changed the title fix(anthropic-ai): use cumulative input and cache token counts from streamed message_delta fix(server-utils): Use cumulative input and cache token counts from streamed message_delta Oct 22, 2026
@RulaKhaled

Copy link
Copy Markdown
Collaborator

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 4ac20c7. Configure here.

@RulaKhaled RulaKhaled left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! logic looks right and matches what the SDK's own MessageStream accumulator does with message_delta usage. one small comment, nothing blocking

CI failures look unrelated

stop_reason: null,
stop_sequence: null,
usage: {
input_tokens: 2285,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l-m: could you add a case where message_start has input_tokens: 0 and the real count only shows up in message_delta?

…in by message_delta

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@pcbeingused333

Copy link
Copy Markdown
Author

Added in dfd9154: a case where message_start reports input_tokens: 0 and the real count (1500) only arrives in message_delta. Runs for both the async iterable and the message stream, and expects input 1500 and total 1540.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external PR from an external contributor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anthropic streaming spans keep message_start input/cache token counts instead of the final message_delta usage

2 participants