Repository navigation
fix(server-utils): Use cumulative input and cache token counts from streamed message_delta - #25623
pcbeingused333 wants to merge 2 commits into
Conversation
…treamed message_delta Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 4ac20c7. Configure here.
RulaKhaled
left a comment
There was a problem hiding this comment.
Thanks for working on this! logic looks right and matches what the SDK's own MessageStream accumulator does with message_delta usage. one small comment, nothing blocking
CI failures look unrelated
| stop_reason: null, | ||
| stop_sequence: null, | ||
| usage: { | ||
| input_tokens: 2285, |
There was a problem hiding this comment.
l-m: could you add a case where message_start has input_tokens: 0 and the real count only shows up in message_delta?
…in by message_delta Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Added in dfd9154: a case where |
handleMessageMetadatain the Anthropic streaming instrumentation only tookoutput_tokensfrommessage_delta, while the input and cache counts stayed at theirmessage_startvalues. Themessage_deltausage is cumulative and these counts can be larger there, for example with server tools such as web search, wheremessage_deltareports more input and cache-creation tokens thanmessage_start. Spans for those streams under-reportedgen_ai.usage.input_tokensand the cache attributes.Input, cache creation and cache read counts from
message_deltanow overwrite the stored ones when they are numbers, and the prompt token total is recomputed from them. Anullor missing count keeps themessage_startvalue, so streams that don't resend them are unchanged. Themessage_deltausage type gains the optional input and cache fields.This change was made with Claude Code.
Fixes #25621