Repository navigation
feat(server-utils): Emit execute_tool spans for Anthropic's beta.messages.toolRunner() - #25320
Conversation
…essages.toolRunner()` Wraps each runnable tool's `run()` in place with `fill()`, covering tools passed to `toolRunner()`, `addTools()` and `setMessagesParams()`, on both the orchestrion and `instrumentAnthropicAiClient` paths. A tool that throws gets an error span and a captured event, since the runner hides the error from the app. Fixes #25288 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… setting them to undefined Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…echanism type Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
size-limit report 📦
|
|
bugbot run |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
isaacs
left a comment
isaacs
left a comment
There was a problem hiding this comment.
Wrapipng tool objects in place rather than wrapping the runner or proxying is the right choice for this SDK: the runner uses private class fields, the client code already avoids Proxies for that reason, and tool.run is the one call point that is stable from 0.63 to 0.133.
Capturing the swallowed error is the useful part of the change, since otherwise those errors disappear.
Open questions are what counts as "handled", and keeping the run contract narrow enough that new kinds of tools entries, like toolsets, don't get wrapped by accident. Otherwise LGTM!
…rors as handled Toolsets are nameless `tools` entries whose `run` takes `(ctx, toolUse)`, so only entries with a string `name` are wrapped now. Tool errors the runner hands back to the model are captured with `handled: true`, as Flue and Mastra do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 5e0cfc3. Configure here.
…on older SDKs SDK 0.85 to 0.96 pass only `toolUseBlock` to a tool's `run()`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
isaacs
left a comment
isaacs
left a comment
There was a problem hiding this comment.
This looks great!
Anthropic SDK 0.133 adds client.beta.sessions.events.toolRunner(sessionID, { tools }) (lib/tools/SessionToolRunner.ts), which also runs BetaRunnableTools. Those tools get no spans unless the same objects were also given to a beta.messages.toolRunner() (in which case they get spans with the wrong framing, see 1). I think that's out of scope for this PR, but worth a follow-up issue so the PR body saying "the tool runner" isn't read as full coverage.
| // Handled: the runner turns the throw into an error result for the model, so the app never sees it. | ||
| // A `ToolError` is the SDK's way to send that result on purpose. | ||
| if ((error as Error | undefined)?.name !== 'ToolError') { | ||
| captureException(error, { mechanism: { handled: true, type: ANTHROPIC_AI_ORIGIN } }); |
There was a problem hiding this comment.
So, fill replaces run on the user's own tool object for the life of the process. Tool objects are often module-level constants, reused in other places: a direct call in app code or tests, an MCP server, another agent framework, or client.beta.sessions.events.toolRunner().
Every such call now gets an execute_tool span with origin: auto.ai.anthropic, and any throw gets captured with mechanism: { handled: true, type: 'auto.ai.anthropic' }. In those contexts the caller does see the error, so "handled by the Anthropic runner" is wrong. The __sentry_captured__ guard stops a second capture when the app also reports it, but the first (wrong) mechanism wins.
There is no clean way to tell a runner call from a direct call on 0.63 to 0.80 (no context arg). On 0.85+ the runner always passes a context object with toolUseBlock.
I don't think we really need to change the code here, but we might want to document this in the PR body and/or a code comment, since it's a deliberate tradeoff. Maybe we could only capture when context?.toolUse ?? context?.toolUseBlock is present, which is true for every runner call on 0.85+ and false for most direct calls. That would keep the span everywhere but limit the "handled" capture to calls we know came from the runner. Older SDKs would then lose capture, which may be acceptable, or keep capture there unconditionally.
There was a problem hiding this comment.
yah you're right on all, I like your proposal, I took your gate to capture only with toolUse/toolUseBlock, span stays everywhere. 0.63 to 0.82 call run(input) bare, so those keep the error span but no event. fwiw the sessions runner also passes that context and swallows, so handled: true is right there
`fill` keeps the tool's `run` wrapped after the runner is done, so a direct call from app code also went through it and its error was captured as handled by the runner, dropping the app's own capture. Only runner calls pass a `toolUse`/`toolUseBlock` context (SDK 0.85+), so capture is limited to those, while the span stays on every call. Also narrows the `toolRunner` hook to SDK 0.63+, where the method first ships. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3 participants
beta.messages.toolRunner()runs the user's tools, but Sentry only recorded its chat calls. Tool calls got no span, and when a tool threw, the runner turned the error into anis_errorresult for the model, so nobody saw it.This wraps each tool's
run()in place withfill(). When a tool throws, its span is marked as an error. The exception is captured as handled only when the call came from a runner. A direct call rethrows to its caller, so it isn't captured, and neither are intentionalToolErrorresults and aborted calls. Toolsets are skipped, since theirruntakes(ctx, toolUse).sessions.events.toolRunner()isn't covered yet.Fixes #25288
🤖 Generated with Claude Code