SMOODEV-3364: Tool spans record argument key names, never values - #572
Merged
Merged
Conversation
Every server's gen_ai.tool span recorded gen_ai.tool.call.arguments: the tool's JSON arguments with only secret-NAMED keys masked. Tool arguments are customer data (a CRM write carries a name, email, phone and address), so every exporter's trace store received that PII verbatim, and no pattern matcher can recognise a name or a note. The span now records gen_ai.tool.argument_keys (sorted top-level key names) instead, in the Rust server + runtime and the TypeScript, Python, Go and .NET servers, with one shared set of test vectors. A host that wraps its tools in its own tracing decorator also got two spans per call, doubling every tool-call count. Rust's ToolProvider gains traces_own_tools() (default false); when true, the runner skips its span for that provider's tools while built-in and extension tools keep theirs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CfZaWemmpghhtofBauYdti
🦋 Changeset detectedLatest commit: 6ae9b3d The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
SMOODEV-3364. Every server's
gen_ai.toolspan recordedgen_ai.tool.call.arguments, which is the tool's JSON arguments with only secret-named keys masked (redact_tool_arguments). Tool arguments are customer data: a CRM write carries a person's name, email, phone and address. So every exporter's trace store received that PII verbatim. Blocking more key names wouldn't fix it, and neither would a pattern matcher, because a name or a note matches no pattern.A second problem: a host that wraps its tools in its own tracing decorator (the smooai monorepo's
TracedTool) got twogen_ai.toolspans per call, one from the decorator and one from the runner. That doubled every "N tool calls" count. In prod, conversation bd75b5d9 shows 2crm__statsspans for 1 call.Change
gen_ai.tool.argument_keys(sorted top-level keys, comma-joined) and no longer recordsgen_ai.tool.call.arguments. This covers the Rust serverrunner.rs,KnowledgeChatRuntime, and the TypeScript, Python, Go and .NET servers. A newtool_argument_keyshelper in each language asserts one shared set of vectors: an object gives its keys;nullor empty input gives""; an array gives<array>; any other scalar gives<scalar>; unparseable input gives<unparsed>.redact_tool_argumentsandGEN_AI_TOOL_ARGUMENTSstay exported for API compatibility, but no span uses them now. Go's unexported copy of the redactor was dead code, so it is removed.ToolProvider::traces_own_tools()(Rust, defaultfalse). When it returnstrue, the runner skips its own span for that provider's tools. Built-in and extension tools keep their spans. An extension tool that replaces a host tool of the same name gets the runner's span back. This seam is Rust-first: TypeScript has a function-typed provider seam, and Python, Go and .NET have no provider seam yet.docs/Operations/Observability.md. Changeset: minor.Defense in depth on the storage side: SmooAI/smooai#5128 scrubs span attributes at api-prime's trace ingest for every emitter.
Verification
tool_argument_keys_never_carry_valuespasses. The span testsrun_turn_records_gen_ai_spansandstreaming_turn_emits_gen_ai_spans_with_org_and_tool_argsnow assert three things: the key list isquery,gen_ai.tool.call.argumentsis absent, and the query text appears in no span field. New testsself_traced_host_tools_get_no_runner_spanandhost_tools_keep_the_runner_span_by_defaultcover the new seam. Results: smooth-operator lib 3/3, telemetry 4/4; server telemetry 5/5. Clippy is clean with-D warnings.telemetry.test.ts, 4 passed. Python:test_telemetry.py, 3 passed. Go:TestStreamingTurnEmitsGenAISpansandTestToolArgumentKeysNeverCarryValuespass, andgo vetis clean. .NET:TelemetryTests, 10 passed.self_traced_host_tools_get_no_runner_span.Follow-up after release
The monorepo still needs to bump
smooai-smooth-operator-serverand implementtraces_own_tools() -> trueon the chat-ws and copilot-ws providers, so each call gets one span.🤖 Generated with Claude Code
https://claude.ai/code/session_01CfZaWemmpghhtofBauYdti