Skip to content

SMOODEV-3364: Tool spans record argument key names, never values - #572

Merged
brentrager merged 1 commit into
mainfrom
SMOODEV-3364-tool-span-pii
Sep 27, 2026
Merged

brentrager merged 1 commit into
mainfrom
SMOODEV-3364-tool-span-pii

Conversation

@brentrager

@brentrager brentrager commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Problem

SMOODEV-3364. Every server's gen_ai.tool span recorded gen_ai.tool.call.arguments, which is the tool's JSON arguments with only secret-named keys masked (redact_tool_arguments). Tool arguments are customer data: a CRM write carries a person's name, email, phone and address. So every exporter's trace store received that PII verbatim. Blocking more key names wouldn't fix it, and neither would a pattern matcher, because a name or a note matches no pattern.

A second problem: a host that wraps its tools in its own tracing decorator (the smooai monorepo's TracedTool) got two gen_ai.tool spans per call, one from the decorator and one from the runner. That doubled every "N tool calls" count. In prod, conversation bd75b5d9 shows 2 crm__stats spans for 1 call.

Change

  • Argument key names, never values, in all five servers. The span now records gen_ai.tool.argument_keys (sorted top-level keys, comma-joined) and no longer records gen_ai.tool.call.arguments. This covers the Rust server runner.rs, KnowledgeChatRuntime, and the TypeScript, Python, Go and .NET servers. A new tool_argument_keys helper in each language asserts one shared set of vectors: an object gives its keys; null or empty input gives ""; an array gives <array>; any other scalar gives <scalar>; unparseable input gives <unparsed>. redact_tool_arguments and GEN_AI_TOOL_ARGUMENTS stay exported for API compatibility, but no span uses them now. Go's unexported copy of the redactor was dead code, so it is removed.
  • ToolProvider::traces_own_tools() (Rust, default false). When it returns true, the runner skips its own span for that provider's tools. Built-in and extension tools keep their spans. An extension tool that replaces a host tool of the same name gets the runner's span back. This seam is Rust-first: TypeScript has a function-typed provider seam, and Python, Go and .NET have no provider seam yet.
  • Docs: docs/Operations/Observability.md. Changeset: minor.

Defense in depth on the storage side: SmooAI/smooai#5128 scrubs span attributes at api-prime's trace ingest for every emitter.

Verification

  • Rust: tool_argument_keys_never_carry_values passes. The span tests run_turn_records_gen_ai_spans and streaming_turn_emits_gen_ai_spans_with_org_and_tool_args now assert three things: the key list is query, gen_ai.tool.call.arguments is absent, and the query text appears in no span field. New tests self_traced_host_tools_get_no_runner_span and host_tools_keep_the_runner_span_by_default cover the new seam. Results: smooth-operator lib 3/3, telemetry 4/4; server telemetry 5/5. Clippy is clean with -D warnings.
  • TypeScript: telemetry.test.ts, 4 passed. Python: test_telemetry.py, 3 passed. Go: TestStreamingTurnEmitsGenAISpans and TestToolArgumentKeysNeverCarryValues pass, and go vet is clean. .NET: TelemetryTests, 10 passed.
  • Mutation-checked. I put each span site back to recording raw arguments. The span tests then failed in all five languages, and removing the self-traced skip failed self_traced_host_tools_get_no_runner_span.

Follow-up after release

The monorepo still needs to bump smooai-smooth-operator-server and implement traces_own_tools() -> true on the chat-ws and copilot-ws providers, so each call gets one span.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CfZaWemmpghhtofBauYdti

Every server's gen_ai.tool span recorded gen_ai.tool.call.arguments: the tool's
JSON arguments with only secret-NAMED keys masked. Tool arguments are customer
data (a CRM write carries a name, email, phone and address), so every
exporter's trace store received that PII verbatim, and no pattern matcher can
recognise a name or a note. The span now records gen_ai.tool.argument_keys
(sorted top-level key names) instead, in the Rust server + runtime and the
TypeScript, Python, Go and .NET servers, with one shared set of test vectors.

A host that wraps its tools in its own tracing decorator also got two spans per
call, doubling every tool-call count. Rust's ToolProvider gains
traces_own_tools() (default false); when true, the runner skips its span for
that provider's tools while built-in and extension tools keep theirs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CfZaWemmpghhtofBauYdti
@changeset-bot

changeset-bot Bot commented Sep 27, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 6ae9b3d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@smooai/smooth-operator Minor
@smooai/smooth-operator-web-chat-example Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@brentrager
brentrager merged commit aa31b5c into main Sep 27, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant