Repository navigation
feat(voltagent): @traceai/voltagent span processor (TH-8239) - #215
Merged
Merged
Conversation
New package typescript/packages/traceai_voltagent. FIVoltAgentSpanProcessor
goes into VoltAgent's ObservabilityConfig.spanProcessors (no Agent patching).
On each ended VoltAgent span it builds a copy with fi.span.kind,
gen_ai.span.kind, openinference.span.kind, session.id, gen_ai.request.model,
gen_ai.provider.name, gen_ai.usage.*, gen_ai.tool.* and exports the copy
through the register() provider's exporter with that provider's resource
(project_name, project_type). The span VoltAgent hands to VoltOps and its
other processors is not modified. Content is off unless captureContent: true.
@voltagent/core is a peer dependency (^2.11.0) and is not bundled.
Tested against @voltagent/core 2.11.0 (npm latest), ai 6.0.97
MockLanguageModelV3, Node 20.20.2 / 22.23.3 / 26.8.1. No vendor calls.
Tests: 48 jest tests (mapping, real VoltAgent agent with mock model, AC-04
subagent, AC-06 coexistence, streamText, failure isolation, global-provider
misconfiguration) and 11 shared-harness contract tests (python/tests/harness
Receiver, real fi-core exporter to /tracer/v1/traces, burst, collector down,
pack check, ESM/CJS on three Node versions).
pnpm-lock.yaml: only this importer plus 115 packages / 124 snapshots it
needs (+1391 lines, 0 removed). pnpm 10.30.1 also re-resolved zod and
ts-node peers in other importers; those changes were left out. Verified
with pnpm install --frozen-lockfile --ignore-scripts.
Deviations from TH-8239 architecture/ADR/docs (paths are @voltagent/core
2.11.0 sources recovered from dist/index.js.map):
- LLM spans use their own keys, not ai.model.name/usage.*: llm.model,
llm.provider, llm.messages, llm.usage.prompt_tokens/... (agent.ts:4552-4558,
4607, 4643-4655). ai.model.name and usage.* are on the agent root only
(trace-context.ts:331, 352-365). Both are mapped.
- usage.* on the root is the operation total (usage-normalizer.ts:16-23,
agent.ts:1446-1455), so it goes to voltagent.usage.*, not gen_ai.usage.*
(the collector promotes token keys on any span and Observe sums the trace).
Promoted keys are written on llm spans only.
- The llm span records response.usage, the last AI SDK step only
(agent.ts:1413-1419). The processor holds llm spans until the operation
root ends; with exactly one successful main call it exports the root total
on that llm span and keeps the step values in
voltagent.llm.last_step_usage.*. Trace token sum now equals the mock
model's calls (24 = 11 + 13; tested).
- Tool name key exists: tool.name, tool.call.id (agent.ts:6451-6452), so
gen_ai.tool.name / gen_ai.tool.call.id are set.
- memory.read is a read (agent.ts:5044-5047, memory.operation=read) ->
RETRIEVER; memory.write / memory.steps.write (memory-manager.ts:123-127,
326-329) -> CHAIN.
- span-helpers.ts (embedding.query, vector.query) is not in 2.11.0;
2.11.0 emits embedding.generate / vector.search with vector.operation and
input (memory-manager.ts:1010-1040). Both vocabularies are handled.
- More content keys than input/output: agent.instructions
(trace-context.ts:315), agent.messages/.ui, agent.context,
agent.stateSnapshot (agent.ts:1270-1279), llm.messages (agent.ts:4607),
middleware.input/output.* (middleware.ts:196-331), guardrail.input/output.*
(guardrail.ts:327-552), tool.search.query (agent.ts:6971). All dropped by
default. Credential-like keys are always dropped.
- Copy, do not delete holds for the original span. The exported copy drops
content, moves promoted keys off non-model spans, and rewrites llm.usage.*
on a reconciled llm span.
- Cost: VoltAgent does set usage.cost, from OpenRouter metadata only
(agent.ts:4666). Passed through, not promoted to gen_ai.cost.total.
- Batching: register() attaches a SimpleSpanProcessor (fi-core otel.ts:385)
and the OTLP exporter rejects more than 30 requests in flight
(otlp-exporter-base 0.202.0 configuration/shared-configuration.js:43-45,
otlp-export-delegate.js:46). A 20-run burst delivered 30 of 200 spans.
register({ batch: true }) writes _registeredSpanProcessors (otel.ts:599),
which SDK 2.x does not read. The processor wraps the provider's exporter in
its own BatchSpanProcessor (queue 2048). It never shuts that exporter down.
- register() defaults to setGlobalTracerProvider=true (otel.ts:486, 605).
VoltAgentObservability then cannot register its provider and takes the
global tracer (node/volt-agent-observability.ts:93-96), so VoltAgent spans
skip every spanProcessors entry, unmapped and with content. Docs require
setGlobalTracerProvider: false; the constructor warns; a test reproduces it.
- wait-until.ts read: setWaitUntil(waitUntil: (promise: Promise<unknown>)
=> void): void (wait-until.ts:30). Documented with forceFlush.
- Cache/reasoning tokens are also written to the traceAI TS keys
gen_ai.usage.cache_read_tokens / gen_ai.usage.output_tokens.reasoning
next to the architecture's OTel names.
- Exported resource is the Future AGI provider's resource; VoltAgent's
service.name=voltagent resource is not merged.
- License: the @voltagent/core 2.11.0 tarball has no LICENSE file.
package.json says MIT; README.md:355-357 and docs/community/licence.md
carry the MIT text ("Copyright (c) 2025 VoltAgent").
approved: Nikhil 2026-10-03 blanket
Refs: TH-8239
fi-collector promotes cost into its hot cost column on any span from
gen_ai.cost.total / llm.cost.total, or, when no total is set, from
input + output: gen_ai.cost.input / llm.cost.prompt plus
gen_ai.cost.output / llm.cost.completion (fi-collector ca5aeda8e
pkg/adapter/adapter.go DeriveHotKeys, costInputKeys / costOutputKeys).
isPromotedUsageKey only knew the two total keys, so a cost part on an
agent, tool or memory span would have been exported on its promoted key
and summed into the trace by Observe.
Add the four cost-part keys to the promoted set: on non-model spans they
move under voltagent.*, on llm spans they stay. VoltAgent 2.11.0 itself
writes none of them (it sets usage.cost from OpenRouter metadata only), so
this closes a latent path rather than a live one. Unit test added first
(failed on isPromotedUsageKey("gen_ai.cost.input") === false); the
contract test's promoted-key list and the README Tokens section now name
the same keys.
approved: Nikhil 2026-10-03 blanket
Refs: TH-8239
…fault With captureContent=false (the default) the Future AGI copy still carried text that @voltagent/core 2.11.0 writes outside the keys the content filter knew: - agent.summary.preview, agent.summary.text: the model-written conversation summary on the span.type=summary span (agent/apply-summarization.ts). - agent.workingMemory.finalContent: the whole working-memory document on the agent root (agent.ts). - planagent.todos: the plan (planagent/planning/index.ts). - planagent.task.description, planagent.task.response_preview: the delegation brief and the subagent's answer (planagent/plan-agent.ts). Their last segments (preview, text, finalContent, todos, description, response_preview) are not content segments, so isContentKey let them through. Add all six to CONTENT_EXACT, list them in the README's dropped keys, and add them to the AC-07 content carrier. A new test checks each is absent with captureContent off and present with it on. Installed-review findings N1 and M2. approved: Nikhil 2026-10-03 blanket Refs: TH-8239
@voltagent/core 2.11.0 sets workflow.stateSnapshot on every workflow root span (workflow/core.ts). Besides the step source code it carries the run input: the snapshot's `inputSchema` field is assigned the input value (`inputSchema: input`), not a schema. With captureContent=false the Future AGI copy exported it, so a workflow's input reached the collector. agent.stateSnapshot was already dropped for the same reason. Found while writing the real-workflow test for span events (M1): with event filtering in place, the workflow input still showed up in the exported attributes through this key. Add it to CONTENT_EXACT, to the README's dropped keys and to the AC-07 content carrier. approved: Nikhil 2026-10-03 blanket Refs: TH-8239
copySpan forwarded span.events unchanged, so the captureContent=false default did not hold for event attributes. @voltagent/core 2.11.0 puts content there: - guardrail.stream.process, one event per streamed chunk on an output guardrail with a streaming handler, carries the chunk in guardrail.chunk.text (agent/streaming/output-guardrail-stream-runner.ts), so the FI copy carried the streamed answer. - workflow.suspended carries suspension.data and suspension.checkpoint, and workflow.resumed carries resume.data (workflow/open-telemetry/trace-context.ts): user payloads. Add mapSpanEvents: every event's attributes go through the same filter as span attributes. Credential keys are removed always, content keys unless captureContent, and exception.* (type, message, stacktrace) is kept, as the README promises for exception events. Event names and times stay. The processor maps events when the span ends, holds them with deferred llm spans, and passes them to copySpan, which takes an optional events argument (default: the span's own). The span other processors hold is not modified. guardrail.chunk.text, suspension.data and resume.data join CONTENT_EXACT so the value type does not matter. Tests: - jest, synthetic spans: guardrail.stream.process, workflow.suspended, workflow.resumed, an exception event and an event with credential keys; content attributes gone with captureContent off and kept with it on, names/times/exception attributes kept, credentials gone either way, and the original events not mutated. - jest, real VoltAgent: streamText with a pass-through streaming output guardrail plus a workflow that suspends and resumes. A recording processor in the same array sees the content in the events (control); the exported copy does not, unless captureContent is on. - jest: the failing-tool exception event keeps exception.type, .message and .stacktrace. - contract: new `events` journey in run_fixture.mjs (same guardrail and workflow), checked on the wire through the shared harness Receiver, off and on, parametrized over TRACEAI_NODE_MATRIX. Installed-review finding M1. approved: Nikhil 2026-10-03 blanket Refs: TH-8239
…ounts ROOT_ROLLED_UP_OPERATIONS listed generateObject and streamObject, and the README said reconciliation applies to such llm spans. @voltagent/core 2.11.0 opens an llm span only for generateText, streamText, generateTitle, a model-resolution failure and runInternalGenerateText (agent.ts createLLMSpan call sites); generateObject and streamObject (deprecated) record usage on the agent root only. Narrow the set to generateText and streamText, list the call sites in its comment, and fix the README sentence (M3). README Tokens gets a Limits list: the promoted total can under-count and never double-counts. - generateObject / streamObject: usage only as voltagent.usage.* on the agent span; the collector's token columns show 0 (M3). - Tool routing with a provider tool: executeProviderToolViaCallTool -> runInternalGenerateText opens a second successful llm:generateText span in the same operation, so nothing is reconciled and the main span keeps last-step usage; the internal call is counted once (N2). - A flush or timeout between an llm span's end and its root's end. Tests: - mapping: a generateObject/streamObject-labelled llm span is not reconciled; generateText/streamText still are (fails on the old set). - mapping: main call plus an internal provider-tool generateText span -> no reconciliation, trace total below root + internal (pins the documented under-count; passes before and after). - processor, real agent: generateObject exports no llm span, no promoted tokens, and voltagent.usage.* on the agent span (pins the README statement; passes before and after). Installed-review findings N2 and M3. approved: Nikhil 2026-10-03 blanket Refs: TH-8239
The README listed Node 20.20.2, 22.23.3 and 26.8.1 as tested, and the contract module said TRACEAI_NODE_MATRIX runs every journey. It parametrizes only the tests that take a `node` argument: the tool journey, the span-event journey and the ESM/CJS import check. jest and the other contract tests (content opt-in, subagent, burst, collector down, packed tarball) run on NODE_BINARY / the PATH node, 26.8.1 here. State that in the Tested versions table, the Development command and the contract module docstring. Also update "Not covered": the tests now run one workflow that suspends and resumes (span events, CHAIN kind on the root); step spans are still not asserted. approved: Nikhil 2026-10-03 blanket Refs: TH-8239
…l decision metadata by default verify-r2 (t_cee7d90e) V1, P2/AC-07: VoltAgent 2.11.0 finalizeScorerSpan stores combineEvalMetadata(...) under eval.scorer.metadata, which holds the run's verbatim input and output; it was exported with captureContent off. eval.expected (the reference answer) and guardrail.result.metadata (a guardrail decision's payload) are content of the same kind. All three are now dropped unless captureContent; eval.input / eval.output already were. Decision (approved: Nikhil 2026-10-03 blanket): guardrail.message and the guardrail.stream.abort guardrail.reason stay. Built-in guardrails write fixed messages; a custom guardrail's message is exported as written (documented), as are suspension/cancellation reasons and workspace.fs path/pattern/cwd. README: evals ARE exported (was "not exported"); key-based secret filter note for JSON values such as llm.provider_options. RED before the fix: 2 failed / 37 passed in mapping.test.ts (new eval and guardrail test, key classification). GREEN: jest 58 passed, contract 14. approved: Nikhil 2026-10-03 blanket Refs: TH-8239
…s on VoltAgent 2.11.0 writes no `input` / `output` on llm spans: the prompt is only in `llm.messages` (agent.ts buildLLMSpanAttributes), and the response text is never recorded on the llm span (createLLMSpanFinalizer sets usage, finish reason and status only). fi-collector fills a span's input/output columns from input.value / output.value only, so with captureContent: true every LLM span in Observe showed no input or output (live run against claude-haiku-5-5: llm:generateText in=0B out=0B while the agent and tool spans had content). With captureContent on: - llm spans use `llm.messages` as input.value (application/json); - when the operation has exactly one successful llm:generateText / llm:streamText span (the same rule usage reconciliation uses), that span gets the agent span's `output` as output.value. Content off is unchanged. Live re-run: llm:generateText in=201B out=47B, llm:streamText in=220B out=17B; content-off run still exports no prompt, answer, instructions or tool text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l object
VoltAgent records only `modelId` for an AI SDK model object (agent.ts
getModelName) and sets llm.provider only from "provider/model" strings,
so agents built with openai("..."), anthropic("..."), google("...") etc.
exported no provider.
When VoltAgent set none, the processor now looks the agent up by the
span's entity.id in VoltAgent's agent registry (the AgentRegistry
singleton on globalThis.___voltagent_agent_registry, filled by
new VoltAgent({ agents })), finds the model object (including fallback
lists) whose modelId matches the span, and maps its provider id:
openai.* -> openai, anthropic.* -> anthropic, google.generative-ai ->
google, google.vertex.* / vertex.* -> vertexai, azure.* -> azure,
amazon-bedrock -> aws, mistral.* -> mistralai, groq, xai, deepseek,
cohere, cerebras, togetherai -> together, fireworks, ollama, huggingface.
Unknown ids (an OpenAI-compatible provider's own name, the gateway) stay
unset. VoltAgent's own llm.provider still wins.
Live: @ai-sdk/anthropic (claude-haiku-4-5-20251001 via CLIProxy) ->
provider anthropic, cost 0.001729; @ai-sdk/google (gemini-3.5-flash) ->
provider google, cost 0.0012585 in ClickHouse and the Observe UI.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The catalog rows for all TH-8103 integrations now live in one PR (docs/th-8103-integration-catalog), so this PR no longer conflicts with its siblings in README.md and can merge in any order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… paths (TH-8239) Re-verification 2026-10-16 against local Future AGI with the real @voltagent/core 2.11.0 (latest on npm, same as the pin), @traceai/fi-core 1.0.0 from npm and live models (api.openai.com gpt-4.1-nano and an OpenAI-compatible endpoint, both through @ai-sdk/openai). Privacy default (TH-8103 coordinator decision 2026-10-16): follow the traceAI TraceConfig convention instead of the package's own captureContent switch. Content is exported by default (fi-core DEFAULT_HIDE_* = false); FI_HIDE_INPUTS / FI_HIDE_OUTPUTS (and FI_HIDE_INPUT_MESSAGES / _OUTPUT_MESSAGES / _INPUT_TEXT / _OUTPUT_TEXT) or traceConfig hide it, with fi-core's parsing (only "true" hides) and precedence (option > env > default). The env vars apply even when no traceConfig is passed. VoltAgent content keys are classified by side (input, output, both); hideInputText / hideOutputText keep message roles and redact the text. Passing the removed captureContent option logs a diag warning. Never exported, in either mode (the real run had exported all three with Rick's content-off default): - stack traces: error.stack and exception.stacktrace are dropped, and V8 frames / Python tracebacks are cut from every exported string, including stack traces inside JSON (a failed tool's error object in llm.messages / agent.messages); - local paths: new privacy.ts rewrites the cwd to [cwd], the home dir to ~, other absolute local paths, file URLs and Windows paths to [path] in every exported string (JSON-aware, re-serialized only on change); - in error text (status message, error.message, exception.message, eval.scorer.error_message, guardrail/suspension/cancellation reasons): emails and this machine's user and host names. With inputs or outputs hidden that text is dropped and an ERROR status carries the error type. copySpan now takes the filtered events and the exported status explicitly (P3 follow-up: no unfiltered default). copyOperationOutput reads the mapped root, so a hidden output never reaches the llm span. Tests: jest 108 passed on Node 26.8.1 and 20.20.2 (new privacy.test.ts, TraceConfig env/precedence, stack/path/error-text cases); contract 14 passed across Node 20.20.2 / 22.18.0 / 26.8.1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…H-8239) new VoltAgent() keeps the process running after forceFlush() (it installs signal handlers for a graceful shutdown); a script that only flushes never exits. Observed with @voltagent/core 2.11.0 on Node 26.8.1: shutdown() flushes every processor and the process exits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Preserve every existing dev lock resolution with pnpm 10.30.1. Disable only the optional development registry in offline tests and add substantive package/exporter CI without forceExit or ignored failures. This local candidate remains held: upstream VoltAgent workflow resume leaks a referenced interval even after observability.shutdown; 108 assertions pass but the process does not exit. Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com> # Conflicts: # typescript/pnpm-lock.yaml
Document the 2.11.0 limitation and full public application shutdown. Keep every resume assertion in a dedicated Jest invocation that propagates its real result before exiting; ordinary suites retain open-handle checks. Limit the contract fixture exit to its resumed-workflow journey. Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Mechanically union the held VoltAgent package closure with dev, then normalize using pnpm 10.30.1 and verify frozen installation. Preserve every existing dev importer, package and snapshot value unchanged. Co-Authored-By: Claude Code <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
@traceai/voltagent: a span processor wired through VoltAgent's nativeObservabilityConfig.spanProcessors. It exports a mapped copy through the Future AGI provider, without patchingAgentor altering spans delivered to VoltOps and other processors.voltagent.*.@voltagent/coreas a peer dependency rather than bundling it.dev, and verify the lockfile with pinned pnpm 10.30.1. Preserve every existing dev importer, package and snapshot resolution unchanged; add the held VoltAgent dependency closure without upgrading other consumers.Known upstream issue — resumed workflows in @voltagent/core 2.11.0
A completed resumed workflow leaves a referenced 500 ms polling interval running, even after
await app.shutdown(). A Node process that resumes workflows may not exit on its own. An independent upstream-only reproduction, importing no traceAI modules, exhibits the same leak: traceAI is not its cause. Ordinary, non-resumed workflows are unaffected by this specific leak.Use
.toWorkflow()once, register and reuse that workflow instance, and call full publicawait app.shutdown(); observability-only shutdown is not full application cleanup. No supported public API in 2.11.0 disposes of the orphaned interval after the resumed step completes. For short-lived scripts only, await application shutdown and flush/shut down exporters you own, then callprocess.exit(0)or a nonzero status on failure. Do not manipulate private timers or force-exit a long-running server.The README documents this limitation. All resume assertions remain: 106 ordinary tests run with open-handle detection and no forced exit, followed by the two resume-event tests in a dedicated Jest invocation. That isolated runner awaits Jest's assertions and teardown, propagates its actual success/failure status, then explicitly exits solely for the upstream leak. Only the resume journey of the contract fixture uses the same documented short-lived-script exit.
Exact-head verification
Head:
d0c92e232fe6311fdf07240abcea079a298edf65.@traceai/voltagent0.1.0 and its exact-head traceAI dependencies, real@voltagent/core2.11.0,ai6.0.309,@ai-sdk/openai3.0.130, andzod4.3.6 on Node 26.8.1.gemini-3.1-flash-litethrough an OpenAI-compatible endpoint; the SDK provider field isopenai. Controlled failing workflow steps cover ERROR export without additional model spend.traceai215-final-d0c92e23. Span-ID and parent-ID parity, model, positive usage counted once, content visibility/hiding, error status, and credential/stack/path scans all pass. Ordinary processes exit naturally; resumed scripts use the documented exit only after full shutdown.Representative Cloud trace IDs (32-character OpenTelemetry form):
1d3128e7b8de3fdfb54f63ac1434c99a,45767ae5fc069448a948165a40c70158e613bce752045d70975f1a7147b13802,8809e4ccccc99b7bb367a96add72d20db20ce1eb957609abc1cfba3c0d06013f,58d1367079db5b711111cbe096e28a00,a440616ea56bc4a573c8112bdc3b7a3bb021b2076ccc584a671fde7461a2b74d,b59c4bf4bbe84827717c58399881affa,7b7baa2d247da029517078d1aaf4fefc🤖 Generated with Claude Code