feat(ai): add agent hooks and OpenTelemetry tracing - #1536
Open
chaojixinren wants to merge 1 commit into
Open
Conversation
Introduce immutable lifecycle observations for interactions, iterations, stages, model calls, and tool calls while allowing hooks to derive context for nested work. Preserve detached Agent cancellation semantics, propagate inbound W3C trace context through Agent and MCP calls, and expose trace IDs on SSE responses. Add configurable OTLP tracing with GenAI semantic attributes, opt-in content capture, logging hooks, graceful interaction draining, CORS trace headers, and regression coverage for lifecycle ordering, panic isolation, fallback metadata, concurrency, and content immutability. Constraint: Hooks may observe state and derive context but must not mutate Agent execution data. Constraint: SSE disconnect cancellation must remain detached from the running interaction. Rejected: Pass mutable Agent objects to hooks | violates the observational hook contract. Confidence: high Scope-risk: moderate Directive: Keep tracing content capture opt-in and register content demand on the tracing hook, not the logging hook. Tested: go test -count=1 ./...; integration-tag full suite; hooks and Agent race tests; go vet; local Jaeger OTLP E2E; local Langfuse 4.11.0 OTLP/HTTP ingestion and v4 Observations verification. Not-tested: Hosted Langfuse cloud and external DashScope/Milvus E2E, which are outside this issue and require credentials.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces a config-driven, telemetry-agnostic agent hooks system in
ai/, withlogging and OpenTelemetry (OTLP) tracing as the first observers. It instruments
the full ReAct lifecycle — interaction, iteration, stage, model call, and tool call —
without coupling the strategy code to any specific backend.
Motivation
Resolves #1525. There was previously no consistent way to observe agent execution or trace
a request end-to-end across the Agent ↔ MCP boundary, making agent behavior hard to debug
and spans impossible to correlate.
What changed
ai/component/hooks/): immutable lifecycle events(
interaction / iteration / stage / model_call / tool_call×start / end), read-onlyStatesnapshots, per-event registration, and panic isolation per hook.context.Contextfor nested work; exactly oneDerivesContextregistration is accepted (a Go context carries one span lineage — fan outvia a Collector instead). Contexts returned by plain observational hooks are ignored.
(
gen_ai.operation.name,gen_ai.request.model,gen_ai.provider.name,gen_ai.conversation.id,gen_ai.tool.name,gen_ai.tool.call.id,gen_ai.input/output.messages,gen_ai.usage.*,agent.fallback.*,error.type, …).Content is serialized lazily only when the span
IsRecording(), and content capture isopt-in (
capture_content: nonedefault;truncated≤ 4096 bytes;full).traceparent/tracestate/baggageare honored inbound,propagated to MCP HTTP calls, and the active trace ID is returned on SSE responses via
X-Trace-ID(CORS-exposed).FallbackReason), writtento both the model-call span and the stage span, with correct
Evidencetext; tool failuresare recorded as
error.type+agent.degradedwithout faking agen_ai.tool.call.result.context.Canceledpropagates cleanly (onlyDeadlineExceededis a timeout); SSE disconnect cancellation stays detached from the running interaction.
type: hookscomponent withlogging/tracingblocks, JSON schemavalidation, and standard OTel env vars for endpoint and credentials.
Design constraints
Agent execution data.
nonefor credential/PII safety; payloads are not serializedon the hot path unless a matching hook opts in.
"*"for all).Agent Hooks + OTel Tracing — Completed Test Checklist
1. Unit Tests
go test -count=1 ./...— all passed (hooks, Agent, server engine, MCP tools, runtime, etc.)2. Integration Tests
go test -tags=integration -count=1 ./...— all passed3. Race / Static Analysis
go test -race ./component/hooks/... ./component/agent/...— passedgo vet ./...— passed4. E2E: Jaeger (OTLP)
f6477cd5b8d5a5cce9ae07a0ad1d84705. E2E: Langfuse (Docker 4.11.0)
AGENT invoke_agent→GENERATION chat qwen-max→TOOL lookup_serviceevents_full: model input/output, 7/5/12 tokens, session ID, tool call IDfdc809bacdcdbf5b7cd7798ba8f7c1cb6. Performance Benchmarks (0 allocs)
BenchmarkDisabledHookFastPathBenchmarkEmptyManagerFastPathBenchmarkHookContentDisabledBenchmarkHookContentLoggingOnly7. Cleanup
7bbcab1TestMultiTurnConversationtherefore shows 0/14