Add manual distributed tracing propagation for LLM Observability - #12402
Draft
ncybul wants to merge 1 commit into
Draft
Add manual distributed tracing propagation for LLM Observability#12402ncybul wants to merge 1 commit into
ncybul wants to merge 1 commit into
Conversation
Adds LLMObs.injectDistributedHeaders/activateDistributedHeaders so applications can manually propagate LLMObs context (ml_app, session_id, agent attribution) across boundaries automatic instrumentation doesn't cover, e.g. an SQS worker reading its own message attributes. Standard APM trace context rides the normal Propagators.defaultPropagator(); LLMObs tags piggyback on the existing x-datadog-tags carrier entry to stay wire-compatible with dd-trace-py/js/go. Claude session: `15543c2c-2abe-408e-b16e-05ddbe972287` Resume: `claude --resume 15543c2c-2abe-408e-b16e-05ddbe972287`
Contributor
|
🎯 Code Coverage (details) 🔗 Commit SHA: ce20e17 | Docs | View more details | Give us feedback! |
Contributor
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an explicit, manual API for propagating LLM Observability context across process boundaries that automatic instrumentation doesn't cover — e.g. an SQS worker reading its own message attributes. Mirrors dd-trace-py's
inject_distributed_headers/activate_distributed_headersmodel (Python/Node/Go already support this; Java did not).Design
TracingIteratorbatch-context-reuse bug in APM's automatic SQS instrumentation rather than fixing it — that's a separate, not-yet-filed issue.Propagators.defaultPropagator(). LLMObs-specific tags (_dd.p.llmobs_ml_app,_dd.p.llmobs_sid,_dd.p.llmobs_pagent_span_id,_dd.p.llmobs_pagent_name) are appended as extrakey=valuepairs onto the samex-datadog-tagscarrier entry the default propagator already writes, rather than registered as a newConcern/Propagatoror added toPropagationTags/PTagsFactoryin dd-trace-core. This keeps the change self-contained toagent-llmobsand preserves wire compatibility with dd-trace-py/js/go, which nest the same_dd.p.llmobs_*tags insidex-datadog-tags/tracestatet.*.LLMObs.injectDistributedHeaders/activateDistributedHeaders(dd-trace-api), delegating to a newLLMObs.LLMObsPropagatorinterface, following the same static-field delegation pattern asSPAN_FACTORY/EVAL_PROCESSOR/FEEDBACK_PROCESSOR. Real implementation isDDLLMObsPropagator(agent-llmobs), wired in viaLLMObsInternal.setPropagator(...); defaults to a no-op when the agent isn't active.Known scope limitations
ml_apprides on the wire for cross-SDK compatibility but is not yet auto-applied to spans started afteractivateDistributedHeaders— callers must still passml_appexplicitly when starting a span.agent_versionis not propagated cross-process (no SDK sends it today).Test plan
DDLLMObsPropagatorTest(JUnit 5) — round-trips a plainMap<String, String>carrier (the shape an SQS message-attribute map would take): trace-id join across inject/activate, session_id/agent-attribution propagation, no-op on missing trace context, null-arg validation, ml_app always present even without session/attribution.:dd-java-agent:agent-llmobs:testpasses (no regressions).spotlessCheckpasses.Claude session:
15543c2c-2abe-408e-b16e-05ddbe972287Resume:
claude --resume 15543c2c-2abe-408e-b16e-05ddbe972287