[opentelemetry-instrumentation-genai-langchain] Record gen_ai.response.model on streamed spans - #505
Merged
lmolkova merged 7 commits intoSep 2, 2026
Conversation
Both attributes were absent from every streamed chat span and present on every non-streamed one. on_llm_end read them from LLMResult.llm_output, which LangChain never populates for a streamed call: it assembles the final LLMResult from the merged chunks alone. The values are still on the message's response_metadata, which LangChain fills with the union of the generation's generation_info and whatever the provider wrote onto the message, so reading that covers langchain-openai (which reports the model through generation_info) and langchain-anthropic (which writes it onto the message) alike. The precedence chain -- generation, then llm_output, then the Responses API served-model header -- moves into one helper so it is stated in one place instead of spread across the callback. gen_ai.response.id stays absent for streamed OpenAI and Anthropic calls. Both providers send a response id on every chunk, but their LangChain adapters drop it on the streaming path; only the OpenAI Responses API records one. message.id is not a substitute, since LangChain fills it with its own run id.
sfc-gh-zeningchen
force-pushed
the
langchain-streamed-response-model
branch
from
August 31, 2026 02:34
105eed3 to
098310a
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes LangChain streamed chat spans missing gen_ai.response.model (and conditionally gen_ai.response.id) by extracting those fields from streamed-generation metadata and centralizing precedence logic.
Changes:
- Add utilities to extract
(response_model, response_id)from a streamed generation’smessage.response_metadataand to resolve final precedence between generation,llm_output, and served-model headers. - Update the LangChain callback handler to use the new resolver so streamed and non-streamed paths consistently populate
response_model_name/response_idwhen available. - Add regression tests covering streamed behavior via both unit-level
on_llm_endinputs and a real.stream()flow with a patched provider stream.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_llm_call.py | Adds an end-to-end streamed ChatOpenAI test asserting gen_ai.response.model is present. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_callback_handler.py | Adds focused unit tests for streamed on_llm_end extraction and precedence rules. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/utils.py | Introduces helper functions to extract/resolve response model/id across streamed and non-streamed results. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/callback_handler.py | Switches on_llm_end to use the new resolver and pulls fields from streamed generation metadata. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/.changelog/505.fixed | Adds a changelog fragment describing the streamed-span attribute fix. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
gen_ai.response.id remains unset for streamed Chat Completions and Anthropic calls, since those adapters do not surface the provider's id. The PR description and commit message cover where it is recorded.
Pull request dashboard statusMerged · refreshed 2026-09-02 00:38 UTC Status above doesn't look right?
|
The InferenceStreamingScenario declared the missing attribute as an expected violation, and the conformance harness fails when weaver stops reporting one, so recording the model turned the scenario red. gen_ai.response.id keeps its entry: the langchain-openai streaming path does not surface the id the provider sends on every chunk.
sfc-gh-zeningchen
force-pushed
the
langchain-streamed-response-model
branch
from
August 31, 2026 06:13
dc32e87 to
b2277f0
Compare
lmolkova
approved these changes
Sep 1, 2026
LangChain merges a generation's generation_info into the message's response_metadata and prefers the message where the two disagree, so reading response_metadata covered every generation that went through .stream() or _generate_with_cache(). A generation dispatched without that merge reported nothing. Read generation_info after response_metadata, keeping the same precedence LangChain applies. Add a test for a generation that carries the model only in generation_info, one that pins response_metadata winning when the two disagree, and an .astream() counterpart to the streamed end-to-end test.
…sponse-model # Conflicts: # instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_callback_handler.py
OpenAIResponsesInputImageScenario declared the missing attribute as an expected violation, and the harness fails when weaver stops reporting one. The Responses API records an id in the message's response_metadata, so recording the id from there turned the scenario red. The streaming scenario keeps its entry, since the Chat Completions streaming path still surfaces no id.
github-project-automation
Bot
moved this to Done
in GenAI Semantic Conventions and Instrumentation libraries
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
gen_ai.response.modelandgen_ai.response.idwere absent from everystreamed
chatspan and present on every non-streamed one.on_llm_endread both fromLLMResult.llm_output. LangChain never populatesthat for a streamed call: it assembles the final
LLMResultfrom the mergedchunks alone. The values are still on the message's
response_metadata, whichLangChain fills with the union of the generation's
generation_infoandwhatever the provider wrote onto the message, so reading that covers
langchain-openai (which reports the model through
generation_info) andlangchain-anthropic (which writes it onto the message at
message_start).The precedence chain, generation then
llm_outputthen the Responses APIserved-model header, moves into one helper so it lives in one place instead of
being spread across the callback.
gen_ai.response.idstays absent for streamed Chat Completions and Anthropiccalls. Both providers send a response id on every chunk, but their LangChain
adapters do not surface it on the streaming path. Only the OpenAI Responses API
records one, and that path is covered here.
message.idis not a substitutebecause LangChain fills it with its own run id.
Fixes #503
Type of change
How has this been tested?
TestOnLlmEndStreamedResponseintests/test_callback_handler.pycoversthe model and id read from a streamed generation,
llm_outputand theserved-model header keeping precedence, both attributes staying unset when no
source reports them, and
message.idnever being used as the response id.test_chat_openai_streamed_response_modelintests/test_llm_call.pydrives a real
.stream(). The model is reported only ingeneration_infothere, and LangChain is what copies it into
response_metadata, so ahand-built
LLMResultwould skip that step and prove nothing.Checklist