Skip to content

fix(integrations): record output for with_raw_response and with_streaming_response - #8039

Open
chrikrah wants to merge 1 commit into
getsentry:masterfrom
chrikrah:fix/integrations-raw-response-output
Open

chrikrah wants to merge 1 commit into
getsentry:masterfrom
chrikrah:fix/integrations-raw-response-output

Conversation

@chrikrah

Copy link
Copy Markdown

fix(integrations): record output for with_raw_response and with_streaming_response

Description

sentry_sdk/integrations/anthropic.py:794 requires hasattr(result, "content"). A .with_raw_response wrapper has no content, so the span ends at unknown_response on line 824. openai.py:919 passes the wrapper on too, and _set_completions_api_output_data finds no choices and records nothing. Both integration docs promise Sentry "will automatically collect information about prompts, tools, tokens, and models". The Anthropic page names messages.create as the supported function. .with_raw_response and .with_streaming_response call that same create behind an X-Stainless-Raw-Response header, so the span opens and records no output.

Before:

$ python -W ignore repro.py    # master 2ffd383, anthropic 1.13.0, openai 3.29.0
anthropic/with_raw_response        {'gen_ai.response.id': False, 'gen_ai.response.finish_reasons': False, 'gen_ai.usage.input_tokens': False, 'gen_ai.usage.output_tokens': False, 'gen_ai.response.text': False} unknown_response
anthropic/with_streaming_response  {'gen_ai.response.id': False, 'gen_ai.response.finish_reasons': False, 'gen_ai.usage.input_tokens': False, 'gen_ai.usage.output_tokens': False, 'gen_ai.response.text': False} unknown_response
openai/with_raw_response           {'gen_ai.usage.input_tokens': False, 'gen_ai.usage.output_tokens': False, 'gen_ai.response.text': False}
openai/with_streaming_response     {'gen_ai.usage.input_tokens': False, 'gen_ai.usage.output_tokens': False, 'gen_ai.response.text': False}

After:

$ python -W ignore repro.py    # this branch
anthropic/with_raw_response        {'gen_ai.response.id': True, 'gen_ai.response.finish_reasons': True, 'gen_ai.usage.input_tokens': True, 'gen_ai.usage.output_tokens': True, 'gen_ai.response.text': True}
anthropic/with_streaming_response  {'gen_ai.response.id': True, 'gen_ai.response.finish_reasons': True, 'gen_ai.usage.input_tokens': True, 'gen_ai.usage.output_tokens': True, 'gen_ai.response.text': True}
openai/with_raw_response           {'gen_ai.usage.input_tokens': True, 'gen_ai.usage.output_tokens': True, 'gen_ai.response.text': True}
openai/with_streaming_response     {'gen_ai.usage.input_tokens': True, 'gen_ai.usage.output_tokens': True, 'gen_ai.response.text': True}

parse() caches its result, so parsing an already-read body where create would costs the caller nothing. A streamed body stays unread. The span fills when the caller parses it, and closes when the caller closes the response. A wrapper the caller neither parses nor closes leaves that span unfinished. The SDK then drops the span where baseline ends it. close() finishes the span wherever the wrapper offers one, and none of the 6 new tests covers that case.

Verification

$ python -m pytest -o addopts="" tests/integrations/anthropic/ tests/integrations/openai/ -q
1290 passed, 1716 warnings in 25.90s

$ python -m pytest -o addopts="" tests/ -q -p no:randomly --ignore=tests/integrations/wsgi --ignore=tests/test_client.py
194 failed, 3083 passed, 171 skipped, 1938 warnings, 390 errors in 198.87s
# baseline 2ffd383, same command: 194 failed, 3077 passed, 171 skipped, 1928 warnings, 390 errors in 179.49s
# not run: the tests/integrations/wsgi and tests/test_client.py collections (werkzeug missing, one integration unavailable here)
$ python -m ruff format sentry_sdk/integrations/anthropic.py sentry_sdk/integrations/openai.py tests/integrations/anthropic/test_raw_response.py tests/integrations/openai/test_raw_response.py && python -m ruff check sentry_sdk/integrations/anthropic.py sentry_sdk/integrations/openai.py tests/integrations/anthropic/test_raw_response.py tests/integrations/openai/test_raw_response.py
4 files left unchanged
All checks passed!

Baseline on 2ffd383 is 1284 passed, 1706 warnings in 26.98s, so the delta is the 6 new tests. Reverting both integration files and keeping the tests gives 6 failed, 10 warnings in 0.44s.

Your #5917, after #5890, stopped wrapping responses without _iterator. It said tracing LegacyAPIResponse was intentionally out of scope. This change keeps that guard and picks up only the deferral. It fills the span for the wrapper without touching _iterator or reading an unread body.

Fixes #7948

@alexander-alderman-webb Your #7948 reply put this on PY-2950. #7891 is the separate parse() gap on a plain create, which I have not touched here. If the openai work has started, I would close this instead. Otherwise it is ready to land ahead of PY-2950. Does that timing suit you?

…ming_response

Signed-off-by: Christopher Krah <48338417+chrikrah@users.noreply.github.com>
@chrikrah
chrikrah requested a review from a team as a code owner October 14, 2026 20:56

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenAI and Anthropic integrations record no output or usage for with_raw_response and with_streaming_response calls

1 participant