Repository navigation
Conversation
…ared attachment manager - fi_instrumentation.attachments.media_capture: shared, public-API-only helpers (preflight before any read, strict base64, deny-wins local TraceConfig folding, per-manager span/position registry, consumed speech audio). - traceai_openai: chat/responses images and files, computer_call_output screenshots, image generation (non-stream and streamed terminal event), images.generate, buffered and consumed speech, transcription/translation bytes. instrument(attachments=...) (or auto's attachment_manager). - traceai_anthropic: image/document blocks and tool_result media with causal computer-screenshot classification. - traceai_groq: opt-in audio spans for speech and transcription/translation. - Telemetry-only projection replaces admitted media with fi-attachment URIs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…xported telemetry Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ackages are not installed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d thread order Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 11, 2026
feat(python): MCP, agent SDK, smolagents and Firecrawl tool media (attachments C01/C03/C05/C06)
#693
Draft
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of TH-2379 (Linear).
What
Python provider attachments (design WP M01, M03, M14), stacked on the Python attachment core (#676). Mirrors the TypeScript OpenAI/Anthropic lane (#682).
fi_instrumentation.attachments.media_capture(public manager API only:preflight+ticket.attach/start_stream). It is not exported fromattachments.__all__.too_large/queue_fulldenial is re-preflighted once at the exact decoded size.TraceConfigfolding: a hide flag for the direction, orpii_redaction, setscapture_content=False, so the core cannot relax it (not even withallow_unredacted_media_with_text_pii).hide_input_imagesdenies input images and screenshots.hide_error_detailsdeniesis_errortool results.ConsumedAudioforwards only chunks the application already read. It reserves 4 MiB and steps down (2 MiB, 1 MiB, 512 KiB, 256 KiB) when the server cap is lower. Partial reads, abort and an unknown container upload nothing.traceai_openai:OpenAIInstrumentor().instrument(attachments=manager)(auto'sattachment_manager=is accepted too). Capture covers:image_urlandfileparts;input_image/input_file;computer_call_outputscreenshots (purposecomputer_screenshot, roletool,tool_call_id=call_id);image_generation_calloutput, non-streamed and from the consumed terminalresponse.completedevent (partial images never);images.generateb64_json;iter_bytes,read()orstream_to_file(the SDK's ownstream_to_fileis bound to the proxy, so its reads are observed). The MIME comes from container magic and is cross-checked withresponse_format, so a mismatch uploads nothing;capture_openai_attachments(...),project_openai_media(...),capture_openai_speech,observe_openai_speech,capture_openai_audio_input,openai_attachment_capabilities.traceai_anthropic: base64 image and PDF document blocks, and one level oftool_resultcontent. A tool-result image is acomputer_screenshotonly when a typedcomputer_YYYYMMDDtool is declared AND an assistanttool_useof that tool has the result's id; otherwise it istool_result. Message and block indices keep causal order for the filmstrip. An invalidtool_use_idfails closed. Public:capture_anthropic_attachments(...),project_anthropic_media(...),anthropic_attachment_capabilities.traceai_groq: with an enabled manager only, new spans foraudio.speech(buffered body),audio.transcriptionsandaudio.translations(bytes inputs), through the same helper. Without a manager nothing is wrapped.fi-attachment:<occurrence_id>in a telemetry-only copy. The SDK request and response are never mutated.Explicitly unsupported (fail closed): remote URLs, provider file IDs, partial images,
images.edit/ variations, open file objects, speechstream_format="sse", aac/pcm/mulaw speech,beta.messages, generated images from Anthropic, Groq streamed speech and Groq path inputs.Evidence (private GCP VM, loopback only; outbound provider traffic is a local fake server, the F01 server is TLS loopback)
python/tests/attachments_providers: real SDKs on flooropenai==1.69.0 anthropic==0.41.0 groq==0.9.0(Python 3.10): 63 passed, 2 skipped (the skips are Groq speech, which groq 0.9.0 does not have). On latestopenai==3.49.0 anthropic==2.8.0 groq==1.7.0(Python 3.12): 65 passed. Coverage: sync and async, streaming and non-streaming, privacy negative controls (each hide flag, PII, audio-only server policy, unsampled, env disable, disabled manager), dedup, exit probes (enabled, disabled, after fork), and the realtraceai.init(attachment_config=enabled)auto path.feat/attachments-python-coreon the same VM, identical counts: openai 1820/1844 passed, anthropic 162/246, groq 60/60, core + auto 435/435. One openai failure,test_parallel_readme_responses_snippet, fails identically on base (pre-existing).stream_to_fileunobserved, partial read finishing, any tool treated as computer, Groq speech dropped,hide_input_imagesignored.Not in this PR
traceaiauto still passesattachment_manager=; renaming it toattachments=(as TS feat(auto): attachments in @traceai/auto (A02 ownership, A03 wire JSON/FormData, A04 consumed audio) #687 did) is left to the auto owner. Both are accepted here.🤖 Generated with Claude Code