Repository navigation
Conversation
Provide deny-wins admission, bounded Bytes/Blob transfers, explicit correlation and provider lifecycle integration. Keep unqualified retrieval and streaming paths closed. Co-Authored-By: Claude Code <noreply@anthropic.com>
Playwright 1.56.1 does not support the Ubuntu 26.04 image now selected by ubuntu-latest; retain the qualified browser on Ubuntu 24.04. Co-Authored-By: Claude Code <noreply@anthropic.com>
Keep ready and flush deadlines referenced only while explicitly awaited. Background requests and retries remain unreferenced. Reproduce prior early CLI exit with a real TLS child process that has no artificial keepalive. Co-Authored-By: Claude Code <noreply@anthropic.com>
This was referenced Nov 11, 2026
feat(openai,anthropic): capture typed images, documents and computer screenshots as attachments
#682
Draft
…lifecycle per the SDK design (WIP) Implements sdk-streaming-design sections 0-9 in fi-core attachments: preflight tickets, PCM recording writer with WAV segmentation and prefix-monotonic manifests, consumed-stream uploads, completed-recording import seam, owned recording file scope and tiers, capability and session renewal by replay, the disabled null object, the one versioned wire adapter (idempotency keys, status table, abort, manifest, import routes) and hand validators for every F01 shape. The legacy four-method transport keeps working through an adapter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Type FinalizationRegistry locally so consumers compiling fi-core source with an ES2020 lib still build. Hold a fallback-order segment's complete until a later segment is sealed or the recording finishes, so the last capability can PUT the final manifest (BR-7). Add T-ST-16 fallback, T-CS-05 forced-GC, T-TR-04 proxy-canary and T-PF-03 preflight replay coverage. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…atus and worker wake A recording-only workload (tryWrite + flush, as Pipecat-style adapters drive it) never reached maybeRenew(), which ran only from the five admission entry points. A recording that outlived the server session lost its tail as `incomplete` gaps. Per design 3.1, renewal is now also checked from: - pump() (worker wake), - flush() and status() (enter() instead of sweep()), - AttachmentRecording.writeFrames, through a deduped host touch() that defers maybeRenew to a microtask so tryWrite does no I/O on the caller's stack. Regression tests: T-RN-09 (120 s session, 150 s recording, tryWrite + flush only: renewal, no rejected writes, 15 segments, published, no gaps) and T-RN-10 (each trigger alone renews: recording write, flush, status, worker wake). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Changes that used to stop capture silently now fail loudly: - A legacy four-method transport (including a wrapper that drops ATTACHMENT_TRANSPORT_V1) emits one DeprecationWarning FI_ATTACHMENTS_LEGACY_TRANSPORT naming what it cannot do and the fix. startRecording, WAV-tier attachOwnedRecording and importCompletedRecording are denied at the call with unsupported_version instead of buffering and uploading segments that can never be bound. Renewal is no longer retried on a wire that has no renewal route. - A response failing its F01 shape check still fails closed as unsupported_version, and now warns once (FI_ATTACHMENTS_WIRE_REJECTED) with the shape name only. - `failed` leaves AttachmentCompletion, so a leftover check is TS2367 like the retired `queued` ready state. A tsc test pins both. - ATTACHMENTS-MIGRATION.md lists every change, how it fails, and the fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds manager.deferredCorrelation(): Bytes, Blob and consumed-stream captures can be admitted before their span exists (wire spans that are emitted only after the body completes). Committed bytes stay in the owned budget, capped at half of it, and nothing is sent until target.bind(span), which re-evaluates policy and sampling. Timeout, cancel() and shutdown fail pending captures with correlation_unavailable. Owned files, recordings and imports never defer, and a span that already has a correlation keeps one reference list. startAttachmentStream now reserves min(maxBytes, negotiated single-upload cap), so servers below 4 MiB still admit consumed streams. shutdown() keeps an earlier invalidate() reason instead of overwriting it with incomplete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bind-time sampling and the bind deadline now cover all captures on a deferred target, not only those already parked at bind: - deferredBound aborts streams still open on the target and releases its outstanding ticket reservations when the bound span is unsampled or not recording (`unsampled`), or when the bind is late (`correlation_unavailable`). - AttachmentDeferredCorrelation.bind asks the host whether the deadline has passed BEFORE binding the identity. A late bind cancels the target and returns false, so tickets and open streams can no longer upload after the deadline when no other manager entry point ran in between. - Backstops: ticket commit and stream finish re-check the bound span's sampled flag before anything is enqueued. Rows T-DT-15..18 cover a stream open at bind and a ticket committed after bind on an unsampled span, a late bind with no intervening entry point, and each backstop with the primary check bypassed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of TH-2379 (Linear).
What does this PR do?
Implements the TypeScript half of the shared SDK attachments design (streaming, recordings, imports and lifecycle) in
@traceai/fi-core, on top of the opt-in attachment foundation already in this branch. Python (#676) implements the same semantics. Draft component; integrated backend/product qualification has not run.Defaults are unchanged: attachments are OFF unless explicitly enabled, creating new recordings is separately OFF, and every deny-wins privacy rule (hide flags, PII redaction without a qualified sanitizer, server policy) is evaluated before any source is read.
manager.preflight(target, {mimeType, direction, maxBytes, ...})): the full deny-wins check without touching the source. It returns a single-use ticket that reserves one descriptor and a byte budget, expires lazily (no timers), is re-checked against the current policy at commit, and can only narrow. Adapters decode or read only afterticket.admitted.startRecording): s16le PCM is cut into independently playable WAV segments on a sample-count clock (at most 30 s and 8 MiB including the 44-byte header, or the negotiated lower limits). All tracks share cut points;track_idappears only forseparate_tracks. Segments upload strictly in order with a window of 2; gaps (queue_full,bootstrap_pending,policy_denied,truncated,incomplete,source_unavailable) are recorded with exact durations. Manifests are cumulative and prefix-monotonic, validated locally against the F01 cross-field rules, and authorized only by the latest completed segment's write capability. If the server refuses that after complete (BR-7), remaining segments switch to init → content → manifest → complete.startAttachmentStream): one bounded upload of chunks the application already read. The core never pulls, clones or tees a stream. Over-cap streams discard everything (too_large); abandoned streams are swept lazily or released by a finalizer; the reference is projected only atfinish().createRecordingFileScopefrom./attachments/node, thenattachOwnedRecording): positional reads from our own handle, identity bind on (dev, ino, size), change detection before every slice. Tiers: one upload up to 50 MiB; PCM WAV above that (or with known channel roles) becomes a segmented recording built from lease ranges without materializing the file; larger encoded files aretoo_large. The residual TS path race (an intermediate directory swapped betweenlstatandopen) is documented in the source and covered by a negative test. Windows is unsupported.importCompletedRecording): resolver seam (default unavailable), submitted with the original credential and never a write capability,im-{recording}idempotency key, no URL, path or credential in the body, one status GET per call.expires_at - now <= max(60 s, 20% lifetime)) by replaying bootstrap with the same session id and nonce (bs-{sid}-{n}). The check runs from every entry point (preflight, attach, startRecording, attachOwnedRecording, importCompletedRecording), fromflush(),status()and worker wake, and from recording writes:tryWriteschedules a deduped check on a microtask, so it does no I/O on the caller's stack. A recording-only workload (tryWrite+flush, as Pipecat-style adapters drive it) therefore renews on its own; upload capabilities renew by replaying the byte-identical initialize (up-{occurrence}). A server that does not mint a fresh capability yieldsexpiredand an abort, never a new upload id.createAttachmentHttpTransportV1): one module for every route, key and status mapping. 404 →not_found, 409 →conflict,Error.coderefines only within its HTTP class, Retry-After capped at 5 s, separate control and content deadlines, a best-effortabortafter any post-initialize failure, strict JSON (duplicate members rejected), 64 KiB success and 4 KiB error bodies.validAttachmentWirehand-validates every F01 shape the SDK sends or parses.createNodeAttachmentManagerreturnsDisabledAttachmentManagerwhen OFF. It has the full public surface and starts no timer, socket, process listener, finalizer or bootstrap, and writes no span attribute. Enabled background timers and sockets are unref'd;shutdownfinishes open recordings within the budget and returns within the timeout even when a transport ignores its abort signal.The legacy four-method
AttachmentTransportkeeps working for single uploads through an adapter. Sincef0a56b37the adapter is loud: oneDeprecationWarning(FI_ATTACHMENTS_LEGACY_TRANSPORT) per process, andstartRecording, WAV-tierattachOwnedRecordingandimportCompletedRecordingreturnunsupported_versionat the call instead of uploading segments that can never be bound.Behaviour changes for stacked adapter lanes (required by the design)
All of these are listed, with how each one now fails and the fix, in
typescript/packages/fi-core/ATTACHMENTS-MIGRATION.md, which ships in the package.status().stateis"ready"after bootstrap, never"queued". Checks for!== "queued"must change:traceai_auto/src/wire/attachments.ts:60,traceai_agent_core/src/attachments.ts:63,traceai_cursor_sdk/src/attachments.ts:32(andtraceai_auto/test/attachments.test.ts:40).f054a68e,status(),flush(), worker wake and recording writes can also trigger that renewal, not only admission calls.failedis retired (incomplete/unsupported_version) and removed fromAttachmentCompletion, so a leftover=== "failed"is TS2367 like"queued".manager.correlation(span)no longer stamps attributes.reason) isunsupported_versionand warns once (FI_ATTACHMENTS_WIRE_REJECTED) with the shape name only.createAttachmentHttpTransport(...)that does not forward[ATTACHMENT_TRANSPORT_V1]becomes a legacy transport and triggers the deprecation warning. The feat(coding-agents): opt-in typed image attachments for coding-agent integrations #680agentAttachmentRuntimeguard is that shape today.Additions in
01b3327a(cross-lane asks; additive, no breaking API)manager.deferredCorrelation({ bindTimeoutMs })returns anAttachmentDeferredCorrelation, which is a subclass ofAttachmentCorrelation). Some wire spans are emitted only after the body completes, such as wire-only FormData (OpenAI transcriptions) and wire-only incremental speech (Deepgram speak).preflight,attachandstartAttachmentStreamnow admitBytes,Bloband consumed-stream captures against a target that is not bound yet, using the full deny-wins check.Blobis retained, not read. Nothing is sent.target.bind(span)checks policy, the environment kill switch and sampling again. Only then does it build the request (with the bound parent and the current policy version), queue the upload and project the reference onto that span.target.cancel()is called or the manager shuts down, pending captures endcorrelation_unavailableand upload nothing.min(maxBytes, negotiated single-upload cap). On a server withmax_upload_bytesbelow 4 MiB, the 4 MiB stream reservations in ElevenLabs (feat(voice): TS LiveKit Agents recording, Pipecat recording link, ElevenLabs speech attachments #685), provider-media (feat(providers): typed provider media capture for TypeScript Gemini, Bedrock, compatible chat and speech #683) and auto (feat(auto): attachments in @traceai/auto (A02 ownership, A03 wire JSON/FormData, A04 consumed audio) #687) are admitted again, and a body over the effective cap is discarded whole (too_large).preflightkeeps exact semantics.shutdown()no longer overwrites a priorinvalidate(reason)withincomplete.ATTACHMENTS-MIGRATION.mdunder "Additions (non-breaking)".Qualified on a temporary private GCP VM (deleted afterwards), loopback only, synthetic data:
test:attachmentsscript (8 files, including the newattachments-deferred.test.cjs) passed 134/134 on Node 24.11.1, 18.20.8 and 22.23.4. The new file passed 17/17 in 5 consecutive runs. Rows: T-DT-01..14, T-SR-01..02, T-SD-01.node:imports.f0a56b37build fails 16 of 17 rows. The base's own suite passes 117/117 on the same VM.Additions in
930876ea(review fixes for the deferred target)unsampledwhen the bound span is unsampled or not recording. Their reserved bytes go back to the owned budget at bind, and the ticket's source is never read.bind()fails closed.AttachmentDeferredCorrelation.bindasks the manager whether the bind deadline has passed before it binds the identity. If it has,bindreturnsfalse, binds nothing and cancels the target, so open streams and outstanding tickets endcorrelation_unavailable. This holds even when no other manager entry point ran between the deadline and the bind.deferredBoundalso refuses a late target's open streams and tickets if the pre-bind check is bypassed.@internalhost hookdeferredExpired(target)(optional onAttachmentDeferredHost). No public API change.target.bind(span)returningfalsenow also means "too late, nothing was bound". Callers must not assumetrue; nothing else is needed, because the target is already cancelled.attachments-deferred.test.cjs.ATTACHMENTS-MIGRATION.mdis updated.Tested on GitHub Actions at head
930876ea(the private test VMs were unavailable for this round):test:attachments138/138 on Node 18 and Node 24 (134 before, plus T-DT-15..18). The packed installs on the OTel floor and on the latest qualified release passed 149/149 each and include the deferred file. The F01 privacy/identity replay and the browser-safe leaf passed.runtime (24)at01b3327awas a runner DNS failure (Temporary failure resolving 'azure.archive.ubuntu.com'while installing browser fonts). It passed on this head.01b3327abuild must fail;930876eaand compared with the control.Why?
The user asked for first-class trace attachments, including Pipecat and LiveKit audio that plays in the UI, plus images, files and computer-use screenshots, in both auto modules. The voice and auto lanes need streaming recordings, preflight before decode, owned-file handoff and completed imports from the core rather than per-adapter uploaders.
How was it tested?
All runs used a temporary private GCP VM with synthetic data and in-memory or loopback doubles; no local tests, provider calls or real credentials.
Head
f0a56b37(loud stacked-lane breakage), on a new VM (deleted afterwards):pnpm run test:attachments(7 files, now includingattachments-compat.test.cjs): 117/117 on Node 24.11.1 and 117/117 on Node 18.20.8. Packed installs on OTel 2.0.1 and 2.12.1: 136/136 each (the CI file list plus the compat file). The compat file passed 8/8 in 5 separate runs. Provider lifecycle 19/19; F01 privacy/identity replay exit 0.createAttachmentHttpTransport(which carries V1) or when attachments are OFF; a wrapper that drops V1 is detected; legacy recording, WAV-tier owned recordings and imports are denied up front with zero extra transport calls and no resolver call; deny-wins still holds (policy_denied/disabledbeatunsupported_version); the legacy session runs toexpired; a status body withoutreasongivesunsupported_versionplus oneFI_ATTACHMENTS_WIRE_REJECTEDwarning that contains no capability, credential, URL or id; andtsc --noEmitontest/types/attachments-compat.tsagainst the built declarations consumes every@ts-expect-error(state !== "queued",status === "failed").f054a68esources fails 5/8 (C-01, C-03, C-04, C-07, C-08), and the updated legacy design row fails there (unpublished). C-02, C-05 and C-06 pass on both: they pin behaviour that must not change.031178f6sources fail T-RN-09 and T-RN-10 (7/9).f054a68eand this head pass 9/9 (Node 24 and 18).node:imports. In a VM context with noprocess, the legacy transport hitsconsole.warnexactly once, with the code.f054a68e: frameworks feat(frameworks): typed framework media attachments (LangChain, LlamaIndex, Vercel AI SDK, Genkit, Mastra) #679 helper 43/43 on both; provider-media feat(providers): typed provider media capture for TypeScript Gemini, Bedrock, compatible chat and speech #683 30/30 on both; openai/anthropic feat(openai,anthropic): capture typed images, documents and computer screenshots as attachments #682attachments-providers44/44 on both. Coding feat(coding-agents): opt-in typed image attachments for coding-agent integrations #680 + this head:tscreports TS2367 on"queued"(agent_core/src/attachments.ts:63,cursor_sdk/src/attachments.ts:45) and TS2322 on the transport type. These are the loud, pre-existing compile errors that lane must fix; they are not new.Head
f054a68e(renewal fix), on an earlier VM:tryWrite+ 1 sflush, no other entry point: renewal happens, 0 writes rejected after 120 s, 15 segments,published,captureCompletetrue, no gaps) and T-RN-10 (each trigger alone renews: recording write with no segment sealed and no transport call on the caller's stack,flush,status, worker wake).031178f6sources, T-RN-09 and T-RN-10 fail (56/58 in the design file), and the review probe A reproduces the bug (renewSession 0,expiredat 116 s, 1500/1500 late writes rejected, gaps [110000,5000,incomplete] and [115000,35000,incomplete],partial). On the head, the same probe gives renewSession 2,readythroughout, 1500/1500 late writes accepted, 15 segments,published. Removing any one trigger (touch, pump, flush, status) turns T-RN-10 red on that trigger.escaped decode allocation performance, the 200 ms wall-clock test); the base sources fail the same test on the same VM, so it is VM speed.Earlier qualification at head
031178f6:disabled,enabled-idle,enabled-blocked,enabled-no-shutdown,mid-recording,import-pending(all exit 0 in under 2 s).f3b0923c62for the shapes the SDK sends or parses agrees with the hand validators.attachments-contract.cjs): all 82 privacy and 11 identity rows throughattachANDpreflight(T-PF-03); 63 exact reasons on both paths, 10 unsupported positive paths and 9 conservative reason differences fail closed, as before.imageCapEscapedDecodePerformance.test.ts(a 200 ms wall-clock threshold, 249-268 ms on the VM). On the same VM the untouched baseb5831771fails the same test 3/3 (249, 258, 253 ms) as the head does (258, 258, 250 ms), so the failure is VM speed, not this change.E2E coverage
not applicable (no
e2e/harness in this repository) · flows hit: n/a · marker: n/a · verdict: not applicable. Product playback (OBS-E2E-061) needs the backend and ingest stages.Checklist
dev.Notes for reviewers
complete(and one of the two worker slots) until the next segment is sealed or the recording finishes, bounded at 35 s, so the last capability can still PUT the final manifest. Reconciliation should confirm this.f054a68e: 129 checks pass, 0 fail (13 skipped by path filters), including TypeScript attachment core and TypeScript core provider tests.f0a56b37: see the checks tab; the TypeScript attachment core workflow runs the new compat file throughtest:attachments.🤖 Generated with Claude Code