Skip to content

feat(python): attachment core with bounded lifecycle, preflight, recordings and imports - #676

Draft
nik13 wants to merge 4 commits into
devfrom
feat/attachments-python-core
Draft

nik13 wants to merge 4 commits into
devfrom
feat/attachments-python-core

Conversation

@nik13

@nik13 nik13 commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Part of TH-2379 (Linear).

Scope

Python attachment core (P01/P02) and auto plumbing (A01), now implementing the shared SDK streaming design (sdk-streaming-design.md, sections 0-9) against the frozen F01 contract f3b0923c62 (#675). Draft; not merge-ready until the voice/backend lanes integrate.

Everything stays OFF by default. Creating new recordings is separately OFF (client record_audio AND server policy.record_audio). Admission evaluates the vendored F01 privacy table deny-wins: hide flags and PII redaction win (raw media under PII redaction is redaction_unavailable without a qualified sanitizer, which v1 does not ship).

Review fix pass (c2, head 5a7e1105)

An independent review of 7b03f8d3 found three blocking defects. Each is fixed here with regression tests, and each test was shown to fail on the pre-fix head.

  1. Privacy lattice: owned WAV admission did not match the wire. attach_owned_recording admitted a wav_recording-tier file with the caller's direction label, but every segment of that tier goes on the wire as mixed (stereo carries both sides in every byte; segments carry no track_id; mono maps to the mixed layout). With hide_outputs=True, direction="input" and stereo wav_tracks, the file was uploaded. Now the tier is admitted, re-checked per chunk and re-evaluated on policy updates as mixed, and every segment request says mixed. The live recording path already evaluated stereo as mixed; it now has a test that a per-track mutation fails.
    • Behaviour change: a wav_recording-tier file is denied when EITHER side is hidden, whatever label the caller passed (this includes a mono WAV labelled input, whose wire direction was already mixed), and when the server's allowed_directions lacks mixed. An invalid caller label is policy_denied up front.
    • The caller's label is kept only for the unparseable-header fallback to tier single, whose wire direction is "as given" (design 4); its checks stay at the stricter mixed.
  2. traceai crashed on import with an older core. traceai imported fi_instrumentation.attachments unconditionally while its floor (fi-instrumentation-otel >=0.1.7) still admits releases without it (1.3.1 is the current release). The import is now guarded: on such a core, import traceai and plain init() work, register() gets no new keywords, instrumentors get no attachment keywords, attachment_config/trace_config raise a clear RuntimeError, attachment_status()["runtime"] is unsupported_version, and session binding reports unsupported_version.
    • The floor is not raised in this PR: python-tests.yml runs poetry install in python/traceai, which resolves the core from PyPI before overlaying the repo core, so a floor above 1.3.1 would fail that job until a core release with attachments exists. The release PR that publishes that core should raise the floor.
  3. limits.recording_bytes was not enforced. The writer compared bytes still buffered against the cap, so a drained recording uploaded past it and ended published. The writer now keeps a cumulative count of bytes sealed into segments, 44-byte headers included, plus every unsealed run and the headers a new part would start. It truncates (gap truncated, capture_complete=false, result partial) once the total would exceed min(limits.recording_bytes, 1 GiB).

Re-proof on a private GCP VM (fagi-omega-staging, no external IP, no service account; Debian 12, uv CPython 3.11 / 3.14, OTel SDK 1.45.2):

New regression tests (23 cases), fix head:               23 passed
Same tests on pre-fix 7b03f8d3 (test files swapped in):   11 of 23 fail
  The live-stereo test passes on both heads (that path was already right) and fails under the mutation below.
Review probe stereo_owned, direction=input, hide_outputs: pre-fix tier=wav_recording, 192,044 B uploaded, request direction ["input"]
                                                          fix: tier=denied, policy_denied, 0 B
  control, no hide flag:                                  fix: validating, request direction ["mixed"], manifest tracks ["input","output"]
Review probe recording_bytes_cap2 (3,000,000 B cap, 120 s fed, draining):
                                                          pre-fix: 5,760,176 B in 4 segments, published, no gaps
                                                          fix: 2,976,132 B in 3 segments, partial, gap "truncated", duration 120,000 ms
traceai from source + published fi-instrumentation-otel 1.3.1:
                                                          pre-fix: ModuleNotFoundError: fi_instrumentation.attachments
                                                          fix: import ok, runtime unsupported_version, init ok, attachment options refused with a clear error
python-tests.yml traceai job shape (package install resolves core 1.3.1, repo core overlay): 25 passed
Negative controls on the fix head, attachments + auto suites:
  stereo evaluated per track (manager.py live path)   -> 2 failed
  owned WAV admitted with the caller label             -> 5 failed
  cap on buffered bytes only (the old check)           -> 2 failed
  segment headers not counted                          -> 1 failed (boundary test: 4 writes fit with headers counted, 5 without)
  traceai hard import                                  -> 1 failed
  register() always given the new keywords             -> 1 failed
Component command (python/tests/attachments + python/traceai/tests): 435 passed on 3.11 and on 3.14
python/tests/attachments x3 on 3.11 (final head):        410 passed every run
Core CI command (fi-core-python-compat, latest OTel), on 45e9d653 (final head differs only by one test's arithmetic):
  3.11 tracing 3373 passed, 6 skipped, 29 deselected; 3.11 [grpc] 3402 passed, 6 skipped; 3.14 tracing 3373 passed
  base dev b5831771 on the same VM: 1 failed (test_base64_image_cap timing assertion, VM speed), 2963 passed

CI blocker: root cause and fix

The 6 latest-OTel tracing compatibility jobs hung at 95% because test_after_fork_resets_workers_without_parent_replay called os.fork() inside the multi-threaded pytest process. OpenTelemetry SDK 1.45.2 registers TracerProvider._handle_fork as an after_in_child hook; in the child it calls get_aggregated_resources -> ThreadPoolExecutor.submit, which deadlocks on an inherited lock. OTel 1.29.0 (floor) has no such hook, so only the latest jobs hung; the stuck child was the orphan process.

  • Reproduced on a private GCP VM with the exact CI command (Python 3.11, OTel 1.45.2): faulthandler and py-spy dumps show the child blocked in TracerProvider._handle_fork.
  • Reproduced with no attachment code at all: a TracerProvider created before concurrent.futures.thread is imported, then os.fork() -> the child hangs.
  • Fix: fork coverage now runs in separate-interpreter exit probes (python/tests/attachments/exit_probe.py, design 8.5); no test forks the pytest process.
  • Also implemented (design section 8; required behaviour, but not what hung): create_attachment_manager returns a DisabledAttachmentManager null object when OFF (no thread, timer, fork hook, atexit hook, wire-adapter or privacy-table load, span attribute); one fork hook per process over a WeakSet that resets live managers in the child without touching inherited locks; workers start lazily; every wait is bounded (<= 1 s slices); shutdown(timeout) is idempotent and returns within timeout + 250 ms even with a blocked transport.

What the core now provides

  • Preflight (section 1): manager.preflight(target, mime_type=, direction=, max_bytes=, operation=, ...) returns a single-use AdmissionTicket after the full privacy check, without touching the source; it reserves a descriptor and byte budget, expires lazily (no timer), is re-checked at use and can only narrow. attach() is preflight + ticket.attach().
  • Multi-modal: F01 MIME -> modality map (audio, image, video, document, file), full Purpose enum including computer_screenshot and document; the server's allowed_modalities is the gate. Per-kind caps (50 MiB single, 20 MiB image, in-memory <= owned budget; mutable buffers copied only up to sync_copy_bytes).
  • Live recordings (section 2): start_recording(...) -> RecordingWriter. Sample-count clock; WAV segments <= 30 s and <= 8 MiB including the 44-byte header (worked vectors asserted: 1 440 044 / 5 760 044 / 8 388 140 bytes); shared cut points across tracks; track_id only for separate_tracks; strictly ordered uploads with a moving window; prefix-monotonic manifests PUT with the latest completed segment's capability (BR-7 fallback order implemented); try_write never does I/O.
  • Renewal (section 3): lazy session renewal replays bootstrap with the same session id and nonce (bs-{sid}-{n}); upload-capability renewal replays the byte-identical init body; a stale capability ends expired without creating a new upload.
  • Owned files (section 4): RecordingFileScope.identify() (secure open + fstat), attach_owned_recording() with tiers single (streams from the lease, one slice of memory) and wav_recording (segments synthesized from pread ranges; never materialized); encoded files above the cap are too_large with zero bytes read.
  • Completed-recording import (section 5): await import_completed_recording(target, request, resolver=) with the resolver seam (default UnavailableResolver), original credentials only, im-{rid} key, no span projection, one-shot status().
  • Consumed streams (section 6): start_attachment_stream() receives only chunks the application already consumed; over-cap discards everything; the reference is projected only at finish().
  • Channel/role (section 7): never sent in UploadRequest; manifest tracks are the authority for recordings.
  • Wire adapter (section 9): protocol.py holds every route, the 0.5 status->reason table with in-class Error.code refinement, 0.6 idempotency keys, separate control/content deadlines, a token-bucket pacer, strict JSON, and schema + cross-field validation.
  • Span carrier (0.8): all four F01 attributes, only on admitted capture targets that are still recording; AttachmentSpanProcessor.on_start no longer stamps every span.

Breaking changes for consumers (voice lane): status is a method returning AttachmentStatus; adapters use preflight() instead of manager._session / manager.policy.evaluate; max_attachment_bytes removed; register()/traceai.init() hold DisabledAttachmentManager when OFF.

Evidence

All runs on a temporary private GCP VM (fagi-omega-staging, no external IP, no service account), synthetic credentials and synthetic audio only; no local tests. Logs are kept with the lane evidence.

Commands (VM: e2-standard-8, Ubuntu 24.04, uv-managed CPython 3.11.14 / 3.14, OTel SDK 1.45.2):

# 1. Reproduce the hang at 9962e50e with the exact CI command (tracing profile, no extras):
python -m build --wheel python && pip install "$wheel" 'pytest>=7,<8' 'pytest-asyncio>=0.21,<0.22' pytest-mock pytest-cov responses
cd python && python -m pytest tests -p no:cacheprovider --ignore-glob='tests/test_framework_*.py' -k 'not grpc' -o faulthandler_timeout=90 -v
# head 9962e50e: timed out (rc 124); faulthandler + py-spy: forked child stuck in
#   TracerProvider._handle_fork -> get_aggregated_resources -> ThreadPoolExecutor.submit
# base dev b5831771, same VM: 2964 passed, 6 skipped, 29 deselected (rc 0)
# standalone probe, no attachment code: provider created, then concurrent.futures.thread imported, fork -> child HUNG

# 2. Same CI command on this branch (both extras, as the workflow runs them):
#   3.11 tracing:        3352 passed, 6 skipped, 29 deselected in 124 s, 0 faulthandler timeouts
#   3.11 [grpc]:         3381 passed, 6 skipped
#   3.14 tracing:        3352 passed, 6 skipped, 29 deselected in 122 s, 0 faulthandler timeouts
#   3.14 [grpc]:         3381 passed, 6 skipped
#   wheel contains _privacy_table.json and _protocol_schema.json (zip inspection)

# 3. Component suite (attachments + auto), PYTHONPATH=python:
python -m pytest -q -o faulthandler_timeout=60 python/tests/attachments python/traceai/tests
# 412 passed; then python/tests/attachments x10 in a loop: 388 passed every run

Design test IDs covered: T-PF-01..08, 10..15; T-ST-01..20; T-RN-01..08; T-LS-01..03, 06..10 (T-LS-04 via the existing lease position test); T-IM-01..10; T-CS-02..07; T-CH-01..06; T-LC-01, 03..16 (exit probes disabled, enabled-idle, enabled-blocked, enabled-no-shutdown, fork, mid-recording, import-pending in separate interpreters); T-TR-01..08 (all F01 wire fixtures for SDK shapes; all 82 F01 privacy fixtures). TS-only rows (T-PF-09, T-CS-01, T-LS-05, T-LC-02) and Windows (T-LS-11) do not apply.

GitHub CI on this head is the authority for the wall-clock fi-core suites.

Not in this PR

  • Pipecat/LiveKit adapter migration to preflight/start_recording/attach_owned_recording and the LiveKit Egress resolver (voice lane, design 12.10/12.11).
  • Backend behaviours BR-1..BR-7 and contract questions CQ-1..CQ-5 run on the design's fail-closed fallbacks until the backend/contract lanes confirm them.

🤖 Generated with Claude Code

Add deny-wins audio admission, immutable correlation handles, bounded provider-owned transfer workers, explicit owned-file leases and session-binding entry points. Keep unqualified retrieval and recording streams fail-closed.

Co-Authored-By: Claude Code <noreply@anthropic.com>
nik13 and others added 2 commits November 10, 2026 22:24
…esign

The latest-OTel compatibility jobs hung because a test forked the
multi-threaded pytest process: OpenTelemetry SDK 1.45's at-fork handler
(TracerProvider._handle_fork) deadlocks the child on an inherited
ThreadPoolExecutor lock. Fork coverage now runs in separate-interpreter
exit probes, so no test forks the runner.

Lifecycle (design section 8): a DisabledAttachmentManager null object when
attachments are off (no thread, timer, fork hook, wire adapter or schema
load), one fork hook per process, lazy daemon workers, bounded waits, and an
idempotent shutdown that returns within its timeout.

Implements the shared SDK streaming design against F01 f3b0923:
preflight tickets evaluated with the F01 privacy table, multi-modal MIME
mapping and per-kind caps, segmented PCM recordings with prefix-monotonic
manifests, session and capability renewal, owned-file tiers with WAV range
segmentation, the completed-recording import seam, consumed single-object
streams, idempotency keys, the status-to-reason table and a pacer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The worker released a recording lane before the writer recorded the
segment's result, so the next segment could read a stale index and
finish() could build the final manifest early. The lane now stays busy
until the writer callback has run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nik13 nik13 changed the title feat(python): add opt-in attachment core and auto plumbing feat(python): attachment core with bounded lifecycle, preflight, recordings and imports Nov 11, 2026
…ing cap, old-core traceai import

- attach_owned_recording: the wav_recording tier admits and re-checks every
  chunk as `mixed`, matching the wire (every segment is sent as `mixed`), so a
  hide flag on either side denies the whole file. The caller's label is kept
  only for the unparseable-header fallback to tier `single`.
- RecordingWriter: limits.recording_bytes now caps the cumulative bytes sealed
  into segments (44-byte headers included), not the bytes still buffered, so a
  drained recording is truncated at the negotiated total.
- traceai: import fi_instrumentation.attachments guarded; on a core without it,
  `import traceai` and plain init() work, attachment options raise a clear
  error, and session binding reports unsupported_version.
- Regression tests for each, including live stereo with one side hidden.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant