Skip to content

fix(agents): preserve oversized structured content after privacy scrubbing - #615

Draft
nik13 wants to merge 12 commits into
devfrom
fix/agent-core-oversized-content
Draft

nik13 wants to merge 12 commits into
devfrom
fix/agent-core-oversized-content

Conversation

@nik13

@nik13 nik13 commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Closes #608.

Keep oversized coding-agent content readable without losing the complete values needed for privacy checks.

  • Match complete configured, registered and environment credentials, including encoded JSON, before PII/media/embedding policies and the 20,000 UTF-8-byte display cut. Keep FITracer admission/export protection intact.
  • Fix the late-registration blocker: retain bounded, memory-only originals for root/tool input, input events and turn-owned model output until finalization. A credential registered after capture but before span completion cannot leave its truncated prefix on the wire. Preserve event timestamps and release originals on consumption, discard and shutdown.
  • Fix punctuation-password over-redaction: distinguish matches inside JSON string leaves or ordinary prose from matches that actually overlap JSON framing. Keep unrelated public text; continue withholding containers whose framing or media/privacy context would be damaged.
  • Fix the same early-shortening path found during self-review in the OpenCode observer. Retain complete bounded originals between hook events, seed capture-time credentials, sanitize during synchronous final emission, and leave the display cut to core. Preserve its existing sanitized adapter-to-tracer contract.
  • Retain the original readable nested-JSON, multibyte, credential-overlap, late embedding-context and 65/100-item OpenCode fixes. Restore the readable-content assertions previously relaxed in f024eb6.

Qualified revision and status

Head: ca34b9f9e3127bbb2cf41c7935716525c816a3fa.

Merged exact current dev, e4aecfe128e29a942f4f637d39872fc8c355c648, without rebasing, renaming the branch or force-pushing. fi-core is 1.2.3 in its package, release manifest and changelog; the 1.2.1/1.2.2 entries are preserved. This package has no separate runtime version constant.

Final E2B qualification passes. GitHub CI is fully green: 121 passed, 13 skipped, no failed or pending checks. The PR remains draft for human/non-author review. No merge, admin override or package publication was performed.

Failing-before / passing-after evidence

Every build/test ran in E2B with whole-repository source stages, synthetic credentials and loopback OTLP collectors; no local tests or customer/provider credentials.

Proof Before Final head
Reported late registration and punctuation-password blockers fu608-fix-before-1108a: 22 of the first 29 committed OTLP regressions fail on the pre-fix code Expanded 46-case finalization/punctuation OTLP suite passes; independent late-registration and normal-credential probes protect all five surfaces and retain public text
Additional OpenCode late-registration path fu608-fix-observer-before-1108a: both normal and PII-overlap credentials leak prefixes on all six content surfaces fu608-fix-observer-after-1108b and full final run: both cases have zero prefix leaks on all six surfaces; all 42 OpenCode tests pass, including the unchanged recorded-flow privacy assertion
Original five issue assertions Exact dev run fu608-fix-base-1108b: all five fail fu608-fix-final-1108b: all five pass
Codex conformance lifecycle Exact dev reports NOT_REGISTERED: the native CLI stand-in advertises hard-coded 0.1.0 after the 0.1.1 package release Separate test-only commit reads the packaged version; the status/doctor assertion is unchanged, and all 30 Codex cases pass

The original five assertions pin readable >20,000-character JSON on all five surfaces; byte-bounded multibyte content; no environment-credential prefix at a prose cut; the late embedding discriminator; and complete ordinary 65/100-item OpenCode inputs.

Full final qualification

fu608-fix-final-1108b ran the exact pushed source and exited 0:

  • fi-core: 62 suites, 1,749 passed, 13 pre-existing skips. Baseline: 61 suites, 1,748 passed, the same skips.
  • 22 agent/adapter package suites: 1,380 passed, 0 failed, 28 pre-existing optional-host skips. Baseline: 1,296 passed, 0 failed, the same skips. Includes 523 agent-core and 42 OpenCode V1 tests.
  • Conformance: 422/422 cases across 14 agent surfaces. Baseline has only the inherited Codex stand-in version failure described above; this PR repairs the fixture rather than waiving the check.
  • Reviewers' original boundary/privacy probes, both controls, 57 adversarial cases under default / embedding hiding / PII, 12 credential-overlap cases, and 117 extended cases all run on baseline and head.
  • Captured-evidence comparison: zero regressions/failures. Credentials and enforced PII/embedding controls do not become worse than baseline. Prose-prefix protection improves in all three modes; late embedding-context protection improves with its hide flag.
  • Both original review probes and the additional OpenCode late-registration probe pass again in the full final run.

All completed E2B sandboxes were killed and deletion verified with 404. The final source-stage identity receipt verifies that all 20 changed files match the pushed commit byte-for-byte.

Exact commands and receipts

Runner and source:

PY=/Volumes/nik-ext-1/tmp/pr3689-merge-ready-xv21xce2/validation/e2b-users-final/sdk-venv/bin/python
RUN=/Volumes/nik-ext-1/tmp/traceai-release-20261104/e2b/e2b_run.py
STAGE=/Volumes/nik-ext-1/tmp/fu608-fix-1108

"$PY" -I "$RUN" --label fu608-fix-final-1108b \
  --src "$STAGE/final-b" --timeout-min 58 \
  --cmd 'bash /work/qualify.sh head; result=$?; node /work/late-registration-probe.cjs head > /work/late.jsonl; node /work/normal-credential-probe.cjs head > /work/normal.jsonl; node /work/observer-late.cjs > /work/observer-late.jsonl; observer=$?; if [ "$result" != 0 ]; then exit "$result"; else exit "$observer"; fi' \
  --fetch evidence proof qualification.json conformance-results late.jsonl normal.jsonl observer-late.jsonl install.log build.log

The runner caps the requested timeout at 55 minutes. qualify.sh runs review.sh (frozen pnpm install, dependency-ordered builds, fi-core Jest, all 22 package suites, reviewer matrices and conformance) followed by prove.sh (byte-safe collectors and the five original assertions). The final committed regressions run through normal pnpm --filter @traceai/agent-core test and pnpm --filter @traceai/opencode-v1-plugin test commands.

  • Final receipt: /Volumes/nik-ext-1/tmp/traceai-release-20261104/e2b/runs/fu608-fix-final-1108b/receipt.json.
  • Exact baseline receipt: /Volumes/nik-ext-1/tmp/traceai-release-20261104/e2b/runs/fu608-fix-base-1108b/receipt.json. Its exit 1 is explicitly attributed to the inherited Codex fixture; the original five proof failures are expected.
  • Comparison: /Volumes/nik-ext-1/tmp/fu608-fix-1108/comparison-final.json.
  • Staged source identity: /Volumes/nik-ext-1/tmp/fu608-fix-1108/stage-identity.json.
  • Scripts and extracted evidence: /Volumes/nik-ext-1/tmp/fu608-fix-1108/ (scripts/, base-current-results/, final-results/, and the named before/after result directories).

GitHub CI

All 121 passing / 13 skipped checks refer to the qualified head. The required Python tests check passed.

An initially cancelled, not assertion-failing, Codex SDK matrix leg caused its integration aggregate to fail. The targeted rerun and dependent aggregate both passed on attempt 2. There are no remaining CI failures to exempt, and no check or assertion was disabled.

CI snapshots: /Volumes/nik-ext-1/tmp/fu608-fix-1108/ci-current.json and ci-integration-rerun.json.

Review, limits and rollout

  • Self-reviewed the actual base-to-head diff against the current internal minimal-code, typing, failure-mode, testing, comment and code-structure standards. The additional OpenCode path was found and reproduced during that review; it is fixed and covered, not deferred.
  • No database migration, product API route, UI or OSS/EE boundary change. Product e2e/ classifier: not applicable (no harness). Actual OTLP wire exports and cross-agent conformance cover this SDK path. Python lint/migration and frontend-contract gates are not applicable to this diff.
  • Originals remain memory-only: 1 MiB per-value inspection bound, 8 MiB UTF-8 pending-content budget in the turn tracer, and the observer's existing 4 MiB cumulative budget. Over-budget data fails closed instead of emitting an uninspected prefix. Capture defaults and destination policy are unchanged.
  • OpenCode adds a direct dependency on the already-used workspace fi-core package for capture-time credential seeding. No new external library.
  • Unchanged baseline limitations remain explicit: a configured credential deliberately used as a tool label is still visible in the metadata probe; embedding hiding can withhold output surfaces wholesale; oversized prose containing an embedded JSON island can still hit FITracer's conservative omission. This PR does not bypass those policies or claim to repair the inherited metadata issue.
  • Original reviewer collectors decode TCP chunks independently; the additional byte-safe copies and committed wire tests buffer UTF-8 correctly. The strict byte-proof pass checks every captured head value is within 20,000 bytes.
  • After human review, publish fi-core 1.2.3 before dependent agent packages. This work did not publish packages. Reverting restores the earlier truncation behavior; no stored-data migration is needed.

🤖 Generated with Claude Code

nik13 and others added 12 commits November 8, 2026 01:46
Preserve FITracer admission and export privacy. Match configured, registered and environment credentials before structural/PII changes, then apply FI policy and keep valid byte-bounded JSON. Inspect complete OpenCode tool inputs rather than cutting at 64 items. Extend the embedding policy to inspect late object discriminators.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Fail closed when matching a complete credential removes a JSON key, a media discriminator or source framing. Reuse fi-core hint recognition without importing its exporter into lightweight hook modules.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Cover embedded and nested JSON plus array-valued MIME discriminators. A literal credential spanning JSON framing fails closed before island parsing can alter part of the match.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Supply the shared discriminator predicate from the full tracer boundary. Lightweight privacy helpers keep their dependency-free startup contract while the sanitizer continues to reuse fi-core recognition.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve the process-resource privacy fix and agent workspace builds from dev. Keep fi-core package, manifest, and changelog aligned for the next patch.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Include the fi-core build corrections from #619 before repeating the baseline and oversized-content regression proofs.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Defer bounded input, event and turn-owned output content until span finalization so newly registered credentials cannot leave cut prefixes. Distinguish JSON string matches from actual framing overlap, retain public text around punctuation credentials, and cover both regressions on the OTLP wire. Reserve fi-core 1.2.3 after the merged 1.2.2 fixes.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
The inherited 0.1.0 stand-in reports NOT_REGISTERED after the 0.1.1 Codex release. Read the packaged version without weakening the status and doctor assertions.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Keep bounded originals in the memory-only observer and seed capture-time credentials; defer PII, credential scrubbing and display shortening to core finalization. The E2B regression exposed late prefixes on all six OTLP surfaces, including a PII-overlap secret.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Keep the existing sanitized adapter-to-tracer contract, but perform it only during synchronous emission and retain the complete value for core admission. Do not weaken the recorded native-flow secret assertion.

Co-Authored-By: Claude Code <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant