Skip to content

fix(fi-core): redact native GenAI message payloads - #416

Open
nik13 wants to merge 2 commits into
devfrom
fix/fi-core-genai-parts-privacy
Open

nik13 wants to merge 2 commits into
devfrom
fix/fi-core-genai-parts-privacy

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Keep @traceai/fi-core and repair native GenAI message privacy before release. GenAI is a semantic-convention family, not an integration package to delete or a reason to add another converter.

  • Mask native role/parts message payloads as well as legacy content. Text hiding covers text, reasoning, compaction, tool arguments/results, server-tool payloads, opaque media and extension fields, retaining only scalar message/part metadata.
  • Recognize image parts, including native blob/URI/file parts with image modality or MIME type. Image-only hiding preserves known text and explicitly non-image media; unknown or malformed shapes and opaque tool payloads fail closed. Unclassified binary data is not assumed safe.
  • Apply input image and text flags together instead of letting the first masking rule bypass text hiding. Apply configured PII redaction to retained text.
  • Document the actual FITracer/FISpan boundary: raw external SDK spans are not automatically protected by FI flags. No provider replacement, global monkeypatch, duplicated spans, dependency changes or public entry-point removal.

Native fixtures follow the pinned OTel GenAI schema, including its generic and multimodal part types. No competitor implementation was copied. This fixes privacy in the existing shared boundary; it does not claim backend normalization parity.

Reproduction and verification

Base: d6a9966dd4d5d5fb344f7f5ab11b7483c2c2a6bc (dev). Before editing, the built base leaked native input/output text and images, including through real local OTLP HTTP export. At both pinned SDK versions, substituting the exact base masking module into otherwise identical packed fixtures fails 73 of 88 regressions.

The corrected installable packages pass:

  • 88/88 regressions with SDK 2.0.1 and exporter 0.202.0 (floor).
  • 88/88 with SDK 2.12.0 and exporter 0.223.0 (current compatible SDK-2 line).
  • 142/142 existing core Jest tests.
  • CommonJS, ESM and ESNext builds, packed ESM import, and git diff --check.

Regression coverage includes creation/active spans, single and bulk setters, links, sampler visibility, text/image combinations, tool payloads, malformed JSON and wrong-type metadata, unknown binary parts, empty/idempotent input, a 1,000-message batch, PII composition, environment flags and broad-hide controls. Real loopback OTLP checks assert no private payload in wrapped spans, unchanged trace/parent IDs and no duplicates; an unwrapped positive control proves the documented external-span boundary.

All runtime tests ran with outbound network denied except loopback. No vendor/model API, real credentials or production backend was used. CI builds packed fixtures for both versions and runs regressions plus existing unit tests in a loopback-only Linux network namespace. Re-run after building with node typescript/packages/fi-core/test/genai-privacy.test.cjs; install-genai-fixture.cjs prepares the packed matrix.

Compatibility and release

E2E coverage

Not applicable (no product e2e/scripts/e2e-coverage.mjs harness in traceAI) · flows hit: n/a · marker: n/a · verdict: not applicable. The package's public built API and OTLP wire are covered offline; no Cloud/UI execution claim.

Standards self-review

Reviewed the final base diff against internal-docs coding standards at 7360c146183c684a96e75e57069b2661445135ef: existing shared layer, typed unknown-JSON boundary, explicit fail-closed privacy, compatible entry points, no arbitrary recursion or new runtime dependencies, regression/failure/large-input tests and scoped CI. No remaining introduced correctness blocker found. Python/migration/frontend-contract gates do not apply to this TypeScript-only package change. Backend normalization and exporter SDK-3 migration are outside this repair.

🤖 Generated with Claude Code

nik13 and others added 2 commits October 31, 2026 09:15
Mask standard parts and opaque payloads in the existing FITracer privacy boundary. Compose text, image and PII controls and fail closed for malformed messages.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
nik13 added a commit that referenced this pull request Nov 2, 2026
Protect stored-prompt inputs and MCP credentials in all telemetry copies. Integrate the shared structured-message redactor and tests from #416 at 8630636 while preserving duplicate-payload masking. Aggregate function/custom deltas by SDK index without overwriting request model metadata. Run core Jest directly inside network-isolated CI.

Co-Authored-By: Claude Code <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant