Skip to content

fix(genkit): enforce error privacy on native spans - #380

Open
nik13 wants to merge 43 commits into
devfrom
fix/TH-8240-genkit-error-privacy
Open

nik13 wants to merge 43 commits into
devfrom
fix/TH-8240-genkit-error-privacy

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fix native Genkit error privacy without wrappers or duplicate spans. Native spans bypass FISpan, so the mapper must enforce the same error-detail policy before exporting its copy.

  • Resolve hideErrorDetails through fi-core's tri-state parser: option > valid environment value > input/output/message/text hiding. Preserve explicit false overrides.
  • Omit hidden status descriptions and exact/namespaced error-detail attributes from spans and all events, after context merging. Preserve status codes, exception types, metadata, native trace/span IDs, parents, and the original native spans.
  • Retain the processor entry point, helper signatures, legacy ContentPolicy objects, and deprecated captureContent fallback.
  • Run the full contract suite, unit tests, and privacy wire tests on Genkit 1.42.0 and 1.43.0 in loopback-only CI.
  • Correct legacy compatibility assertions: Genkit 1.43's Dev UI uses independently identified direct instrumentation. Check exact native OTel identity preservation and equivalent complete execution trees instead of requiring Dev UI IDs to equal OTel IDs. Six negative controls reject missing, duplicate, renamed, reparented, or reidentified spans.
  • Replace the README's QA diary with customer setup guidance, explain Dev UI ID differences, and exercise the setup example verbatim against the loopback collector.

Related: future-agi/docs#1016. Updated this existing PR with normal merges from dev through b059859f; no force-push, package version change, or lockfile regeneration.

Verification

All local execution tests and builds ran under the prescribed outbound-denied sandbox (localhost allowed), with synthetic credentials and no vendor/model API calls. Final floor/latest runs used an isolated test home and empty ambient environment.

Check Result
Current dev 2c0bea07, native Genkit 1.42.0 → real fi-core HTTP exporter → localhost OTLP Original regression reproduced: 20/24 privacy cases fail
Merged PR before compatibility-test repair, Genkit 1.43.0 Reproduced all three previously reported legacy failures: two Dev UI/OTel ID equality assertions and one hard-coded version assertion
Final Genkit 1.42.0 and 1.43.0 × Node 20.20.2 and 26.8.1 Each combination: CJS/ESM/esnext build succeeds; 121/121 unit tests, 24/24 privacy wire cases, 28/28 full contracts pass
Full contracts Native trace identity and parenting, Dev UI reflection/telemetry, content/errors/schema failures, model tokens, stream completion, session mapping, SIGTERM, collector failure, inventory, package contents, CJS/ESM imports, and the README example
Clean packed consumer, registry fi-core 1.1.0 + Genkit 1.43.0 24/24 privacy wire cases and CJS import pass; wire execution uses the ESM entry point
Release consistency python3 .github/release/release_plan.py --check passes (129 packages)
Full TypeScript frozen install pnpm install --frozen-lockfile --ignore-scripts passes; lockfile unchanged
Final diff git diff --check passes

The final additional base merge contains only Go/Java/release changes; TypeScript, the Genkit workflow, and the shared Python receiver are identical to the tested tree. Release consistency and the frozen install were rerun after that merge.

The three legacy Genkit 1.43.0 failures are resolved, not skipped or waived. The CI workflow now runs those full contracts in both matrix jobs. CI is triggered by this PR branch update; no workflow dispatch was used.

Release scope

No runtime dependencies, API routes, migrations, or frontend changes. Keep the native mapper and release through the normal release-please flow. This update does not merge, tag, or publish anything. Update the existing customer docs PR #1016 after the corrected package is published; no duplicate docs PR was created or modified here. Independent final review remains required.

Update: one privacy policy for every attribute family

The per-attribute masking is replaced by a closed allowlist of non-content metadata. That policy now covers span, context-merged, event, link and resource attributes, on both the manual processor and the automatic mapping path. It includes vector query predicates and nested legacy projection fields. The automatic exporter keeps its separate error-detail policy and its credential, media and PII safeguards. Aggregate usage copies keep their metadata. Current dev is merged (b5831771).

Verification ran on a fresh isolated test VM with synthetic credentials and no vendor or model calls:

Check Result
Privacy wire tests on Genkit 1.42.0, 1.43.0 and 1.44.0 All 240 cases and 187,200 checks over 780 derived attributes pass. No positive-control or placeholder failures.
Full core suites Pass: 2,263 tests on this branch and 2,254 on current dev. The same 13 tests are skipped on both.
Legacy suites 99 commands pass. The other 69 fail the same way on current dev in the same environment, so they predate this PR.

🤖 Generated with Claude Code

nik13 and others added 30 commits October 31, 2026 07:17
Resolve hideErrorDetails through fi-core, omit native status descriptions and error detail attributes before export, and preserve explicit opt-outs and native span identity. Cover the policy with offline unit and localhost OTLP checks at Genkit 1.42.0 and 1.43.0.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
…ersions

Preserve native OTel identities while checking the independent Dev UI tree on Genkit 1.43. Run the full contract suite in the floor/latest CI matrix, add negative topology controls and a verbatim README example, and remove customer-facing QA diaries.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Apply input privacy to native generation response requests and remove serialized failure details before both OTLP output copies are emitted. Cover real successful and failing generate calls across the privacy matrix and retain status, usage, public output, and native span identity.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Document embedded request masking, serialized error fields, and message flags on surrounding generate spans.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
…usage

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
…rces

Co-Authored-By: Claude Code <noreply@anthropic.com>
…licy

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
nik13 and others added 13 commits November 10, 2026 04:43
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant