Skip to content

fix(mastra): enforce privacy and preserve SDK lifecycle semantics - #529

Closed
nik13 wants to merge 9 commits into
devfrom
fix/opus-followup-traceAI-494
Closed

nik13 wants to merge 9 commits into
devfrom
fix/opus-followup-traceAI-494

Conversation

@nik13

@nik13 nik13 commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up to the recall review of #494 and its first two independent reviews. Fix verified runtime and privacy findings without changing the supported Mastra floor or adding another tracer/provider.

  • Apply FI content/error hiding after native conversion and FI enrichment, covering both copies of I/O, system instructions, stop sequences, tool descriptions, status messages and exception attributes.
  • Scrub structured credentials on the peer floor as well as latest, including camelCase client secrets/private keys, AWS access-key fields, complete Cookie/Set-Cookie lists, nested JSON-stringified headers, and full Basic/Bearer credentials in error text. Recursive decoding is bounded and native inputs are not mutated.
  • Honor opt-in FI_PII_REDACTION with the existing shared FI PII implementation, including native/FI aliases and error status/events. The shared helper is now exported by @traceai/fi-core; this change must ship through the dependency-first release train with that core change, not as an isolated Mastra publication against the old core artifact.
  • Preserve native outcome/stop-reason fields. Errors and aborts are ERROR; interrupted/suspended/limited/unknown executions are not marked OK. Existing ERROR status is never downgraded.
  • Track export work before BaseExporter's asynchronous preprocessing. Flush/shutdown drain accepted conversions before the batch processor; shutdown is idempotent and stops accepting new events.
  • Resolve factory project selection as explicit option, then FI_PROJECT_NAME, then serviceName.
  • Determine generation/inference semantics from the emitting runtime's feature support, not the converter's potentially nested observability dependency. For mixed peer versions, reuse native request conversion with a shallow type view and export only once. Remove aggregate usage from generation wrappers so counts and token totals are not duplicated.
  • Preserve native remote parent IDs and links when an older converter omits them; retain existing converter context/link metadata without appending duplicate links.
  • Classify workflow sleep, wait-event and conditional-evaluation spans as CHAIN, including a real workflow sleep execution.
  • Supply the integration VERSION to instrumentation scope and telemetry.sdk.version.
  • Exercise real-SDK/decoded-OTLP loopback contracts on floor, latest and both mixed SDK sets, through both CJS and ESM entrypoints.

Granular input/message/text/image flags conservatively redact complete duplicated I/O aliases because native and FI blobs do not share a reliable typed schema. Explicit FI_HIDE_ERROR_DETAILS=false still overrides its implied setting.

Reproduction and validation

The first follow-up reproduced the five original runtime findings. The independent-review regressions also failed before these corrections: floor 9 failures, latest 8, latest-runtime/floor-converter 9, and floor-observability/latest-converter 9. The failures cover native request text, credential forms, opt-in PII, mixed-version LLM classification and unknown version metadata.

Set @mastra/core @mastra/observability @mastra/otel-exporter
Declared floor 1.16.0 1.0.0 1.0.0
Latest resolved from npm 1.81.0 1.23.0 1.5.0
Latest runtime, floor converter 1.81.0 1.23.0 1.0.0
Floor observability, latest converter 1.81.0 1.0.0 1.5.0

Passing checks:

  • Full frozen workspace install with ignore-scripts; lockfile unchanged.
  • Dependency and Mastra CJS/ESM/esnext builds.
  • Mastra Jest suite: 88/88 on Node 22.23.4 and Node 26.8.1.
  • All 88 unit assertions also passed against each of the four isolated real-SDK sets using native Node test registration and Jest expect, without mocked SDK imports.
  • Shared FI core suite: 142/142, including its existing PII policy tests.
  • Built Mastra/core tarballs installed without force or legacy-peer-deps. Each CJS/ESM entrypoint on each of Node 22.23.4 and Node 26.8.1 passed the 40-case wire suite: floor and floor-observability/latest-converter 37 passed with 3 intentional skips; latest and latest-runtime/floor-converter 40 passed with no skips. The skips concern outcome/link producers absent from old observability; reconciliation itself is unit-tested against all four sets. Across 16 runs: 616 passes, 24 skips, zero failures.
  • Contracts assert one LLM/chat observation for one real synthetic model request, unchanged total input/output tokens, native/FI content protection, status/events, exactly-once export, parentage, delayed/failed conversion, immediate flush/shutdown and project precedence.
  • Release-plan check: 129 packages consistent. Four-row CI YAML with both entrypoints parses; git diff --check passes.
  • Refreshed and normally merged origin/dev at 7cb1d3e42d9e3f80f3f80dc51a87417acdeb0c67; post-merge build, unit tests and release-plan validation passed.

Round-two failing-first wire tests reproduced credentials and workflow classification on all four sets (6 failures each) plus missing mixed-version remote parents and links (9 failures in that set). All 17 original independent-review assertions now pass using their unchanged real-SDK/decoded-wire probes. The expanded unit suite first reproduced 14 credential/workflow failures before the implementation changes.

All executable local tests ran in the outbound-network-denied sandbox with synthetic credentials/models and localhost OTLP only. No vendor/Cloud calls, registry publication or workflow dispatch. New-head CI is separate from these local results.

Existing work and scope

The README correction remains in existing open #398 (fix/mastra-readme-observability); this PR does not duplicate it. That separate PR remains necessary to replace the obsolete imports and telemetry configuration. That PR is being aligned to the restored explicit-option/environment/service project precedence and must ship with this runtime correction.

At the earlier review stage, no package versions or release manifests were manually changed. The following release-compatibility follow-up updates fi-core 1.1.1 and its manifest entry. No labels, protected branches, PR merge state or issue state were changed. The earlier nonblocking integration-version metadata finding is now included and fixed.

Additional review follow-up (2026-11-02)

  • Scrub credentials in JSON embedded after diagnostic prefixes, including nested sessionToken and auth_token, and redact entire signed AWS4-HMAC-SHA256/Digest Authorization values rather than only the scheme. Added failing-first privacy regressions for status, event, and tool error attributes; no mutation of native spans.
  • Map native processor_run spans to CHAIN and assert the kind from real Mastra agent tool execution on the latest runtime.
  • Merged the refreshed origin/dev normally (no force push). Frozen-lockfile install, release-plan check, build, and Mastra unit suite (93/93) passed. Packed CJS/ESM OTLP contracts passed against peer floor (41 pass, 3 version-specific skips per entrypoint) and latest (44 pass, 0 skips per entrypoint), all in the network-denied sandbox. Real-agent/decoded-OTLP probes for both versions found no synthetic credential leakage; latest emitted four processor spans classified CHAIN.

Released-core compatibility follow-up (2026-11-02)

Independent review reproduced zero exported spans when FI_PII_REDACTION=true with published fi-core 1.1.0: that release does not export redactPiiInValue, while the previously packed Mastra dependency ^1.1.0 still admitted it. The new failing-first manifest regression rejected workspace:^ before the fix. This branch stages fi-core 1.1.1 (package and release manifest) and pins Mastra to workspace:^1.1.1, which packs as ^1.1.1 and excludes published 1.1.0. This is release preparation, not a claim that 1.1.1 has already been published. The dependency-ordered TypeScript release workflow must publish and verify fi-core 1.1.1 before a new Mastra package version is published; do not publish Mastra independently or reuse its already-published 0.2.1 tarball.

Validation: release-plan check passed (129 packages), frozen workspace install and builds passed, fi-core Jest 164/164 and Mastra Jest 94/94 passed. Clean packed consumers with the staged 1.1.1 core passed the real OTLP contracts against Mastra floor (41 pass, 3 version-specific skips) and latest (44 pass, 0 skips). The staged Mastra tarball declares ^1.1.1; 1.1.0 does not satisfy that range. All executable tests ran in the outbound-network-denied sandbox using synthetic credentials and loopback OTLP. No registry publication or workflow dispatch was performed.

🤖 Generated with Claude Code

nik13 and others added 5 commits November 2, 2026 01:25
Honor FI content and error controls across native and FI aliases, scrub credentials, preserve runtime outcomes, and drain conversions before flush or shutdown. Respect environment project routing and classify only actual model calls as LLM observations. Add offline floor/latest SDK contracts and CI.

Co-Authored-By: Claude Code <noreply@anthropic.com>
…eAI-494

Co-Authored-By: Claude Code <noreply@anthropic.com>
Cover native request text, structured and error-text credentials, and opt-in shared PII redaction. Classify calls from emitting-runtime feature support, reuse native conversion across mixed peer versions, and emit deterministic integration metadata. Exercise CJS/ESM wire contracts across floor, latest and both mixed SDK sets.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Cover AWS credential fields, full cookie headers and nested JSON error serialization. Reconcile remote parents and links with older converters, and classify native workflow control-flow spans. Add failing-first SDK/OTLP contracts and unit coverage.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
nik13 and others added 4 commits November 2, 2026 13:43
…ssors

Co-Authored-By: Claude Code <noreply@anthropic.com>
Stage fi-core 1.1.1 for dependency-first release and constrain Mastra to a core build that exports redactPiiInValue. Add a regression guard for the packed dependency floor.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@nik13

nik13 commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Superseded: #567 (b4b0b8f) consolidates the Mastra runtime, privacy and lifecycle work and adds tool-history and schema fixes. The branch is kept. This was confirmed in a read-only triage of the open PR backlog, with an independent check of each closure.

@nik13 nik13 closed this Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant