Skip to content

fix(pii): scope redaction to free-text attributes, require Luhn-valid card shape - #244

Closed
jjcav84 wants to merge 1 commit into
future-agi:devfrom
jjcav84:fix/pii-redaction-scope
Closed

jjcav84 wants to merge 1 commit into
future-agi:devfrom
jjcav84:fix/pii-redaction-scope

Conversation

@jjcav84

@jjcav84 jjcav84 commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes FI_PII_REDACTION corrupting correlation identifiers and over-redacting numeric content (#195). Two changes:

  1. Key scoping. TraceConfig.mask() ran the PII scanner on every attribute value. It now scans only attributes that carry free text — input.value/output.value, message content/contents.*.text, document content, tool-call arguments/results, retrieval/reranker queries. Structured identifiers (session.id, user.id, metadata) and typed scalars pass through untouched.

  2. Card-like card detection. The credit-card pattern \b(?:\d[ \-]*?){13,19}\b matched any bare digit run — ~2% of random UUIDs, timestamps, request ids. Candidates must now be card-shaped (contiguous, or 4-6-digit groups like Visa 4-4-4-4 / Amex 4-6-5) and pass the Luhn checksum. Also fixed: the phone pattern gains a leading \b so it can no longer claim the tail of a longer digit run after the card check clears it, and card checking is restored to its original ordering before the phone pattern (phone would otherwise eat card digits first).

Issue repro, verbatim:

cfg.mask(SESSION_ID, "73630065-0794-4450-a1f9-8cc987a02b09")  # now: unchanged
cfg.mask(USER_ID, "1234567890123")                          # now: unchanged
cfg.mask(METADATA, '{"request_id": "20240115103045123"}')   # now: unchanged
cfg.mask(INPUT_VALUE, "card 4111 1111 1111 1111")           # still: <CREDIT_CARD>

Deliberate tradeoff (please sanity-check): keys outside the free-text set are never scanned — a stray custom attribute carrying an email won't be redacted. This favors identifier correctness over coverage, matching the issue's expected behavior. Widening the allowlist later is one line.

Why?

Closes #195 — silent data corruption: session/user correlation ids mangled by a PII token, invisible in the UI.

How was it tested?

  • Unit tests added / updated — TestPiiRedactionScope in python/tests/test_config.py (9 cases: issue repros, Luhn-valid cards still redact, non-Luhn runs survive, nested message-content keys still scanned, identifier keys never scanned)
  • Integration tests pass — N/A (no gateway behavior)
  • pytest tests/test_config.py — 29/29 pass; baseline failures in test_otel.py/test_context_attributes.py exist on clean main (unrelated, pre-existing)
  • Public types/env vars unchanged — FI_PII_REDACTION semantics narrowed only to correct attributes

Checklist

  • Branch is off main
  • Conventional Commits (fix(pii): …)
  • No TODOs or commented-out code
  • No secrets in the diff

Notes for reviewers

The phone-regex \b addition is load-bearing — without it 20240115103045123 (17 digits, not a card) still loses its last 10 digits to <PHONE_NUMBER>. The card↔phone ordering is restored to the pre-existing convention (card first).

@jjcav84

jjcav84 commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Bump — scopes PII redaction to free-text attributes + requires Luhn-valid card numbers. Mergeable.

@jjcav84
jjcav84 changed the base branch from main to dev November 6, 2026 00:55
PII scanning ran on every attribute value, and the credit-card pattern
matched any bare 13-19 digit run — corrupting session.id UUIDs, user.id,
and numeric metadata fields (future-agi#195).

- mask() now scans only keys that carry free text: input.value,
  output.value, message content/content-text, document content,
  tool-call arguments/results, retrieval/reranker queries
- credit-card candidates must be card-grouped (4-4-4-4 / 4-6-5 /
  contiguous) and pass the Luhn checksum
- phone pattern gains a leading word boundary so it cannot claim the
  tail of a longer digit run

Generated with Devin
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@jjcav84
jjcav84 force-pushed the fix/pii-redaction-scope branch from 64f4fff to 6843391 Compare November 6, 2026 00:56
nik13 pushed a commit that referenced this pull request Nov 9, 2026
… card shape (#244)

Fixes `FI_PII_REDACTION` corrupting correlation identifiers and over-redacting numeric content (#195). Two changes:

1. **Key scoping.** `TraceConfig.mask()` ran the PII scanner on *every* attribute value. It now scans only attributes that carry free text — `input.value`/`output.value`, message `content`/`contents.*.text`, document `content`, tool-call arguments/results, retrieval/reranker queries. Structured identifiers (`session.id`, `user.id`, `metadata`) and typed scalars pass through untouched.

2. **Card-like card detection.** The credit-card pattern `\b(?:\d[ \-]*?){13,19}\b` matched any bare digit run — ~2% of random UUIDs, timestamps, request ids. Candidates must now be card-shaped (contiguous, or 4-6-digit groups like Visa `4-4-4-4` / Amex `4-6-5`) **and** pass the Luhn checksum. Also fixed: the phone pattern gains a leading `\b` so it can no longer claim the tail of a longer digit run after the card check clears it, and card checking is restored to its original ordering before the phone pattern (phone would otherwise eat card digits first).

Issue repro, verbatim:

```python
cfg.mask(SESSION_ID, "73630065-0794-4450-a1f9-8cc987a02b09")  # now: unchanged
cfg.mask(USER_ID, "1234567890123")                          # now: unchanged
cfg.mask(METADATA, '{"request_id": "20240115103045123"}')   # now: unchanged
cfg.mask(INPUT_VALUE, "card 4111 1111 1111 1111")           # still: <CREDIT_CARD>
```

**Deliberate tradeoff (please sanity-check):** keys outside the free-text set are never scanned — a stray custom attribute carrying an email won't be redacted. This favors identifier correctness over coverage, matching the issue's expected behavior. Widening the allowlist later is one line.
nik13 added a commit that referenced this pull request Nov 9, 2026
#244 stops the phone pattern from matching digits inside a longer
identifier, so parallel.search_id is no longer rewritten to
search_<PHONE_NUMBER>. Update the stale expectation and the README note.
@nik13

nik13 commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Thanks! This change is now on dev in e917d0e as part of the traceAI release train (squashed into one commit).

@nik13 nik13 closed this Nov 9, 2026
nik13 added a commit that referenced this pull request Nov 9, 2026
#244 exempts correlation identifiers and metadata (session.id, user.id,
conversation ids, metadata) from PII redaction and stops the phone pattern
from matching digits inside a longer identifier. Hide flags and the image cap
still apply. Agno's context-privacy tests now pin that scope and keep checking
redaction of free text; discoveryengine derives the project-number expectation
from the installed core and documents both behaviours.
nik13 added a commit that referenced this pull request Nov 9, 2026
fi-security ruling on the #244 scope: identifier fields (session.id, user.id,
conversation ids) stay exempt from PII redaction (#195), but free-form metadata
is redacted when FI_PII_REDACTION is on. Tests only; they fail until core is
changed.
nik13 added a commit that referenced this pull request Nov 9, 2026
… fields

fi-security ruling on the #244 scope: session.id, user.id and conversation ids
stay exempt from PII redaction (#195), but free-form metadata, where opted-in
users put emails and phone numbers, is redacted again when FI_PII_REDACTION is
on. #244's pattern fixes (no phone match inside longer ids, Luhn-valid cards)
are unchanged. Changelog documents the #195 scope.
nik13 added a commit that referenced this pull request Nov 9, 2026
#244's (?<!\w) lookbehind keeps digits inside longer identifiers intact, but a
JSON-escaped newline, tab or return before a phone number (for example in
serialized metadata) put a word character in front of it and the number was
exported. Also start a match after a backslash escape; long identifiers after
an escape are still not phone numbers.
nik13 added a commit that referenced this pull request Nov 10, 2026
…14 and Java 1.0.3

Core 1.3.1 also ships the PII-redaction scoping fixes (#195/#244) that sat under
Unreleased, and the 1.3.0 heading gets its release date. OpenAI 0.1.14 also
ships the Azure metadata (#154), Baseten (#541), Azure Government (#502),
OpenRouter host (#547), full multi-message input (#245) and exhausted-stream
changes. Java 1.0.3 also ships #543 and #605. Notes the remaining blank
OTEL-env Authorization limitation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nik13 added a commit that referenced this pull request Nov 10, 2026
… traceAI-langchain 0.2.2, Java 1.0.3 (#636)

* chore(release): bump core, OpenAI, LangChain and Java patch versions

Synchronize runtime versions, reactor parent references and release changelogs for #155, #348, #397, #393 and #425. Dependency floors are unchanged.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* chore(release): list every change shipping in core 1.3.1, OpenAI 0.1.14 and Java 1.0.3

Core 1.3.1 also ships the PII-redaction scoping fixes (#195/#244) that sat under
Unreleased, and the 1.3.0 heading gets its release date. OpenAI 0.1.14 also
ships the Azure metadata (#154), Baseten (#541), Azure Government (#502),
OpenRouter host (#547), full multi-message input (#245) and exhausted-stream
changes. Java 1.0.3 also ships #543 and #605. Notes the remaining blank
OTEL-env Authorization limitation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Code <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PII redaction (FI_PII_REDACTION) corrupts session.id/user.id/metadata - credit-card regex matches any 13+ digit run

2 participants