feat(ai-guard): apply sensitive data redaction from evaluate responses

[manuel-alvarez-alvarez]

What Does This Do

/evaluate responses may now carry a top-level redaction_replacements array of {path, replacement} entries. Its presence is the signal to redact — there is no separate flag. The service returns the fully redacted string per path, so the tracer resolves the path and overwrites that one string verbatim: no offsets, no encodings, no merging.

• Evaluation.getMessages() returns the evaluated messages, redacted when redaction was applied; getRedactionReplacements() reports what was applied. Callers should forward getMessages() to their LLM provider.
• Meta-struct reports the redacted conversation, including on the blocked path. AIGuardAbortError deliberately carries no messages.
• DD_AI_GUARD_REDACTION_ENABLED (default true) is a kill switch, checked before any redaction work.
• New ai_guard.redacted span tag and a redacted tag on the ai_guard.requests telemetry metric. Both are omitted entirely when the kill switch is off, so an absent tag means "redaction is off" and stays distinguishable from false.

Redaction is applied copy-on-write — the caller's list is never mutated, only redacted messages are rebuilt, and the same list reference is returned when nothing changed. Unresolvable paths, non-string targets, and missing or conflicting replacements are skipped fail-safe and counted, never raised.

Motivation

Until now, AI Guard's backend applies a Sensitive Data Scanner (SDS) and returns matches in the sds_findings field of its API response. Those matches are purely informative today: they are surfaced through the SDK and presented in the UI, but nothing is transformed.

This PR extends that capability to sensitive data redaction. Sensitive data is removed from LLM interactions (before a prompt reaches the provider, and on the provider's responses), and the same redaction is applied to the data sent to the backend for use in the UI.

Additional Notes

Implements the tracer side of the AI Guard Sensitive Data Redaction RFC.

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels
• Avoid using close, fix, or any linking keywords when referencing an issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors
• Once approved, use merge queue to merge the PR

APPSEC-69390

[datadog-datadog-prod-us1-2]

An enabled redaction request that fails before response processing has no redaction tag on its span or error metric. The missing tag makes the request look like the kill switch is off.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 49d1167 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 49d1167c8e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[IlyasShabi]

I think we should also report redaction_error telemetry on failure

[manuel-alvarez-alvarez]

Where is it defined in the RFC? (I couldn't find it 😓)

[IlyasShabi]

I was mentioned on the tracer recommended behavior

If two entries for the same path carry different replacement values (a backend bug), the tracer skips that path and records a telemetry error rather than guessing

But not listed on telemetry part.

[manuel-alvarez-alvarez]

Can we update the RFC or add and addendum?, just to set redaction_error as the tag for this particular case

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.07 s	13.98 s	[-0.3%; +1.5%] (no difference)
startup:insecure-bank:tracing:Agent	12.93 s	12.99 s	[-1.2%; +0.3%] (no difference)
startup:petclinic:appsec:Agent	17.05 s	17.47 s	[-6.7%; +1.9%] (no difference)
startup:petclinic:iast:Agent	17.45 s	17.54 s	[-1.3%; +0.3%] (no difference)
startup:petclinic:profiling:Agent	17.31 s	17.41 s	[-1.6%; +0.4%] (no difference)
startup:petclinic:sca:Agent	17.12 s	16.65 s	[-3.5%; +9.1%] (unstable)
startup:petclinic:tracing:Agent	16.68 s	16.67 s	[-1.0%; +1.1%] (no difference)

Commit: d8d7fb79 · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.