Make auction creative rewriting optional

[ChristianPavilonis]

Summary

• Make winning-bid creative rewriting configurable while keeping server-side sanitization mandatory.
• Preserve rollback compatibility and make the operator migration, privacy, and rendering consequences explicit.
• Fix protocol-relative rewrite-exclusion matching and record the associated first-party signing validation behavior.

Changes

File	Change
CHANGELOG.md	Document optional rewriting and protocol-relative exclusion behavior.
crates/trusted-server-cli/tests/config_env_overlay.rs	Cover migrated overlays, default blob omission, and clean local config diffs.
crates/trusted-server-core/src/auction/README.md	Distinguish mandatory sanitization from configurable rewriting.
crates/trusted-server-core/src/auction/endpoints.rs	Document /auction sanitization and rewrite behavior.
crates/trusted-server-core/src/auction/formats.rs	Use the centralized auction creative processing policy and cover both modes.
crates/trusted-server-core/src/auction/orchestrator.rs	Update the exhaustive configuration test literal.
crates/trusted-server-core/src/auction_config_types.rs	Add the default-true setting and document rollback-safe serialization.
crates/trusted-server-core/src/config_payload.rs	Test and label the legacy blob schema snapshot.
crates/trusted-server-core/src/creative.rs	Normalize protocol-relative exclusions and centralize sanitize-then-rewrite processing.
crates/trusted-server-core/src/proxy.rs	Prove proxy response rewriting remains independent and reject excluded signing targets.
crates/trusted-server-core/src/settings.rs	Parse the setting and warn when enabled auctions disable rewriting.
docs/guide/auction-orchestration.md	Document sanitize-only and sanitize-and-rewrite modes.
docs/guide/cli.md	Document typed config validation, diff, and push flow.
docs/guide/configuration.md	Document setting scope, migration, rollback, and environment overrides.
docs/guide/creative-processing.md	Explain privacy, direct-resource, runtime, and rendering effects.
trusted-server.example.toml	Add and explain the operator setting.

Closes

Closes #914

Test plan

• cargo test-fastly && cargo test-axum
• cargo clippy-fastly && cargo clippy-axum
• cargo fmt --all -- --check
• JS tests: cd crates/trusted-server-js/lib && npx vitest run
• JS format: cd crates/trusted-server-js/lib && npm run format
• Docs format: cd docs && npm run format
• WASM build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
• Manual testing via fastly compute serve
• Other: cargo test-cloudflare, cargo test-spin, all target-matched clippy aliases, and ./scripts/test-cli.sh

Checklist

• Changes follow CLAUDE.md conventions
• No unwrap() in production code — use expect("should ...")
• Uses tracing macros (not println!) — N/A; this project requires log macros.
• New code has tests
• No secrets or credentials committed

[aram356]

Summary

The runtime branch itself is well scoped: sanitization remains mandatory, rewriting is gated after sanitization, both modes have substantive tests, and /first-party/proxy remains independent. I am requesting changes for rollback compatibility and two operator-facing configuration/privacy contracts described in the inline comments.

Non-blocking

🏕 camp site

• Update the internal auction README: crates/trusted-server-core/src/auction/README.md:139, :251, and :382 still describe creative rewriting as unconditional. Please consider updating those passages to distinguish mandatory sanitization from default-on, configurable rewriting.

📌 out of scope

• Track the pre-existing iframe[srcdoc] sanitizer gap: the generic handler at crates/trusted-server-core/src/creative.rs:385 neither removes nor recursively sanitizes srcdoc, while the normal renderer grants allow-scripts and allow-same-origin at crates/trusted-server-js/lib/src/core/render.ts:14. This exists at the base SHA and should not block this PR, but it deserves a security follow-up that removes srcdoc and adds regression coverage through both rewrite modes.

👍 praise

• The implementation keeps sanitization strictly before the configuration branch, avoids logging creative contents, covers default/disabled behavior and legacy blob loading, and verifies that proxy HTML/CSS rewriting is independent. No new dependency, OS API, or WASM-incompatible construct is introduced.

CI Status

• fmt and all adapter/target clippy checks: PASS
• Rust tests and builds (Fastly, Axum, Cloudflare, Spin, parity, CLI): PASS
• JS formatting and Vitest: PASS
• integration, browser, Fastly EC lifecycle, and CodeQL checks: PASS

[ChristianPavilonis]

Implemented and pushed the requested review fixes in 6fcbee0d.

Also addressed the non-blocking internal auction README cleanup. The pre-existing nested iframe[srcdoc] sanitizer gap remains out of scope for this PR and is now tracked in #929.

Validation completed locally:

• all Fastly/Axum/Cloudflare/Spin test aliases
• native CLI tests and cross-adapter parity
• all target-matched clippy aliases plus CLI clippy
• Rust/JS/docs formatting and Vitest

[prk-Jr]

Summary

Adds a default-true [auction].rewrite_creatives setting: creative sanitization for POST /auction stays mandatory, while first-party resource/click rewriting and TSJS injection become optional. Backward-compatibility handling (default omitted from serialized/legacy blobs, explicit false preserved, legacy schema round-trip, EdgeZero env-overlay pre-existing-leaf quirk) is thorough and well tested. No blocking issues found.

Non-blocking

⛏ nitpick

• "unre-written" wording: docs/guide/configuration.md and docs/guide/creative-processing.md repeat the hyphenation "unre-written" (e.g. "sanitized but unre-written HTML") several times. Suggest "not rewritten" or "unrewritten" for readability — no behavior impact.

👍 praise

• Backward-compatibility test coverage: the skip_serializing_if default-omission design plus tests across settings.rs (TOML omitted/explicit-false), config_payload.rs (legacy JSON blob round-trip and legacy-schema deserialization), and the new CLI integration test (config_env_overlay.rs, proving the EdgeZero "env overlay only overrides pre-existing leaves" quirk is handled) directly cover the real rollback/migration edge cases operators will hit. The proxy.rs test proving /first-party/proxy rewriting stays independent of the new flag is a good isolation check too.

📝 note

• Change table is stale: the PR description's file table lists 12 files; the actual diff touches 16, including creative.rs (see inline comment — an independent exclusion-matching bug fix bundled here), crates/trusted-server-cli/tests/config_env_overlay.rs, crates/trusted-server-core/src/auction/README.md, and docs/guide/cli.md. Worth syncing the table with the real diff before merge.

CI Status

• fmt: PASS
• clippy (fastly/axum/cloudflare native+wasm/spin native+wasm): PASS
• rust tests (fastly/axum/cloudflare/spin/cross-adapter parity/CLI): PASS
• js tests (vitest): PASS
• js/docs format: PASS

[aram356]

Summary

Narrowly-scoped, backward-compatible feature with strong test coverage (default, explicit false, TOML parse, blob round-trip, legacy-schema acceptance, CLI env overlay, and a regression test proving /first-party/proxy rewriting is unaffected) and careful documentation. Two items need attention before merge: an undeclared drive-by behavior change in to_abs, and the fact that the rollback-critical skip_serializing_if carries no in-code rationale. Both are inline.

Blocking

🔧 wrench

• Undeclared behavior change in to_abs: moving the exclusion check onto the normalized URL is a real fix — is_excluded never matched protocol-relative URLs because url::Url::parse fails on //host/path — but it is unrelated to rewrite_creatives and appears in neither the PR description nor the CHANGELOG. It also silently tightens the /first-party/proxy tsurl validator (proxy.rs:1633), which is untested. (crates/trusted-server-core/src/creative.rs:53-73)
• skip_serializing_if is load-bearing but unexplained: AuctionConfig is deny_unknown_fields, so omitting the default is what keeps pushed blobs readable by an older binary during rollback. That rationale exists only in docs and a TOML comment, so a future cleanup of the attribute silently breaks rollback. (crates/trusted-server-core/src/auction_config_types.rs:15-19)

❓ question

• Client-side behavior with rewrite_creatives = false: disabling also drops data-tsclick and the tsjs-unified.min.js runtime injection. The docs frame this purely as a privacy trade-off. Is the creative render bridge tolerant of a missing runtime, or does the creative fail to size/render? If rendering depends on it, that is a functional consequence and belongs in the warning block in docs/guide/creative-processing.md.
• ts config diff with the leaf present: raised inline on trusted-server.example.toml:119.

Non-blocking

♻️ refactor

• Gate the flag inside creative: crates/trusted-server-core/src/auction/formats.rs:256-268 (inline).
• Warn when the privacy default is off: crates/trusted-server-core/src/auction/formats.rs:258 (inline).

🤔 thinking

• Hand-mirrored LegacyAuctionConfig will drift: crates/trusted-server-core/src/config_payload.rs:52-72 (inline).
• CLI test does not pin the working directory: crates/trusted-server-cli/tests/config_env_overlay.rs:36-63 (inline).

📌 out of scope

• /first-party/proxy sign handler bypasses to_abs for protocol-relative URLs: the //-prefixed branch builds the absolute URL inline and never calls to_abs, so it skips the exclusion check entirely — the exact inconsistency this PR fixes on the creative path. Pre-existing; worth a follow-up issue now that the sibling path was corrected. (crates/trusted-server-core/src/proxy.rs:1624-1632)
• [debug].inject_adm_for_testing embeds raw, unsanitized adm: correctly documented here as debug-only, but it remains the one creative path with neither sanitization nor rewriting. Follow-up issue. (crates/trusted-server-core/src/publisher.rs:2144-2147)

⛏ nitpick

• rewrite_mode string is unnecessary: crates/trusted-server-core/src/auction/formats.rs:264-268 (inline).

Verification notes

• The docs change from "env vars baked at build time" to "typed CLI overlay" was verified accurate: no TRUSTED_SERVER__ handling remains in crates/trusted-server-core/build.rs.
• [debug].inject_adm_for_testing referenced in the new docs exists as described (settings.rs:1923).
• convert_to_openrtb_response has exactly two production callers, both in auction/endpoints.rs (219, 340), so the new flag's blast radius matches what the docs claim.

CI Status

Taken from the PR's own checks — not re-run locally.

• fmt: PASS
• clippy / CodeQL (Analyze (rust), Analyze (javascript-typescript), Analyze (actions)): PASS
• rust tests (cargo test, axum native, cloudflare, spin, ts CLI native, cross-adapter parity): PASS
• js tests (vitest), format-typescript, format-docs: PASS
• integration tests, Fastly EC lifecycle, browser integration tests: still pending at review time