Repository navigation
Axum readiness RC - #1541
Draft
ChristianPavilonis wants to merge 262 commits into
Draft
Axum readiness RC#1541ChristianPavilonis wants to merge 262 commits into
ChristianPavilonis wants to merge 262 commits into
Conversation
Fastly and Axum read request heads through parts, headers and extensions, propagate into_request errors, and convert bodies through into_body_payload. Hooks::routes returns Err(startup_error(..)) on a settings failure; core maps the root status to the matching EdgeError constructor. Axum main assembles the app with App::build and exits non-zero before binding when startup fails, and its startup_error_router is gone. Fastly keeps its startup router and hand-built App; its run_custom and serve_custom callbacks return Ok(()). The Axum route tests and the CLI proxy test now serve the real app through AxumTestServer over loopback TCP.
…egress policies Core gains edge_app.rs. install_edge_policies sets a response write deadline 60 s after egress starts and installs x-ts-version as the app-level synthesized response header set. Every adapter's Hooks::configure calls it, and Fastly also adds x-ts-fastly-version from FASTLY_SERVICE_VERSION. Fastly's hand-built app still logs a configure error instead of failing. Cloudflare and Spin read request heads through parts, headers and extensions, propagate into_request errors, convert bodies through into_body_payload, and convert ConfigValue to String at the config store boundary. Their startup_error_routers are gone: routes() returns Err, mapped by startup_error on Cloudflare and always service_unavailable on Spin. Spin opens its config store with for_label, normalizes RequestParts, and its entry point returns SpinResponse. sanitize_forwarded_headers takes a HeaderMap. Spin's outbound client detaches its tasks with spawn_local, because the repinned spin-sdk deprecates spawn. Cloudflare and Spin route tests drive the assembled app and check that EdgeZero's canonical 404 and 405 carry x-ts-version.
The Axum route tests now prove that TrustedServerApp::configure installs the synthesized response headers over real TCP. TRACE /auction gets the canonical 405 with a sorted Allow and x-ts-version. A raw OPTIONS * request, the one target the publisher fallback does not claim, gets the canonical 404 with x-ts-version.
EdgeZero PR 275 hands inbound bodies to handlers as deadline-bounded streams, so the old readers broke: POST /auction and EC batch sync read an empty body, and the sign, rebuild, request-signing, and cache purge endpoints answered 400 for any streamed body. Add http_util::read_request_body, which rejects a single valid Content-Length over the cap without polling the body and otherwise reads with Body::into_bytes_bounded. Errors map by status: 413 to RequestTooLarge, 408 to the new TrustedServerError::RequestTimeout, another 4xx to BadRequest, and a 5xx to Proxy (502). A duplicate or malformed Content-Length skips the precheck and the bounded read governs. Body bytes are never logged. Move every inbound reader onto it with its existing cap: /auction (keeps the text/plain 413), EC batch sync (keeps the body_too_large body), cache purge, first-party sign and rebuild, verify-signature, key rotate and deactivate, the six integration proxies, and the GTM beacon. Delete request_body_bytes, collect_body_bounded, enforce_max_body_size, and the GTM PayloadSizeError path. The request-signing handlers and batch sync become async, and the adapters await them. Add a shared body-read case table so each reader asserts the declared 413, the streamed 413, 408, 400, and 502 through its handler, plus Axum listener tests that sign a URL and parse an auction body sent as streams.
ScriptedHttpClient implements EdgeZero's OutboundHttpClient for core and adapter tests. It records each prepared request with its options and answers from a first-in, first-out script per method and URI, with chunked, delayed, held, and failing replies. Its forbid mode fails and records every request. TestClock pairs a hand-advanced clock with a timer factory that wakes on advance, so batch cutoffs, timeouts, and slot timing follow EdgeZero's batch ledger deterministically.
Start TestClock an hour ahead of the process clock, so a deadline built from MonotonicClock::default() is already expired on it and code that mixes clocks fails its tests. Expect a buffered body for begin and batch instead of silently recording an empty one.
RuntimeServices gains two required services, outbound_client (EdgeZero's HttpClient) and clock (MonotonicClock), beside the old http_client and backend, which stay until the old outbound layer is deleted. Every RuntimeServices::builder() call sets both. Test helpers and fixtures use a forbid-mode ScriptedHttpClient and a TestClock, and the parity suite's ForbiddenLifecycle checks on drop that no outbound request was sent. Each adapter's per-request builder takes the client from the request context with the proxy marker off, and the clock from the context. A request with no client is an internal EdgeError, not a panic. The oneshot route helpers in the Fastly, Cloudflare, and Spin tests install a forbid-mode client, as EdgeZero's request conversion installs the adapter default. The new outbound module holds the constant timeouts and caps, the buffered and streamed request helpers, and outbound_error. outbound_error maps a 413 to RequestTooLarge, a 408 to RequestTimeout, and a 400 to BadRequest only for a forwarded client request. Every other failure keeps the site's own error. The report carries the EdgeError, its budget source, and its provider failure detail, and the detail is logged at warn without provider text or URLs. No outbound site uses the new client yet.
Serve the Axum, Cloudflare, and Spin apps on AxumTestServer, each assembled under its own platform metadata with its production policies and a forbidding scripted client in router state. Add the routes_with_settings_and_outbound_client testing seam to the three adapters so that client can reach router state. Add test 19 (EZ-23 criterion 13): a 9000-byte target gets 414 with the synthesized headers from all three apps, and a bodiless GET whose inbound body is an unread stream reaches a loopback origin with no body. Add test_support::insert_outbound_client and use it in the Fastly, Cloudflare, and Spin oneshot route helpers. Check the forbidden lifecycle script explicitly at the end of each trace parity test. Drop the tower, http-body-util, and urlencoding dev-dependencies the crate no longer uses.
…routers Record the loopback origin's request headers, and assert that the bodiless GET forward carries no Transfer-Encoding and no Content-Length other than 0. Say that the router-state Axum client is staged for Task 7, when the publisher forward moves to EdgeZero. Split each adapter's router assembly into a private router_builder, so production builds no client branch. Only the routes_with_settings_and_outbound_client seam adds router state. Build the Axum route tests' settings-based servers through that seam with a forbidding script, drop the ForbiddenLifecycle drop check now that every test asserts explicitly, and fix the test_settings doc.
Port Sourcepoint, Lockr, Permutive, DataDome, DataDome Protection, and Didomi from the platform HTTP client onto services.outbound_client(), with the settings, caps, decoding, and timeouts from the spec's site map. Delete the backend registration helpers and test doubles those paths used, and move their tests to the scripted client with request readback. Reject proxy.certificate_check = false at deploy validation and ignore it with a warning at runtime, and drop it from fixtures, scripts, and CI. Keep EdgeError text out of outbound_error reports, and put the services client in router state at the Axum, Cloudflare, and Spin services seams.
proxy_request, which serves the first-party proxy, GTM, the Prebid bundle, testlight, GPT, and the JS asset proxy, now checks the initial target as before and then calls HttpClient::follow with a four-redirect limit, the 15 s per-hop timeout, and a 60 s chain deadline. A 'static redirect policy owns the allowlist and the HTTPS rule and checks each hop in the old order. Callers that do not follow redirects use send. Buffered calls keep 10 MiB (testlight 2 MiB) and the original coding, and streamed calls get 60 s. The asset proxy sends the final GET streamed and the final HEAD and the S3 HEAD preflight buffered, all in their original coding. Image Optimizer options are mapped from the resolved profile to EdgeZero's options, with Trusted Server's region aliases mapped to the seven regions, and go on the final request only. A profile the builder rejects serves the image unoptimized with a warning. Delete proxy_with_redirects, ProxyRedirectPolicy, ProxyRequestHeaders, REQUEST_BODY_HEADERS, the platform-response converters, build_asset_platform_request, AssetProxyResponse::into_response, the old test doubles, and the StubHttpClient members only these paths used. Port the redirect, allowlist, asset, and Image Optimizer tests to the scripted client, and add the deadline, userinfo, cross-origin, non-ASCII Location, send-path, readback, and SigV4 host cases.
Add an Axum-binary integration case that signs a first-party proxy URL through /first-party/sign and proxies a chain against a loopback origin. A permitted hop is followed to the final origin, a hop to a host outside proxy.allowed_domains answers 403, and a Location with userinfo answers the proxy's 502. RecordingOrigin now answers from a responder, so the chain origin reuses it. Reuse UPSTREAM_RTB_MAX_RESPONSE_BYTES for the testlight cap instead of a duplicate constant, and declare the ring feature the SigV4 TLS origin uses on the integration crate's rustls dev-dependency.
… late-registration path
7 tasks done
Collaborator
Author
|
Integrated #1540 commit db1ba33 through merge 86797e0, preserving the RC existing work. One AWS example now contains nested retained DynamoDB provisioning, protected CloudFront defaults, explicit EC configuration/task IAM and the checksum regression. Hostname/cookie scope and genuine consent transport are documentation-only operator responsibilities; optional production additions remain deferred. Combined RC local checks pass: 3,437 core unit cases plus 18 envelope/storage/digest cases, 110 AWS-feature Axum cases, targeted strict Clippy, docs build, formatting, workflow tests and changed-line domain lint. The Terraform example files exactly match the separately tested PR tree, so the 8+83+1 mock-plan evidence is reused. Hosted AWS offline checks pass at the PR commit: https://github.com/IABTechLab/trusted-server/actions/runs/43306496934 . Full hosted RC checks are now running and are not yet claimed passed. DynamoDB Local and real Next.js cases remain prerequisite-dependent ignores; live AWS/browser acceptance remains unverified. No main merge or deployment. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
83d7a6296d285e135c4ea70021b19e6c8dca7e9f, not refreshed draft462.Exact inputs
Head:
c7f8ca7e1ad09bdbae8de40ceaa50d658ef5b7dc.Tree:
b3cb8e8dd02390d9822880bee38196cd538520dc.Review base:
baseline/demo-main-20261110at historical TS main2dd968aeebeb2cc0d5110115f64e0ad58e2e1d13, already merged into original demo2423a2ed.The user selected baseline cleanup and deferred current-main forwarding/provenance compatibility. This branch intentionally omits newer-main functionality; do not retarget/merge it into current main as an ordinary forward integration. Seven EdgeZero declarations and ten locked git sources use83d. Registry package/version sets match original final demo2423; Cargo reconciles four permitted dependency edges within that identical set.
Review sequence
b1216763aaffb0d0a41cfba70a2a8e61218fc568207dbe2672968af14eefa67b9e1ff2cf46f621112ee7f68fcfc7f8ca7e1The existing ten heads/bases and ready states are unchanged. New baseline port1545 requests aram356, prk-Jr and dhruv8sh. Source1498 remains main-based atf22a83fce without head/base/readiness changes. Integration1541 remains draft regardless of constituent readiness.
Relay1498 inclusion
The user authorized source1498 in this stack/RC. Baseline port1545 applies its feature-only delta from
f22a83fceac98767979ac0dd406701e4a000c1db, not its newer-main ancestry.integrations.prebid.relay_soft_vendor_exceptionremains false by default and testing-only; omitted/false leave publisher rules unchanged. Explicittrue retains Purpose2, mapped GVL vendor and unrelated-bidder controls. No operator/deployment input is enabled.The replay exposed the previously observed automatic-CMP ownership defect when explicitly enabled: later publisher denial plus stale CMP grant caused a stubbed identity request/cookie. The existing guarded-setter correction fromd827b6c7 applies cleanly and is included with normal/deferred/throwing-subscriber regressions. Public ownership transfer, listener retirement, zero requests/cookies after denial and finally restoration are exercised. This extra correction is in port1545, not claimed already present in source1498. No new owner/ledger, public bypass, production timer, retry, dependency pin or new consent policy.
34 guard/ownership and11 actual production-artifact cases pass, along with180 focused Rust Prebid cases. The baseline shim measures41,948 chars under a narrow42,500 cap with Prebid-free/separate-artifact controls retained; the newer-main54KB allowance is not copied. Original red receipts are retained.
Scope and corrections
Existing native snapshots, body/head/response ownership, supported deadlines, EC generation/grants/consent and protected CloudFront Geo behavior are retained. Unset caller deadlines delegate to EdgeZero through its existing dispatch_budget API instead of the original TS30-second fallback; three regression tests cover unset/default, explicit120-second and earlier explicit deadlines. No new production timer, retry, result ledger, unlimited workaround or raw HTTP fallback.
Routine fixes retain failed receipts: redundant test imports, HTTP table formatting and documentation host/port placeholders. Hosted docs build then caught a relative Terraform link outside the site; Native EC8b11dbad3 replaces it with the repository path without disabling dead-link checking. Docs ESLint/VitePress build and format pass at their changed inputs.
A hosted no-listener assertion failed while the child logged expected startup rejection. The old log lacks socket endpoints, so its exact subtype is unconfirmed. A controlled Linux reproduction proves successful TCP self-connection with no listener. Native8490c691d changes only the acceptance fixture to exclude equal local/peer endpoints; real-listener/self-connection controls pass. Nonzero exit, diagnostic, secret-redaction and the original startup deadline remain mandatory. Native default and final9356 all-feature acceptance9 cases/strict lint/fmt pass. No assertion waiver.
Older browser CI failures show Playwright1.58.2 refusing ubuntu-latest's new Ubuntu26.04 before installation. Nativeb1216763a pins only the browser job to ubuntu-24.04, preserving dependency versions, tests and assertions. Direct Actionlint passes at native/final inputs; actual supported-host browser execution remains pending. The only9356 ->ee7 difference is this CI host/comment.
Checks and outcomes
sha256:8878057fb97f2e1d13ca1049acd18c4d7d21f67d5b1403851d29f898019fa435, linux/amd64 UID/GID10001, startup rejection, health/auth/TSJS, read-only root/writable mounts and in-flight SIGTERM drain. Prior images are retained separately and not substituted for this new JS/config input. No image push.Passing receipts are reused only where relevant inputs remain unchanged, not claimed as a fresh whole-tree rerun. Evidence root retains ts-baseline-final-proof.md, ts-baseline-combined/, ts-baseline-check-fixes/, ts-baseline-image-probe-fixed/, startup-control and docs logs, and the failed hosted receipts. All local parent runners are complete.
Limits and preserved work
The existing ten feature PRs retain their observed ready states without head/base changes. New port1545 and the advanced RC await exact-current-head hosted checks;1545 is draft and1541 remains draft. Local pass is not a hosted all-green claim. Obsolete own CI runs without failed jobs were cancelled to free the queue; recorded failed jobs remain retained. Cancellations/ignores/unrun checks are not passes. Historical-base main-only CodeQL is not triggered and is not claimed passed.
No external AWS/DynamoDB account, fleet/cloud durability, real publisher, external Next.js or combined wrapper-image/ECS-volume qualification. Geo is default-off protected CloudFront; KV Geo454 stays standalone prototype-only. Operational Terraform remains in trusted-server-deployer; TS examples contain fictional inputs only.
Original work, dirty Prebid builder inputs, recovery refs and old RCs remain preserved. Current-main1525/1526 ready and1527/1528 draft retain their exact heads/bases. EdgeZero stack465/draft462 and deployer40 are untouched. No1499/independent1506 implementation, unrelated privacy/group/codec/phase experiment, new worker, deployment, live input switch, main merge or old-RC closure. Relay work is limited to source1498 and the existing ownership correction described above.
Closes / spec
Refs #1530 and #1498; do not close the tracker or source PR from this draft publication. This is existing-demo extraction with narrow caller/build/fixture fixes, not a new forwarding or consent design. Historical specs/reviews remain input-bound records, not backdated approval of this combined source.
Qualified semantic prerelease
Published and verified lightweight tag v0.1.1-rc-axum.202611.3 at 7dd2e30, tree d1ebc72adaf108e5bc520b536d832fa23456a0f3. All 33 exact-head RC hosted checks pass. Across the twelve current PR heads, all 568 checks are successful. This PR remains draft and targets main; the ten feature PRs remain ready in native stack #1564.
The combined tree retains every prior feature head, previous public RC, current main 06e51be, direct tag-bound EdgeZero interfaces, relay consent ownership, the latest Sourcepoint TCF mirror and consent-only safe cookie reconstruction.
Local qualification includes 3,440 Core passes with two ignored, 111 native passes with five ordinary ignores, and 3,372 JS passes with one skip. The approved shim budget is 54,000 characters for the measured 53,628-character artifact. No other artifact assertion changed.
Hosted failures were repaired without weakening assertions. Header comparisons keep exact equality without printing credentials; the later-batch browser fixture avoids a deadline-boundary assumption; isolated Cloudflare fixtures use their actual public origin and correctly resolve signed URLs; native port allocation binds without listening. CLI browser CI pins released Chrome for Testing 157.0.8103.78, with actual macOS browser fixtures passing. Production shutdown/deadline budgets are unchanged.
Fresh source-bound production-feature image, exact CLI/envelope, AppConfig Agent/Moto snapshot/restart/fail-closed controls, nine DynamoDB identity/fault/lifecycle controls and the literal wrapper pass. Images remain local and unpublished. Actual runtime/browser/parity/deadline/EOF and celld proofs remain tied to their tested inputs; skips and ignores are not passes.
Limits remain explicit. Viceroy host-worker cancellation warnings and the reproduced pre-existing extra combined Core/CLI/codegen preserve_order-sensitive golden mismatch are retained separately. Synthetic local proof is not live AWS/IAM, fleet durability or genuine publisher/CMP qualification.
Old tags are unchanged. No main merge, GitHub release publication, image publication or deployment occurred. Tag-triggered binary builds run separately; signing/public release publication is disabled.