[yummybomb]

Stacked on #117. Folds in #115's checkout intents and #120's page-based grading, so the end-state check is reviewed once, in its final form.

Summary

Lets bench intents walk a site's flow part way, or through a checkout up to the button that pays, and grades where the agent left the browser from the page itself. Adds three such intents.

Prepare intents. The bench so far only measured single lookups with read-only tools, so it could not measure multi-step flows such as booking a flight, where tools should help most. A prepare intent has a check, which the runner applies to the page the agent left, in both arms, never through the tools under test:

• its rules see the page's url, title and visible text;
• extract maps names to regexes whose first group is read from that text as extracted.<name>, with amounts parsed as numbers;
• fields maps names to CSS selectors of inputs, read as fields.<name>, with missing and invalid lists, for pages such as a third-party checkout form;
• same_as_answer lists {answer, end_state} path pairs that must match, so the answer reports what is actually on the page.

The page is read through Playwright on Kernel, or the DevTools protocol locally, and saved as end-state.json.

Actions. A domain's domain.json lists actions, tools that change site state, each with a reason, registered for its prepare intents; and checkout_actions, registered only for intents that allow checkout. Delta's actions are select_fare and select_fare_experience; its seat, Review and Pay and trip protection tools are checkout_actions, so the cart-only intent never gets tools that go past Trip Summary. Blue Bottle's are add_to_cart, and continue_to_checkout for checkout. Tools that sign in, pay, book or send stay out.

Policy. The system prompts keep "never pay, place an order or complete a booking" absolute and change the other two rules to "unless the task says you may": carts and checkout, and entering personal or payment details (only the ones the task gives). A prepare intent opts in with allow:

• checkout: add to the cart and go through checkout up to, never including, the button that pays or books;
• payment_details: enter the details the task gives, which must be test data.

The runner adds the same fixed sentences to both arms' prompts.

Guard. An intent that allows checkout must have a guard: hosts, a button-text regex, and optionally selectors, form_submit and a request regex. The runner builds it into a small Chrome extension (named by a hash of its content, uploaded once and reused) and loads it in both arms' browsers; locally it loads as a second unpacked extension. Its script runs in the page's own world, in every frame, from document start, so it survives navigation, unlike an init script added through playwright execute, which I checked does not. It blocks clicks on buttons whose text matches, or on anything inside an element matching selectors, and, with form_submit, a form's submit buttons, submit events, form.submit()/requestSubmit() and Enter in form fields. Its declarativeNetRequest rules block requests matching requests from any frame, and with form_submit, POST navigations started on the guard's hosts. That covers Playwright's page.evaluate, which runs in an isolated world where the page-world overrides are not installed. Blocked clicks and submissions are counted in a cookie on the page's host; the runner reads the cookies of the whole browser after the agent, so an attempt counts even if the agent then left the page, and a trial with any is crashed ("tried to submit"). This follows the espresso runner's submission guard and is defense in depth: checkout intents also use a test card or a test account with no saved card.

Signed-in intents. An intent can carry its own profile and signed_in_check, so one signed-in intent doesn't force a profile on the domain's other intents. A signed_in_check can also take rules, for a tool that reports signed-out state instead of returning an error.

Intents.

• delta-cart-main-classic: put the cheapest Delta Main Classic one-way fare in the cart and stop on Trip Summary, graded from Trip Summary's text: the route and One Way header, Cabin class/Cabin Name equal to Main Classic, 1 passenger, and Amount Due matching the answer. It reads both Trip Summary variants Delta serves (/purchase/flights/ and /completepurchase/).
• bluebottle-checkout-espresso: the espresso task. One bag of Opascope Espresso as a one-time order; checkout filled with fictional details and test card 4242 4242 4242 4242; all ten checkout fields read from the page and checked. Checkout's native fill_payment_form belongs to the site, so both arms can use it.
• delta-checkout-signed-in: on the bench-delta profile, book the cheapest Main Classic one-way fare up to Review and Pay's Payment section (fare, Trip Summary without seats, prefilled passenger details, trip protection declined), checked from the page's text and its Amount Due. The sign-in check is get_page_context with signed_in: true.

Also: fill_payment_form returns awaiting_submission, its success state when it doesn't submit, which invoke ok and the tool failures table counted as a failure.

Testing

• node --test "bench/**/*.test.mjs": 92 pass on Node 26. New tests cover a trial whose guard count or end state cannot be read, which aborts (it used to be graded as if nothing was blocked, or wrong), sign-in check rules validated like intent rules, opt_in intents, and validation of check (only fields, extract, rules and same_as_answer; one capture group per extract pattern), allow, guard and intent-level profiles, actions and checkout_actions reaching the right intents, the prompt sentences, the guard extension's manifest and content-hash name, page-text extraction (including a group that takes no part in the match), page-based grading and same_as_answer, grading the espresso intent against page states (formatted card, expiry and phone values; wrong card; invalid field; wrong URL), and awaiting_submission. The stub run test covers a guarded checkout intent: the extension is uploaded once and loaded for the smoke gate and both arms, a trial whose guard blocked an attempt is crashed, and a second run reuses the extension. Dropping the extension from browser creation, or ignoring the count, fails it.
• Guard, live on Blue Bottle's checkout (a Stripe live-mode session) in a Kernel browser, with the extension loaded: a click on the submit Pay button and form.requestSubmit() were blocked and counted, Enter in the email field was blocked and not counted, and the page stayed on checkout.
• Guard and evaluate, locally in headless Chromium 154: the guard loaded next to the xwebmcp extension; Pay and a form's submit button were blocked; a Pay with card button outside a form was not.
• Real runs on Kernel:
  • delta-cart-main-classic, pi with openai/gpt-5.4-mini: both arms reached Trip Summary with DL1150, Main Classic, $173.40 (treatment 61s, control 243s), but my first amount regex missed. The saved page text showed two Trip Summary variants and that "Main Classic" also appears in the Comfort upsell; the check now reads the cabin line, and regrading both saved pages gives correct. A second run: treatment correct in 86s; control hit the turn cap after 409s on the results page.
  • Earlier, under a tool-based check that read the cart with get_trip_summary (since replaced by the page check): 3 pairs on Haiku 5.5 at low thinking, all 6 trials reached Trip Summary with DL 1162, Main Classic, ATL→MCO, $143.40 in the cart; treatment was faster in all 3 (1.3×).
  • bluebottle-checkout-espresso, pi with openai/gpt-6-luna at medium thinking, 2 pairs: all 4 trials correct, every field verified, no submit attempts. Treatment used find_product → get_product → add_to_cart → get_cart → continue_to_checkout → fill_payment_form and finished in 0:59 and 1:18; control in 1:51 and 1:47, filling the form with Playwright. About $0.04. Re-run with openai/gpt-5.4-mini, 1 pair: both arms correct, 0 blocked attempts; treatment 0:43, control 5:34.
• get_page_context on www.delta.com in a signed-out Kernel browser returns signed_in: false, so the Delta sign-in check catches a lapsed session.
• Not run: delta-checkout-signed-in. It needs a bench-delta profile on a Delta test account that cannot pay: no saved card, zero SkyMiles, and no eCredits, gift cards or certificates (see "Signed-in trials" in the README). It is opt_in, so default runs skip it even once the profile exists, and only --intent delta-checkout-signed-in runs it (stopping if the profile is missing). Rehearse it once on the live site after the profile is created, then drop opt_in. Its guard's selectors and its page check are also untested on Delta's live Review and Pay.

Guard fixes

• The count survives leaving the page. It was kept in the page's sessionStorage and read on the page the agent ended on, so a blocked click on the checkout host followed by navigating to another origin read as 0 (reproduced in headless Chromium). It is now a cookie on the page's host, summed over context.cookies() on Kernel and Storage.getCookies locally. Checked on a Kernel browser: a blocked click on example.com still read 1 after navigating to iana.org.
• Network-layer blocking. On Kernel, form.submit() called from page.evaluate submitted the form despite form_submit, and fetch to a requests URL from page.evaluate went through, because Playwright evaluates in an isolated world. requests is now a declarativeNetRequest rule (validated as RE2: no lookarounds or backreferences), and form_submit adds one blocking POST navigations from the guard's hosts. Re-checked on Kernel: both are blocked; a GET form.submit() still goes through. Neither network block is counted.
• Delta's guard also blocks [data-cy^="complete-purchase"] (the Complete Purchase, PayPal and Apple Pay buttons), and its text regex matches a label that starts with the button's name rather than equals it, so Complete Purchase $143.40 is caught. It still has no form_submit, since Review and Pay's Save & Continue buttons may be form submits. Checked in headless Chromium against a page with those buttons; not yet checked on Delta's live Review and Pay.