[yummybomb]

Stacked on #112.

Summary

Two additions to bench/: checkout intents, and per-tool failure reporting.

Checkout intents. The bench so far only measured single lookups, and only registered read-only tools, so it couldn't measure multi-step flows such as booking a flight, where the tools should help most. A new prepare intent:

• lists actions: tools that change site state, registered alongside the read-only ones;
• has a check: a read-only tool, input and rules. Once the agent is done, the runner invokes that tool in the browser and grades its output. It does this in both arms; control gets the tools registered only at that point. So the end state is read from the site rather than trusted from the agent's answer.

The first one is delta-cart-main-classic: put the cheapest Delta Main Classic one-way fare in the cart and stop on Trip Summary, checked with get_trip_summary (step, passengers, one slice, origin, destination, cabin). The system prompt's checkout rule becomes "do not pay, place an order or complete a booking, and go no further into a checkout than the task asks".

Tool failures. Each WebMCP invocation now records the tool's own status and an error message. Before this, invoke ok counted any invocation that finished as a success, including tools that returned {"status": "error"}, and calls rejected for a stale tool_ref were recorded as unknown. The report adds a table per tool: calls, failures, and the most common errors.

Testing

• node --test "bench/**/*.test.mjs": 66 pass. New tests cover prepare-intent validation, action registration, end-state grading, and failure capture for three cases: the call failed, the tool returned an error, and the MCP server rejected the call.
• Live: 3 pairs of delta-cart-main-classic on Haiku 5.5 at low thinking, stealth browsers, $0.13. All 6 trials reached Trip Summary with DL 1162, Main Classic, ATL→MCO, $143.40 in the cart, verified by get_trip_summary. Treatment was faster in all 3 pairs (1.3×). The failure table shows delta.com/search_flights failing all 6 calls with UI_CHANGED: missing button, which is why agents finished by clicking through instead.
• Re-summarising the transcripts of the last 16-intent run with this code gives real errors: Ticketmaster's tool reporting bot-protection 403s, Kroger failing to capture store context, AMC agents passing unknown input fields, and Delta's search_flights failing on every call.

Review fixes

• The answer must match the end state. A check can list same_as_answer path pairs; delta-cart-main-classic requires the answer's amount_due to equal the cart's, so the agreement check on amount_due compares what was actually in the cart rather than what the agent reported.
• The tool failures table says how many invocations made through Playwright or the REPL have no recorded result and are not counted.

[yummybomb]

Folded into the stack: the tool failure reporting moved into #112 (it fixes #112's own invoke ok), and the checkout intents moved into #118, where they are graded from the page instead of with check.tool. #117 now sits directly on #112.