[abrichr]

Stacked on #1335. This branch contains #1335's current head (4999d26) and current main. Until #1335 merges, the diff against main includes #1335. This PR's own changes are openadapt_flow/agent_guard/, benchmark/agent_guard/, docs/GUARD_AN_AGENT.md, docs/commands/11-guard-an-agent-you-already-run/, the guard tests, the confirm_checks/proposed() change in compile_agent/pipeline.py, and the doc and capability rows.

What it adds

openadapt-flow guard: a way in for developers who already run a browser agent (a Playwright script, Playwright MCP, browser-use, a computer-use agent with a Chromium). They keep the agent. The one change is a DevTools port on the browser it launches.

openadapt-flow guard --cdp http://127.0.0.1:9222 --config guard.yaml --task task.json --out runs/task-0001

Run 1, guarded (openadapt_flow/agent_guard/):

• The guard connects to the agent's tab over CDP on its own connection and pauses requests with the Fetch domain. POST, PUT, PATCH, DELETE, unknown methods and GraphQL mutations to the app's origin are held as saves. Reads and GraphQL queries go on at once. Writes to other origins are blocked unless guard.yaml lists them under not_saves.
• For each save it runs the guarded-save contract (feat(flow): guarded save around any agent's write #1258) on the agent's behalf. The record on screen must be the one the task names (check_identity), and the baseline is read. The task is committed once, so a second save is blocked. With approval: each_item, a person approves first. After the save, verify-run reads it back.
• A blocked save gets a 409 with a JSON body, or a page for form posts, plus X-OpenAdapt-Guard and a role="alert" notice on the agent's page that names the fields that didn't match. A verified save leaves nothing on the page, so run 1 is recorded as the app drew it.
• Fail-closed by default. Saves still paused when the guard stops or errors are blocked before the CDP session detaches, because Chromium lets paused requests through on detach. --observe-only never blocks; it logs what it would have blocked and still reads the save back.
• The attach-mode recorder (record --browser-cdp-endpoint) records run 1 with network capture, on its own thread and connection. A recorder refusal can't stop the guard. An earlier version shared one connection, and a recorder refusal ended the guard and let a wrong-patient save through. The benchmark caught that, and it is fixed here.

Run 2, compiled (agent_guard/handoff.py): a run that reads back VERIFIED becomes a JIT draft in the teach dir, the same place serve --teach (#1335) uses:

• Compiled the way compile-agent compiles. compile proposes an API step when the save replays cleanly; the GUI steps are the fallback.
• Run 1's record change is checked again with check_run_record.
• The record check comes from scaffold-verifier (feat(flow): build a working record check from the app's own reads #1284), built from the app's reads, teach.db, or the verify.yaml read. It is pinned to the record's identity. If it misses a fault, it is kept for review and the before/after proposal is used instead.
• The draft is recorded as made by the attached agent with no agent principal, so only a person or an admitter key can admit it.
• A person confirms the check and admits with compile-agent admit. serve --teach --teach-dir then serves it as taught_<name>.

One change to shared code (compile_agent/pipeline.py, confirm_checks): a check built from several reads may claim the new record once per read, routed to that read. Every claim must still claim exactly one record, pin every expect field and read back every input. Extra expected_count: 0 "nothing else changed" checks are allowed. Each check keeps its surface= route, and the proposal's own verifier: section is used, with the reads shown in proposed(). A proposal with one plain claim, which is what the existing teach path writes, is handled exactly as before.

Measured on MockMed (synthetic, no paid model calls)

The page is MockMed's SPA in front of the effect_e2e SQLite store. The outside agent is a scripted Playwright agent with a 1 s pause before each of its 6 actions. There are 15 scenarios × 3 arms × 2 trials = 90 runs, one browser at a time, each judged by the effect_e2e ground truth. Both trials gave the same outcomes. Details: packages/flow/benchmark/agent_guard/RESULTS.md, raw rows in results.json.

Load: the host load average was well above 12 during the measurement, and the measurement (11:38 to 12:15) overlapped arm 5's timed run, which started at 11:26: 16 to 249 (median 29) for the matrix, 19 to 23 for the run 1 / run 2 pairs. Every row records its own load. The times are a same-load comparison of run 1 with run 2, not absolute speeds.

• Setup the developer adds: 1 line of agent code, 1 command, guard.yaml (9 lines) and verify.yaml (16 lines) once per app, and a task file (12 lines) per task from whoever assigns the work.
• effect_e2e faults (8 of the 9 leave a wrong record; timeout leaves the right one):

Of the 8	Blocked by guard	Caught by verify-run	Got through
Agent alone	n/a	n/a	8 (6 shown as saved)
Guard, run 1	0	7	1 (collateral_unaudited: the REST read doesn't cover billing)
Compiled program, run 2	0	8	0 (scaffold-verifier's check includes the DB read path)

• What the guard blocks is the agent's side: a save with the wrong patient on screen (nothing written, and the agent's page said why) and a page sending the save twice (one record). Observe-only lets both through and reports them on read-back. timeout reads back VERIFIED. When MockMed's page gives up after 1.2 s, the agent alone shows an error for a save that landed, and the guard reads it back VERIFIED.
• Run 1 against run 2, 5 back-to-back pairs at the same load: the guarded agent took a median 6.26 s with 0 model calls (scripted; a model-driven agent makes about 6, one per action). The taught_triage_note tool over MCP took 4.39 s with 0 model calls. The guard held each save a median 7.4 ms and read it back in 41 ms; the agent's time with and without the guard was the same.

What it doesn't do yet

• No API step in run 2. Run 2 ran under the JIT standard profile (durable, governed authorization). No API step was proposed; the compiler's recorded reason is "the recording has no parameters to map into a request". MockMed's JSON save wouldn't be proposed either, and on main an API step falls back to GUI steps in durable and governed runs anyway. So the run-2 times are GUI times. Follow-up.
• An agent that types with Playwright's fill without clicking the field (Playwright MCP does this) leaves no focus step. The compiled program then refuses at the canary, fail-closed. The scripted agent clicks first. Follow-up.
• One task and one tab per guard session. Saves before the guard attaches and saves over WebSocket aren't seen. The identity check reads screen text (set screen_selector).
• scaffold-verifier treats MockMed's UI-echo banner table as a read of the record. The runs reset the store between saves, so a false stop from it wouldn't show here.

Tests

• tests/test_agent_guard.py: 34 unit tests (request classification, guard decisions against MockMed's fault server through a fake page, fail-closed CDP adapter, config, handoff helpers). Passes locally.
• tests/e2e/test_agent_guard_e2e.py: a real Chromium. The guard blocks the wrong patient and verifies the right one, the run is handed over, a person admits it, and the tool catches 5 faults. Both files are added to flow-critical. Locally I ran the same steps through the harness with mcp 2.3.0; the system Python's mcp 1.6.0 is older than required.
• Also run locally: test_capabilities_map, test_record_sor, test_compile_agent, the agent README commands test; ruff and mypy on the new code. Not run locally: the full suite and the feat(agent): teach a workflow over MCP, then call it with no model calls #1335 teach e2e (CI runs them).

🤖 Generated with Claude Code

[abrichr]

Included in batch #1384, which runs CI once for several reviewed PRs. This PR is marked merged when #1384 merges. Please don't push here unless asked; a new commit leaves it out of the batch.