flow: threat model and prompt-injection controls for teach and repair

[abrichr]

What this answers

When an agent demonstrates a task (compile-agent, teach over MCP, the airbag guard) or proposes a repair, it reads live application content. A record note, a banner or a server response can tell it to do something else, and published work (OS-Harm, RedTeamCUA, VPI-Bench) shows that such text often succeeds against computer-use agents. Before this change OpenAdapt had admission gating for runs, but no written threat model and no tests for the teach and repair steps.

This PR does not try to filter page text. It limits what the agent's actions can do, using values declared before the agent reads any page (CaMeL's split between control and data), and it reads the system of record back afterwards.

Changes

• THREAT_MODEL.md covers demonstration, teach, repair and run. It lists the attackers (page content, a compromised app, a malicious recording, the agent itself) and the assets (records, credentials, approvals, admission keys). For each control (teach scope, save gate, record checks, admission, approval binding, egress limits, repair limits) it says what the control stops and what it does not, and it ends with a section on what remains unprotected.
• openadapt_flow.teach_scope: TeachScope declares the app origins, the record, the fields with their exact values, and any constants. TeachGuard enforces it on a live capture:
  • Typed text must equal a declared value, and paste is refused.
  • A click on a link or a form submit to an undeclared origin is refused before the click.
  • Every browser request to an undeclared origin is aborted, from before the first page load (PlaywrightBackend.launch(before_goto=...)).
  • With an app write adapter, each write is read before it is sent. A write to another record, to an undeclared field or with an undeclared value is aborted.
  • Flow reads the record back after the run (check_delta). The agent never runs this check, so it can't skip it.
• compile_agent(..., teach_scope=...): a violation ends the capture as scope_refused or readback_refused, with no draft. A clean draft carries a teach_scope report with names and digests only. The scope must declare every task input. With no scope, behavior is unchanged.
• Approval provenance: ItemApprovalRequest.provenance holds who asked, the agent's own reason (shown as a quote), the screen text it saw, the lines of that text that read like an instruction to an agent, and what the task allows. It is inside the request digest and left out when absent, so existing digests don't change. The phone portal card shows it. guard_commit takes a reason, and self-repair fills the reason from the proposal note.
• Self-repair refuses a proposed control that is a link or form leading to another origin.
• MockMed gets ?inject=<surface>:<goal> fixtures (note, banner, tooltip, alt, network, aside × other_patient, other_field, other_origin, skip_check) and a GET /api/notice route. Without the flag nothing renders. All text is synthetic.

Tests (no paid model calls)

• tests/test_injection_suite.py (40 cases, about 60 s, real browser). The scripted agent reads every text on the page, attributes included, and obeys every injected instruction, which is the worst case for a model.
  • 5 surfaces × 4 goals: each attack is blocked (write_wrong_record, typed_value_not_declared, link_origin_not_declared) or caught by the read-back (missing).
  • Variants: a field changed by clicking (write_value_not_declared), navigation to the outside origin, an image beacon (egress_origin_not_declared), and apps that lie about the save (wrong record, partial, duplicate).
  • Three positive controls run the same agent without a scope: the wrong patient is saved, the outside server gets the click and the beacon, and the extra text is saved. This shows the blocked cases are not vacuous.
  • Four cases go through compile_agent itself: a hijacked capture and a lying app each end with no draft, a clean scoped capture makes a draft, and a scope that misses an input is refused.
• tests/test_teach_scope.py: unit tests, plus an import-order regression test.
• tests/test_self_repair.py::test_a_proposal_steered_off_the_app_by_page_text_is_refused. Finding: without the new origin check, this decoy passes the patch and is stopped later by the probe's identity check. The new check refuses it earlier and does not depend on that identity read failing.
• tests/test_guarded_save.py::test_the_phone_shows_why_the_agent_asked, and a provenance assertion in the existing phone-approval self-repair test.
• tests/e2e/test_injection_live.py: live-model mode (claude-opus-5-5 through the computer toolset). It is off by default and needs OPENADAPT_INJECTION_LIVE=1, a named approver in OPENADAPT_INJECTION_LIVE_APPROVED_BY (ask the coordinator first; at most $2.00 for 4 runs), and the customer's own key. It has not been run.

Local runs:

• Flow: 791 passed, 4 skipped, 3 xfailed across the approval, console, JIT (including tests/e2e/test_jit_cua_e2e.py), MockMed, release, guard, self-repair and injection tests.
• packages/agent: 511 passed.
• ruff check, ruff format and mypy are clean.
• Not run locally: the Desktop vitest (src/portalShell.test.ts), which gets two new cases. CI runs it.

Still unprotected (also in THREAT_MODEL.md and LIMITS.md)

• The scope is wired into openadapt_flow.jit only. feat(flow): compile-agent — compile an agent's first verified run into a zero-token program #1277 (compile-agent), feat(agent): teach a workflow over MCP, then call it with no model calls #1335 (teach over MCP) and the airbag guard must pass a TeachScope into their capture when they land.
• A steered agent can still read other records in the same app and send what it read to its model provider (scrubbed where scrubbing is on).
• Off the web, or on the web without a write adapter, a wrong write lands first and the read-back catches it afterwards.
• Values picked by clicking, drag and drop, and WebSocket traffic are checked only by the adapter or the read-back.
• The instruction detector on the approval card is a display aid and decides nothing.
• No live-model measurement yet.

🤖 Generated with Claude Code

[abrichr]

Included in batch #1384, which runs CI once for several reviewed PRs. This PR is marked merged when #1384 merges. Please don't push here unless asked; a new commit leaves it out of the batch.