feat(agent): teach a workflow over MCP, then call it with no model calls

[abrichr]

An agent connected over MCP can now turn a task it can't do yet into a workflow it calls, without leaving the conversation. It does the task once through OpenAdapt; the run is read back, compiled and admitted; and the result is a new tool that runs with no model calls.

Stacked on #1277 (compile-agent). Its base is that branch; merge #1277 first, then this retargets to main.

How it works

openadapt-agent serve --teach verify.yaml --url <test app> (or --config deploy.yaml for a native or remote app) adds:

Tool	What it does
teach_start	Opens the operator's app and starts run 1 of the agent's task (inputs, the record fields the save must produce, the record's identity on screen).
teach_observe, teach_act, teach_screen	The agent does the task one action at a time, through compile-agent's harness and save gate.
teach_finish	Compiles the run and reads the system of record. Returns the draft: steps, parameters, the proposed record check, and what still needs a person.
teach_confirm_checks	A person confirms the record check in the client's form, with their name. No argument stands in for them.
teach_admit	Canaries, then JIT admission under a person-signed policy.
taught_<name>	One per admitted program; also listed in list_workflows / run_workflow.

The connected agent is the inner agent of FirstRunCompiler: compile_agent/driven.py runs record() on its own thread and feeds it one action per tool call. Nothing in the record check, the proposed checks, the canaries or the emit step is new code.

Admission stays inside JIT_ADMISSION.md

• The operator gives the server an agent key and a signed policy (--teach-key, --teach-policy). An admitter key is refused: on a server the agent drives, it would let the agent approve its own draft.
• The server records run 1 as that agent's principal, so teach_admit is self-admission. Lifecycle refuses it unless the policy sets allow_self_admission, before any canary runs.
• No key: the draft names no agent principal, so only a person (or an admitter key) can admit. teach_admit returns the terminal command: openadapt-flow compile-agent admit TASK_DIR --admit-as NAME (new).
• After admission the server sends notifications/tools/list_changed (it now advertises tools.listChanged). A server started later lists every taught program.

Every backend

FirstRunCompiler(surface=<deployment backend section>) uses a new SurfaceHarness off the web. It gates the same actions, but it can't read which record a native or remote form shows, so a person approves every gated action in the client's form; a client without forms can't save. The approval is bound to the control the person was shown. Native backends list no controls yet (BACKEND_PARITY gap), so the agent reads the screen (--teach-screens scrubbed, fails closed) and clicks pixels. On RDP and Citrix, admission is refused with the gate's reason (pixel identity is not armed yet). Network RDP is refused at start.

Tests (synthetic data, no model calls)

• tests/e2e/test_mcp_teach_e2e.py (added to the flow-critical job, which now installs the agent extra): a scripted MCP SDK client on MockMed. Full loop: record, read back, person confirms, agent self-admits under policy, tools/list_changed, the new tool runs done with 0 model calls. Refusals: admit before the check is confirmed; no policy; recording agent without allow_self_admission (no canary runs); a save on the wrong record fails the read-back; the agent can't confirm the check without a person. Also a person admitting at a terminal.
• tests/test_mcp_teach_surfaces.py: teach over MCP through the real RemoteDisplayBackend and CitrixWorkspaceBackend (kiosk window, pixels only); every click approved by a person; canary refused with its reason.
• tests/test_compile_agent_surface_harness.py, tests/test_mcp_teach_cli.py: unit tests for the surface harness and the server flags.

Local: those files pass; agent shim tests (mcp_server, cli, authoring, readme commands, bridge) pass; ruff, ruff format, mypy clean. The host's load average was 60-270 during runs, and one browser test flaked once while starting the app; CI is the real check.

Review fixes (independent review, commit 328e4fb)

Finding	Fix
A person's save approval was a label string, so it could apply to another run or a changed screen	The approval is bound to the run id, the exact action and the count of actions performed; any change refuses it and nothing is done. The form says which action it is (click at x=.., press Enter).
Off the web, Space after Tab, a coordinate match on a wrapper, and Yes/OK/Add clicks passed the gate	Every click is gated unless it is on a ref'd text field, check box, tab or plainly navigating control; x,y clicks, Enter, typing outside a text field, single-letter keys and shortcuts are gated; focus resets on Tab.
On the web, a typed space on a focused button was not gated	Gated.
A timed-out action could still run later	The run ends instead.
A run that failed to start could overlap a second run 1	It stays tracked until its thread ends.
Run 1, canaries and taught runs could write at the same time	One app lock.
An unfinished run 1 kept its browser after the server exited	serve stops it on exit.

Documented, not changed: on the web without --teach-hold, the agent names the identity the gate checks; the run-1 record check and the person who confirms it are what catch a weak one.

Docs

packages/agent/README.md (Teach a new workflow from your agent, linked from Connect your own agent), packages/agent/docs/DESIGN.md, packages/flow/docs/COMPILE_AGENT.md, packages/flow/docs/BACKEND_PARITY.md, and CAPABILITIES.md (summary, agents table, and the command list).

🤖 Generated with Claude Code