Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
665d8a8
Merge remote-tracking branch 'origin/main' into claude/mcp-teach-comp…
abrichr Oct 19, 2026
e8ec967
feat(agent): teach a workflow over MCP, then call it with no model calls
abrichr Oct 19, 2026
a8e1eb0
docs: link the teach-over-MCP PR in CAPABILITIES.md
abrichr Oct 19, 2026
568a111
Merge remote-tracking branch 'origin/claude/crystallize-from-prompt-8…
abrichr Oct 19, 2026
328e4fb
fix(agent): bind each person's approval to one action; gate more off …
abrichr Oct 19, 2026
41dacc2
ci: pin two reviewed gitleaks false positives (key_dir in MCP teach t…
abrichr Oct 19, 2026
ca1020d
Merge origin/main into guard branch (brings in API steps #1269)
abrichr Oct 19, 2026
54d9962
feat(flow): guard an agent you already run (commit guard over CDP, ru…
abrichr Oct 19, 2026
9774017
test(flow): agent guard benchmark harness, e2e test, docs draft
abrichr Oct 19, 2026
60eebbb
fix(flow): guard on its own connection, recorder on its own thread; r…
abrichr Oct 19, 2026
6db638f
bench(flow): measure the agent guard on MockMed; docs and capability map
abrichr Oct 19, 2026
6c7f103
chore(flow): regenerate the public artifact inventory for the agent g…
abrichr Oct 19, 2026
f8a88ef
style: format the agent guard report
abrichr Oct 19, 2026
f045824
docs: link the agent guard PR in the capability map
abrichr Oct 19, 2026
42cec77
docs: state run 2's profile and the recorded reason no API step was p…
abrichr Oct 19, 2026
13e7a80
Merge origin/main into claude/mcp-teach-compile-a652e8
abrichr Oct 19, 2026
54924e6
Merge remote-tracking branch 'origin/claude/mcp-teach-compile-a652e8'…
abrichr Oct 19, 2026
36a5fb7
Merge origin/main into the agent guard branch
abrichr Oct 19, 2026
3553c9b
test(flow): don't load mcp when collecting the MCP teach tests
abrichr Oct 19, 2026
4999d26
Merge origin/main into claude/mcp-teach-compile-a652e8
abrichr Oct 19, 2026
9c67a6d
Merge origin/main into the agent guard branch (split files)
abrichr Oct 19, 2026
623b614
Merge #1335's updated branch into the agent guard branch
abrichr Oct 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/packages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -363,7 +363,7 @@ jobs:
python-version: "3.12"
cache: pip
- name: Install
run: pip install -e ../types -e '.[dev]'
run: pip install -e ../types -e '.[dev,agent]'
- name: Resolve Playwright version
id: pw
run: |
Expand Down Expand Up @@ -399,6 +399,9 @@ jobs:
tests/e2e/test_structural_action.py \
tests/e2e/test_compile_agent_e2e.py \
tests/e2e/test_code_action_e2e.py \
tests/e2e/test_mcp_teach_e2e.py \
tests/test_agent_guard.py \
tests/e2e/test_agent_guard_e2e.py \
--basetemp=runs/ci-critical --junitxml=runs/critical-junit.xml
- name: Fail if any critical test was skipped or none ran
run: |
Expand Down
4 changes: 4 additions & 0 deletions .gitleaksignore
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,10 @@ eb78575c688f06d7edc04afc69389aad10e1f5fa:packages/flow/tests/fixtures/terminal_v
# branch commit that added them. The lines carry gitleaks:allow from the next commit on.
3f0df2191a649540e79a621ac5406ebe2ff228e2:packages/flow/openadapt_flow/jit/authority.py:generic-api-key:101
3f0df2191a649540e79a621ac5406ebe2ff228e2:packages/flow/openadapt_flow/jit/authority.py:generic-api-key:390
# MCP teach (PR #1335): the generic-api-key rule matched the variable name `key_dir`
# in a principal_from_key(...) call in two tests. No credential.
e8ec96725c4451aa64cb97230c18d04b55d98503:packages/flow/tests/test_mcp_teach_surfaces.py:generic-api-key:147
e8ec96725c4451aa64cb97230c18d04b55d98503:packages/flow/tests/e2e/test_mcp_teach_e2e.py:generic-api-key:122
# PR #1367 (network capture redaction): the first commit's synthetic test values in
# test_network_capture_redact_json.py; the next commit builds them at run time.
52502adf407793c211fb3c0ac5aba37556206d8c:packages/flow/tests/test_network_capture_redact_json.py:generic-api-key:17
Expand Down
27 changes: 24 additions & 3 deletions CAPABILITIES.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,10 @@ shipping in the next release; a pull request number means it is being merged.
the verified run into a program (`openadapt_flow.jit`, released;
prompt-to-program `compile-agent`, landing; the agent can act through code
instead of screenshots, [#1305](https://github.com/OpenAdaptAI/OpenAdapt/pull/1305)). Agents can drive the recording
on web, native Windows, RDP and Citrix.
on web, native Windows, RDP and Citrix. An agent connected over MCP can
teach itself a new workflow and then call it as a tool
(`openadapt-agent serve --teach`,
[#1335](https://github.com/OpenAdaptAI/OpenAdapt/pull/1335)).
2. **Compiles to the fastest layer the app allows.** Page structure first (DOM,
Windows UI Automation, macOS Accessibility), direct API calls where the app's
own traffic allows them (`ApiActuator`, released; compiled from captured
Expand Down Expand Up @@ -63,7 +66,9 @@ shipping in the next release; a pull request number means it is being merged.
9. **Is a safety layer underneath any agent.** An agent's MCP calls, a task
ledger that keeps a long job's rules, items, questions and checked saves on
the server ([#1332](https://github.com/OpenAdaptAI/OpenAdapt/pull/1332)), a guarded save around any agent's write
([#1258](https://github.com/OpenAdaptAI/OpenAdapt/pull/1258)), `verify-run`
([#1258](https://github.com/OpenAdaptAI/OpenAdapt/pull/1258)), a guard for
an agent you already run that holds its browser's saves with no change to
the agent ([#1357](https://github.com/OpenAdaptAI/OpenAdapt/pull/1357)), `verify-run`
to judge any executor's write from the system of record, and Agent Skills and
a CLI for coding agents. Released except where a PR is named.
10. **Runs locally and keeps data private.** Healthy runs make no model calls
Expand Down Expand Up @@ -218,6 +223,8 @@ improvising or retrying when something is off.
| Code as action: the run-1 agent acts through one terminal tool (structure first, actions by role and name, a sandboxed read-only shell, screenshots on request); every action is a structured step checked against the compiled program | In review ([#1305](https://github.com/OpenAdaptAI/OpenAdapt/pull/1305)) | Real app (local OpenEMR) and synthetic | `ClaudeAgent(vision="code")`, `CODE_AS_ACTION.md` ([#1305](https://github.com/OpenAdaptAI/OpenAdapt/pull/1305)) |
| Admit a draft by a person, an admitter key, or an agent under a person-signed policy, including self-admission when the policy allows it | Released | Unit tests | [JIT_ADMISSION.md](packages/flow/docs/JIT_ADMISSION.md) |
| MCP server: your agent decides what to enter, OpenAdapt makes the entry and checks it | Released | Synthetic sandbox | `openadapt-agent`, [packages/agent](packages/agent/README.md) |
| Teach over MCP: an agent does a new task once through OpenAdapt; the run is read back, compiled and admitted (a person confirms the record check; the agent admits only under a person-signed policy), and becomes a tool it calls with no model calls. Web, and Windows, macOS, Linux, RDP and Citrix with a person approving each save | In review ([#1335](https://github.com/OpenAdaptAI/OpenAdapt/pull/1335)) | Synthetic (MockMed over MCP, and through the real RDP and Citrix backends) | `openadapt-agent serve --teach`, [packages/agent](packages/agent/README.md#teach-a-new-workflow-from-your-agent) |
| Guard an agent you already run: attach to its own Chromium over CDP (one added line), hold each save until the record on screen is the one the task names, block it and tell the agent why when it isn't, read the save back; a run that reads back correctly becomes a draft program with a record check from scaffold-verifier, which a person confirms and admits, then a `taught_*` tool with no model calls. `--observe-only` never blocks | In review ([#1357](https://github.com/OpenAdaptAI/OpenAdapt/pull/1357), stacked on [#1335](https://github.com/OpenAdaptAI/OpenAdapt/pull/1335)) | Synthetic (MockMed, scripted Playwright agent, `effect_e2e` faults) | `openadapt-flow guard`, [GUARD_AN_AGENT.md](packages/flow/docs/GUARD_AN_AGENT.md) |
| Task ledger per job: rules locked when the job opens and enforced on every run, item status set only from OpenAdapt's own results, and yes-or-no questions a person answers on the phone or in Desktop, including when a run stops | In review ([#1332](https://github.com/OpenAdaptAI/OpenAdapt/pull/1332)) | Synthetic (30-item MockMed worklist, scripted agent, no model calls) | MCP tools `open_job`, `get_job`, `add_items`, `record_evidence`, `ask_user`, `get_answer`, `change_constraint`, `close_job`; [Track a long job](packages/agent/README.md#track-a-long-job) |
| Compile an agent's first verified run from a prompt (`compile-agent`), and a code-as-action authoring agent | In review ([#1277](https://github.com/OpenAdaptAI/OpenAdapt/pull/1277), [#1305](https://github.com/OpenAdaptAI/OpenAdapt/pull/1305)); the agent harness behind it runs on every backend, on main ([#1283](https://github.com/OpenAdaptAI/OpenAdapt/pull/1283)) | Synthetic | `compile-agent` |
| Score harness optimizers (Harness², Meta-Harness) on GUI tasks with the record read-back as the reward, with full trace export | On main ([#1299](https://github.com/OpenAdaptAI/OpenAdapt/pull/1299), [#1300](https://github.com/OpenAdaptAI/OpenAdapt/pull/1300), [#1301](https://github.com/OpenAdaptAI/OpenAdapt/pull/1301)) | Synthetic (scripted proposer) | `integrations/harness2/` |
Expand Down Expand Up @@ -246,7 +253,7 @@ A test fails when a command has no file or this list is out of date.

**Record and build a program:**

- `compile-agent`
- `compile-agent`: With `admit TASK_DIR`, a person confirms and admits a waiting draft, such as one an agent taught over MCP (`openadapt-agent serve --teach`).
- `compile`
- `compose`
- `demo-record`
Expand Down Expand Up @@ -341,6 +348,10 @@ A test fails when a command has no file or this list is out of date.
- `demo`
- `tutorial`

**Guard an agent you already run:**

- `guard`: Attaches to the agent's own Chromium over CDP, holds each save until the right record is on screen, reads it back, and hands a verified run to the teach dir.

<!-- END generated by scripts/build_command_map.py -->

## Measured results
Expand All @@ -352,6 +363,16 @@ A test fails when a command has no file or this list is out of date.
took 38.6 s, slower than the agent. Raw data and
figures: [#1251](https://github.com/OpenAdaptAI/OpenAdapt/pull/1251)
(`benchmark/openemr_referral_timing/` once merged).
- **Guard an agent you already run, MockMed.** A scripted Playwright agent
in its own browser, 90 runs at a high shared load (median load average 29).
Of the 8 `effect_e2e` faults that leave a wrong record, the agent alone
reported 6 as saved; with the guard, the read-back caught 7 and 1 (a stray
write to a table the REST read doesn't cover) got through; the compiled
program caught all 8. The guard blocked the saves it can see: a save on the
wrong patient and a second submit. Same-load pairs: the guarded agent took
6.3 s (6 actions, each a model call for a real agent), the compiled tool
4.4 s with no model calls:
[GUARD_AN_AGENT.md](packages/flow/docs/GUARD_AN_AGENT.md#measured-on-mockmed).
- **Fault test.** We injected 8 kinds of bad save, 9 runs each, alongside
clean runs: 72 runs left a wrong record behind. A check that trusts the screen's "Saved" banner reported 54 of those
as success; reading the record back over the app's REST API missed 9; the
Expand Down
Loading
Loading