ci: one file per openadapt-flow command for the capability map

CAPABILITIES.md

@@ -230,19 +230,112 @@ improvising or retrying when something is off.
 
 ## Every command, by capability
 
-Each `openadapt-flow` command, grouped by the capability it serves. A test
-fails when a command is added without being listed here.
-
-- **Record and build a program:** `record`, `demo-record`, `compile`, `induce`, `disambiguate`, `for-each`, `compose`, `process`, `intake`, `visualize`, `pull`, `variants`, `route`, `compile-agent`.
-- **Check and admit:** `lint`, `scaffold-verifier`, `check`, `api`, `bind-record-checks`, `certify`, `qualify`, `propose-qualification`, `known-ids`, `jit`, `admit`, `approver-grant`.
-- **Run and answer:** `run`, `replay`, `resume`, `approve`, `item-approval`, `triage`, `business-decisions`, `explain`, `console`, `deferred`.
-- **Judge any executor's write:** `verify-run`, `serve-reward` (scores agent training episodes by reading the system of record).
-- **Repair and learn from halts:** `teach`, `teach-branch`, `repair`, `fleet`, `undo` (the undo steps a person approves after a stop).
-- **Give agents the program:** `emit-skill`, `emit-mcp`, `serve-execute`.
-- **Proof:** `seal`, `local-seal`, `countersign`, `attach-seal`.
-- **Run on premises:** `service`, `no-outbound`.
-- **Share safely with OpenAdapt Cloud:** `sanitize`, `review-sanitized`, `approve-sanitized`, `materialize-approved`, `push`, `validate-hosted`, `report-break`, `report-run`, `connect`, `login`, `connector`.
-- **Try it and measure it:** `demo`, `tutorial`, `bench`, `benchmark`.
+Each `openadapt-flow` command, grouped by the capability it serves. Each
+command has one file under
+[`packages/flow/docs/commands/`](packages/flow/docs/commands/): to add a
+command, add `<group>/<verb>.md` and run `python scripts/build_command_map.py`.
+A test fails when a command has no file or this list is out of date.
+
+<!-- BEGIN generated by scripts/build_command_map.py -->
+
+**Record and build a program:**
+
+- `compile-agent`
+- `compile`
+- `compose`
+- `demo-record`
+- `disambiguate`
+- `for-each`
+- `induce`
+- `intake`
+- `process`
+- `pull`
+- `record`
+- `route`
+- `variants`
+- `visualize`
+
+**Check and admit:**
+
+- `admit`
+- `api`
+- `approver-grant`
+- `bind-record-checks`
+- `certify`
+- `check`
+- `jit`
+- `known-ids`
+- `lint`
+- `propose-qualification`
+- `qualify`
+- `scaffold-verifier`
+
+**Run and answer:**
+
+- `approve`
+- `business-decisions`
+- `console`
+- `deferred`
+- `explain`
+- `item-approval`
+- `replay`
+- `resume`
+- `run`
+- `triage`
+
+**Judge any executor's write:**
+
+- `serve-reward`: Scores agent training episodes by reading the system of record.
+- `verify-run`
+
+**Repair and learn from halts:**
+
+- `fleet`
+- `repair`
+- `teach-branch`
+- `teach`
+- `undo`: The undo steps a person approves after a stop.
+
+**Give agents the program:**
+
+- `emit-mcp`
+- `emit-skill`
+- `serve-execute`
+
+**Proof:**
+
+- `attach-seal`
+- `countersign`
+- `local-seal`
+- `seal`
+
+**Run on premises:**
+
+- `no-outbound`
+- `service`
+
+**Share safely with OpenAdapt Cloud:**
+
+- `approve-sanitized`
+- `connect`
+- `connector`
+- `login`
+- `materialize-approved`
+- `push`
+- `report-break`
+- `report-run`
+- `review-sanitized`
+- `sanitize`
+- `validate-hosted`
+
+**Try it and measure it:**
+
+- `bench`
+- `benchmark`
+- `demo`
+- `tutorial`
+
+<!-- END generated by scripts/build_command_map.py -->
 
 ## Measured results
 

CLAUDE.md

@@ -11,9 +11,9 @@ under `packages/flow`, next to `packages/agent`, `packages/types` and
 `packages/capture`. The standalone openadapt-flow, openadapt-agent and
 openadapt-types repositories are frozen; changes go here.
 [CAPABILITIES.md](CAPABILITIES.md) is the map of what exists and how proven
-it is; update it in the same pull request as any capability change, and list
-any new `openadapt-flow` command under "Every command, by capability" (a test
-enforces it). Refresh the review brief with `python scripts/build_llms_full.py` when you
+it is; update it in the same pull request as any capability change. For a
+new `openadapt-flow` command, add `packages/flow/docs/commands/<group>/<verb>.md`
+and run `python scripts/build_command_map.py` (a test enforces it). Refresh the review brief with `python scripts/build_llms_full.py` when you
 change README.md, CAPABILITIES.md, SURFACES.md, LIMITS.md or the flow
 CHANGELOG, and always before a release.
 

packages/flow/docs/commands/01-record-and-build/README.md

@@ -0,0 +1 @@
+# Record and build a program

packages/flow/docs/commands/02-check-and-admit/README.md

@@ -0,0 +1 @@
+# Check and admit

packages/flow/docs/commands/03-run-and-answer/README.md

@@ -0,0 +1 @@
+# Run and answer

packages/flow/docs/commands/04-judge-any-executor/README.md

@@ -0,0 +1 @@
+# Judge any executor's write

packages/flow/docs/commands/04-judge-any-executor/serve-reward.md

@@ -0,0 +1 @@
+Scores agent training episodes by reading the system of record.

packages/flow/docs/commands/05-repair-and-learn/README.md

@@ -0,0 +1 @@
+# Repair and learn from halts

packages/flow/docs/commands/05-repair-and-learn/undo.md

@@ -0,0 +1 @@
+The undo steps a person approves after a stop.

packages/flow/docs/commands/06-give-agents-the-program/README.md

@@ -0,0 +1 @@
+# Give agents the program

packages/flow/docs/commands/07-proof/README.md

@@ -0,0 +1 @@
+# Proof

packages/flow/docs/commands/08-run-on-premises/README.md

@@ -0,0 +1 @@
+# Run on premises

packages/flow/docs/commands/09-share-with-cloud/README.md

@@ -0,0 +1 @@
+# Share safely with OpenAdapt Cloud

packages/flow/docs/commands/10-try-and-measure/README.md

@@ -0,0 +1 @@
+# Try it and measure it

packages/flow/docs/commands/README.md

@@ -0,0 +1,17 @@
+# openadapt-flow commands, by capability
+
+One file per command: `<group>/<verb>.md`. The group directory names the
+capability; its `README.md` holds the title, and the numeric prefix sets the
+order. A command's file is empty or holds a one-line note.
+
+To add a command, add its file to the right group and run:
+
+```bash
+python scripts/build_command_map.py
+```
+
+That rewrites the "Every command, by capability" list in the root
+`CAPABILITIES.md`. `packages/flow/tests/test_capabilities_map.py` fails when a
+command has no file, a file names no command, or the list is out of date.
+Because each command is its own file and the list has one command per line, two
+pull requests that each add a command rarely edit the same line.

packages/flow/tests/test_capabilities_map.py

@@ -1,21 +1,27 @@
 """CAPABILITIES.md at the repository root lists every openadapt-flow command.
 
 Reviews of OpenAdapt keep missing capabilities that ship because nothing tied
-the capability map to the code. A new command fails this test until it is
-listed, by capability, in CAPABILITIES.md.
+the capability map to the code. A new command fails this test until it has a
+file under packages/flow/docs/commands/<group>/ and the list in CAPABILITIES.md
+is regenerated with scripts/build_command_map.py. One file per command keeps
+two pull requests that each add a command from editing the same line.
 """
 
 from __future__ import annotations
 
 import argparse
-import re
+import importlib.util
 from pathlib import Path
 
 import pytest
 
 from openadapt_flow.__main__ import build_parser
 
-MAP = Path(__file__).resolve().parents[3] / "CAPABILITIES.md"
+REPO = Path(__file__).resolve().parents[3]
+SCRIPT = REPO / "scripts" / "build_command_map.py"
+pytestmark = pytest.mark.skipif(
+    not SCRIPT.exists(), reason="runs from the monorepo checkout"
+)
 
 
 def _verbs() -> list[str]:
@@ -25,12 +31,30 @@ def _verbs() -> list[str]:
     raise AssertionError("openadapt-flow has no subcommands")
 
 
-@pytest.mark.skipif(not MAP.exists(), reason="runs from the monorepo checkout")
-def test_every_command_is_listed_in_the_capability_map() -> None:
-    text = MAP.read_text(encoding="utf-8")
-    section = text[text.index("## Every command, by capability") :]
-    listed = set(re.findall(r"`([a-z][a-z-]*)`", section))
-    missing = [verb for verb in _verbs() if verb not in listed]
+def _command_map():
+    spec = importlib.util.spec_from_file_location("build_command_map", SCRIPT)
+    assert spec and spec.loader
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+def test_every_command_has_a_capability_file() -> None:
+    documented = _command_map().documented_verbs()
+    missing = sorted(set(_verbs()) - set(documented))
     assert not missing, (
-        f"add {missing} to 'Every command, by capability' in CAPABILITIES.md"
+        f"add packages/flow/docs/commands/<group>/<verb>.md for {missing}, "
+        "then run: python scripts/build_command_map.py"
+    )
+    stale = sorted(set(documented) - set(_verbs()))
+    assert not stale, f"no such openadapt-flow command; remove the files for {stale}"
+    duplicated = sorted({v for v in documented if documented.count(v) > 1})
+    assert not duplicated, f"each command belongs to one capability: {duplicated}"
+
+
+def test_capability_map_command_list_is_current() -> None:
+    command_map = _command_map()
+    text = command_map.MAP.read_text(encoding="utf-8")
+    assert command_map.splice(text, command_map.render()) == text, (
+        "run: python scripts/build_command_map.py"
     )

scripts/build_command_map.py

@@ -0,0 +1,84 @@
+"""Render CAPABILITIES.md's "Every command, by capability" list from files.
+
+Each openadapt-flow command has one file, packages/flow/docs/commands/<group>/<verb>.md.
+The group directory names the capability (its README.md holds the title; the
+numeric prefix sets the order). A command's file is empty or holds a one-line
+note shown after it. Adding a command adds a file, and the rendered list has
+one command per line, so two pull requests that add commands rarely touch the
+same line.
+
+    python scripts/build_command_map.py          # rewrite the section
+    python scripts/build_command_map.py --check  # fail if it is out of date
+
+packages/flow/tests/test_capabilities_map.py fails when a command has no file,
+a file names no command, or the section is out of date.
+"""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+MAP = ROOT / "CAPABILITIES.md"
+COMMANDS = ROOT / "packages/flow/docs/commands"
+HEADING = "## Every command, by capability"
+BEGIN = "<!-- BEGIN generated by scripts/build_command_map.py -->"
+END = "<!-- END generated by scripts/build_command_map.py -->"
+
+
+def groups(root: Path = COMMANDS) -> list[tuple[str, dict[str, str]]]:
+    """Return (title, {verb: note}) per capability, in directory order."""
+    result: list[tuple[str, dict[str, str]]] = []
+    for directory in sorted(path for path in root.iterdir() if path.is_dir()):
+        title = (directory / "README.md").read_text("utf-8").strip()
+        if not title.startswith("# "):
+            raise SystemExit(f"{directory}/README.md must be a single '# Title' line")
+        verbs = {
+            path.stem: path.read_text("utf-8").strip()
+            for path in sorted(directory.glob("*.md"))
+            if path.name != "README.md"
+        }
+        result.append((title[2:], verbs))
+    return result
+
+
+def documented_verbs(root: Path = COMMANDS) -> list[str]:
+    return sorted(verb for _, verbs in groups(root) for verb in verbs)
+
+
+def render(root: Path = COMMANDS) -> str:
+    lines = [BEGIN, ""]
+    for title, verbs in groups(root):
+        lines += [f"**{title}:**", ""]
+        for verb, note in verbs.items():
+            lines.append(f"- `{verb}`" + (f": {note}" if note else ""))
+        lines.append("")
+    lines.append(END)
+    return "\n".join(lines)
+
+
+def splice(text: str, block: str) -> str:
+    start, end = text.index(BEGIN), text.index(END) + len(END)
+    if text.rfind(HEADING, 0, start) == -1:
+        raise SystemExit(f"the generated block must sit under '{HEADING}'")
+    return text[:start] + block + text[end:]
+
+
+def main(argv: list[str]) -> int:
+    text = MAP.read_text("utf-8")
+    updated = splice(text, render())
+    if "--check" in argv:
+        if updated != text:
+            print(
+                "CAPABILITIES.md command list is out of date; "
+                "run: python scripts/build_command_map.py"
+            )
+            return 1
+        return 0
+    MAP.write_text(updated, "utf-8")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main(sys.argv[1:]))