From 803f63f34321daf271ff58d816b40cec05e3fc11 Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:08:48 -0400 Subject: [PATCH 01/13] Plan Foundry cost changes around measured evidence --- docs/plans/chain-economics.md | 153 ++++++++++++++++++++++++++++++++++ 1 file changed, 153 insertions(+) create mode 100644 docs/plans/chain-economics.md diff --git a/docs/plans/chain-economics.md b/docs/plans/chain-economics.md new file mode 100644 index 0000000..f8e9253 --- /dev/null +++ b/docs/plans/chain-economics.md @@ -0,0 +1,153 @@ +--- +id: chain-economics +status: PROPOSED +created: 2026-08-05 +--- + +# Make Foundry cheaper in the places the evidence actually supports + +Foundry's current cost advice points at the review rounds. That is the wrong emphasis. The source project's July 27 to August 3 evidence says the expensive part is carrying a large conversation through many tool resumptions, repeated reads, duplicate checks, and mechanical close-out work. The review rounds were usually short, and late rounds plus independent graders kept finding real defects. Calling the two-round light path "a fraction of the cost" is stronger than the evidence allows and quietly encourages people to cut the part that was still earning its keep. + +This change will give Foundry a more honest economic story and a few small defaults that follow from it. It will keep the full and light paths, but describe the light path as a lower-assurance trade rather than a proven fraction-of-cost shortcut. It will put executable ceilings behind the context budgets Foundry already claims. It will stop asking for a live voice-gate call on every short stage report, stop reprinting unchanged files in the middle implementation rounds, and define one clean validation ladder: targeted checks while working, one complete project check in test-it, and one final complete check in wrap-up. + +The claim is intentionally modest. These changes remove known sources of repeated context and preserve the quality bar seen in local use. They do not come with a percentage or dollar-saving promise. The three-build batching and reread pilot held quality, but its median turns were flat, tool calls rose, and billed cost was unavailable. That is enough to keep the safer defaults. It is not enough to advertise savings. + +## Calls made here + +- Keep all five review rounds in the full path. The local record shows later rounds and independent graders still found consequential bugs. +- Keep the light path, but remove "a fraction of the cost" and explain that its saving depends on the tool, model, task shape, and amount of context already carried. +- Make the voice gate selective: final plan prose, frame-it, quiz, wrap-up, handoff, and any response the agent itself suspects is dense or awkward. Frame-it stays because it is the one human interview and runs only once. Quiz stays because it is opt-in, outside the ordinary chain, and often produces long teaching prose. Short structured round reports rely on their tight format; committed prose still meets the blocking jargon check. +- Keep full-file reads in implementation rounds 1 and 5. Rounds 2 through 4 read files changed since the last inspection plus whatever their new angle needs. Compaction, intervening edits, or unseen context force a full read. +- Run a complete project check once in test-it and once at wrap-up on a clean chain. Failures or substantive later changes can force extra runs. Targeted tests remain unlimited. +- Enforce the existing 8 KiB `AGENTS.md` ceiling and add a 110 KiB ceiling for the nineteen source command files together. The command ceiling has about 6 percent headroom from today's 105,823-byte baseline; new instructions must displace old ones instead of silently growing the chain. + +## What this deliberately does not ship + +- No Foundry cost estimator. The local transcript meter was delayed, tool-specific, and commonly undercounted by 40 to 75 percent after compaction. Billing or usage exports remain the authority. +- No automatic or prompted compaction for savings. Five measured boundaries lowered subsequent event cost but produced a median net result of -19.3 percent once summary cost was included. +- No cheaper-model recommendation. The local model switch was confounded, but the observed correction burden was bad enough that it was reversed. Foundry is also deliberately model-neutral. +- No generic wrap-up runner. It paid off in a project with many known deterministic chores. Foundry cannot safely invent those chores for every consumer project. +- No new administrative wrapper. A later audit found the remaining administrative turns were mostly real decision points or already batched. +- No claim that coherent batching saves a fixed amount. It stays as tool-use hygiene because it removes needless resumptions without combining dependent work. +- No durable chain ledger in this patch. Local evidence supports it as recovery and sequence protection, not as a cost reduction. It deserves a separate Foundry plan so its added guard turns are judged against the missed-stage risk honestly. + +## Risks + +The selective voice gate could let an awkward short chat response through. The mitigation is the strict report shape, the agent's "this feels dense" escape, and the existing blocking scan for committed prose. Freshness-aware rereads could miss adjacent context if an agent falsely remembers what it saw; the escape conditions are therefore mechanical and rounds 1 and 5 remain full reads. A hard command-size ceiling could reward cryptic prose; the check only blocks growth, while the existing voice, link, generated-shape, and behavioral tests continue to judge quality. + +--- + +## Inputs + +- Source-project evidence: Sol Wilds' July 27 build-chain cost audit, especially the session-734 measured-cost correction, session-737 model reversal, session-752 economic-gains adjudication, session-757 no-code verdict, and session-766 compaction rejection. +- Foundry's current public claims: `README.md` and `docs/light-path.md`. +- Foundry's current context guidance and budgets: `AGENTS.md` and `docs/wiki/engineering/context-engineering.md`. +- Foundry's current stage behavior: `.cursor/commands/build-it.md`, `.cursor/commands/test-it.md`, `.cursor/commands/wrap-up.md`, and `.cursor/commands/challenge-implementation-{1..5}.md`. +- Current size baseline: `AGENTS.md` 6,343 bytes; nineteen `.cursor/commands/*.md` files 105,823 bytes total. +- Current vendor contracts, used only to keep the prose neutral: [Cursor summarization](https://docs.cursor.com/en/agent/chat/summarization) documents context management, not a savings guarantee; [Cursor pricing](https://docs.cursor.com/account/pricing) and the [Codex rate card](https://help.openai.com/en/articles/20001106-codex-rate-card) expose tool-specific token economics; [Anthropic pricing](https://docs.anthropic.com/en/docs/about-claude/pricing) exposes separate cache-write and cache-hit rates. These are dated examples checked on 2026-08-05, not Foundry policy. + +## File-tree change + +```diff + foundry/ + ├── AGENTS.md + ├── README.md + ├── package.json + ├── .cursor/commands/ + │ ├── build-it.md + │ ├── test-it.md + │ ├── wrap-up.md + │ ├── challenge-implementation-2.md + │ ├── challenge-implementation-3.md + │ ├── challenge-implementation-4.md + │ └── selected stage files with repeated voice-gate boilerplate + ├── docs/ + │ ├── light-path.md + │ └── wiki/ + │ ├── INDEX.md ++│ └── engineering/chain-economics.md + ├── scripts/ ++│ └── check-context-budgets.js + └── tests/ ++ └── check-context-budgets.test.js +``` + +Generated `.claude/commands/` and `.agents/skills/` copies change only through `npm run shapes`. + +## Program design + +```text +npm run check +├── check-context-budgets --check +├── node --test +│ └── context-budget behavior tests +├── generate-command-shapes --confirm +├── check-links +└── check-jargon + +check-context-budgets --check +├── measure AGENTS.md +├── measure the .cursor/commands/*.md source set +└── fail with current, limit, and overage when either ceiling is crossed +``` + +Proposed script surface: + +```js +export function measureContextBudgets(rootDir): { + agentsBytes: number, + commandBytes: number, + commandCount: number +} + +export function evaluateContextBudgets(measurement, limits): { + ok: boolean, + violations: Array<{ name: string, actual: number, limit: number }> +} +``` + +The command line supports `--check` for CI failure and a default human-readable report. It reads files only and has no vendor or transcript dependency. Foundry's own `npm run check` makes the ceiling mandatory in this repository. Consumer projects receive the script through the existing installer, but enforcing it in their own check command is opt-in because Foundry does not own their package scripts. + +## Build steps + +1. Add the economic record and correct the public claim. Create the proposed chain-economics page under `docs/wiki/engineering/`, add it to the wiki index, add one `See also` pointer from `docs/wiki/engineering/context-engineering.md`, and revise `README.md` plus `docs/light-path.md`. Include the measured positive, negative, and uncertain findings. Do not publish a savings percentage or duplicate the economic record across both wiki pages. +2. Put teeth behind context size. Add the read-only budget script and behavioral tests, integrate it into `npm run check`, and pin both missing-file and over-budget failures. The first slice is runnable here: the current tree passes, while a temporary oversized fixture fails with an exact overage. +3. Remove paid ritual from the voice gate. Put the selective rule in `AGENTS.md`; retain explicit live-gate calls in construct-the-plan, frame-it, quiz, wrap-up, and handoff; let the self-suspected dense-response clause cover other unusual prose. Remove the repeated `## Voice` blocks from build-it, start-up, test-it, security-scan, all five plan challenges, and all five implementation challenges. Regenerate the Claude and Codex shapes and keep the total source-command set below its new ceiling. +4. Port freshness-aware review reads. Change implementation rounds 2 through 4 only. Each round fully reads files changed in the preceding round and any files its angle newly requires. It fully rereads everything when prior context was compacted, a file changed since inspection, or the new angle needs unseen surrounding code. Keep rounds 1 and 5 unchanged. +5. Define the validation ladder. Update build-it, test-it, and wrap-up so targeted checks run during implementation and fixes; test-it runs one complete project check after its behavioral tests are green; wrap-up runs the final complete check after cumulative review. Extra complete checks require a red result or a substantive change after the last one. Remove the unconditional immediate rerun of an already green deterministic suite. +6. Add coherent tool-use guidance without a runner. Build-it gets one compact paragraph: run independent reads together, keep dependent steps ordered, write large outputs to a file and return a short summary, use bounded reads when only one section is needed, and never combine unrelated edits to improve a metric. +7. Regenerate all command shapes, run `npm run check`, and inspect the final size report. The patch fails its own bar if it increases the source command total above 110 KiB or pushes `AGENTS.md` above 8 KiB. + +## Acceptance bars + +- `README.md` and `docs/light-path.md` no longer claim the light path costs "a fraction" or imply that review rounds are the dominant cost. +- The new economic page separates measured outcomes, modelled estimates, and uncertain inference. It includes the compaction rejection, model reversal, no-code administrative verdict, quality-held reread pilot, and the observed wrap-up movement after project-specific runners landed. It does not attribute that movement to the runner alone. +- `docs/wiki/engineering/context-engineering.md` links to the economic page without repeating its evidence table or operational rules. +- Current official vendor facts appear only as dated examples of why costs vary. No model name, price, cache duration, transcript path, billing threshold, or claim that every provider discounts cache use becomes Foundry policy. +- `npm run check` fails when `AGENTS.md` exceeds 8,192 bytes or the nineteen source commands exceed 112,640 bytes, and passes at the current baseline. +- Budget tests cover pass, exact-boundary pass, one-byte-over failure, missing source directory, and deterministic file ordering. +- Only construct-the-plan, frame-it, quiz, wrap-up, handoff, and the central self-suspected dense-response rule require live voice-gate calls. Generated command shapes remain byte-derived from `.cursor/commands/`. +- Foundry's own `npm run check` enforces the context ceilings. The installer copies the checker, documents how a consumer opts in, and does not rewrite a consumer project's package scripts to force adoption. +- Implementation rounds 2 through 4 carry all three full-reread escape conditions. Rounds 1 and 5 still require a full surrounding-context read. +- A clean chain requests one complete project check in test-it and one in wrap-up. Targeted checks, red-result reruns, and post-check substantive-change reruns remain allowed. A new test involving time, randomness, concurrency, or an outside process gets one targeted canary rerun even when the first result is green. +- Tool guidance never parallelizes work with a real data dependency and never encourages hiding output, combining unrelated edits, or optimizing a turn-count target. +- `npm run check` is green after regeneration, link checks, jargon checks, and the new budget gate. + +## Demoted-claims tracking + +| Round | Claim | Outcome | Plan change | +|---|---|---|---| +| R1 | Frame-it can rely on the dense-response escape instead of a live voice check. | Demoted. Its questions are the chain's one human interview and the stage runs once, so the quality trade is not earned. | Retained an explicit live voice check in frame-it. | +| R1 | Adding the checker to Foundry's package script enforces the ceiling in installed projects. | Demoted. Foundry does not own a consumer's package scripts. | Made Foundry enforcement mandatory and consumer enforcement documented opt-in. | +| R2 | The later wrap-up turn reduction measures the runner's savings. | Demoted. The runner, batching rule, and task shapes changed together. | Report the stage movement as observed and refuse per-lever attribution. | +| R2 | Every already-green test suite can lose its immediate second run. | Demoted. Flake-prone tests need a narrow reproducibility check. | Keep a targeted canary rerun for new time, randomness, concurrency, or outside-process tests. | +| R3 | Quiz can rely on the dense-response escape instead of a live voice check. | Demoted. Quiz is outside the ordinary chain and commonly produces long teaching prose, so removal adds risk without ordinary-chain savings. | Retained an explicit live voice check in quiz. | +| R4 | Discounted cached input is a durable cross-vendor rule. | Demoted. It holds in the current examples, but vendors can change pricing and cache behavior independently. | Use dated official examples and make live measurement, not a cache assumption, the durable rule. | +| R5 | R1 frame-it demotion | Confirmed after rereading `/frame-it`; this is the only required human interview and the saved call is one per chain. | No change. | +| R5 | R1 consumer-enforcement demotion | Confirmed in `scripts/install.js`; scripts copy, consumer package scripts do not. | No change. | +| R5 | R2 runner-attribution demotion | Confirmed in the source audit; runner and batching changes overlap and the later sample was heterogeneous. | No change. | +| R5 | R2 blanket-rerun demotion | Confirmed against `/test-it`; narrow flake canaries preserve the reason the second run existed without repeating every complete suite. | No change. | +| R5 | R3 quiz demotion | Confirmed in `/quiz`; it is opt-in and outside ordinary-chain cost. | No change. | +| R5 | R4 cross-vendor cache demotion | Confirmed against the dated official sources; current examples agree, but the contract is vendor-owned. | No change. | + +## Deviations From 03a2e738866e9d155605c5ea6daa7ff6b1c1e9d4 Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:17:06 -0400 Subject: [PATCH 02/13] Challenge chain economics plan against execution risks --- docs/plans/chain-economics.md | 46 ++++++++++++++++++++++++++++------- 1 file changed, 37 insertions(+), 9 deletions(-) diff --git a/docs/plans/chain-economics.md b/docs/plans/chain-economics.md index f8e9253..441470b 100644 --- a/docs/plans/chain-economics.md +++ b/docs/plans/chain-economics.md @@ -69,7 +69,8 @@ The selective voice gate could let an awkward short chat response through. The m ├── scripts/ +│ └── check-context-budgets.js └── tests/ -+ └── check-context-budgets.test.js ++ ├── check-context-budgets.test.js ++ └── chain-economics-contract.test.js ``` Generated `.claude/commands/` and `.agents/skills/` copies change only through `npm run shapes`. @@ -97,7 +98,8 @@ Proposed script surface: export function measureContextBudgets(rootDir): { agentsBytes: number, commandBytes: number, - commandCount: number + commandCount: number, + commandFiles: Array<{ path: string, bytes: number }> } export function evaluateContextBudgets(measurement, limits): { @@ -106,15 +108,19 @@ export function evaluateContextBudgets(measurement, limits): { } ``` -The command line supports `--check` for CI failure and a default human-readable report. It reads files only and has no vendor or transcript dependency. Foundry's own `npm run check` makes the ceiling mandatory in this repository. Consumer projects receive the script through the existing installer, but enforcing it in their own check command is opt-in because Foundry does not own their package scripts. +The command line supports `--check` for CI failure and a default human-readable report. Report mode exits zero after printing measurements. Check mode exits nonzero on a ceiling violation, missing input, unreadable file, or empty command set. Importing the module exports functions and does nothing else; CLI execution sits behind the same direct-run guard used by Foundry's other importable scripts. It reads files only and has no vendor or transcript dependency. Foundry's own `npm run check` makes the ceiling mandatory in this repository. Consumer projects receive the script through the existing installer, but enforcing it in their own check command is opt-in because Foundry does not own their package scripts. + +Measurement is portable: read UTF-8 text, normalize CRLF and lone CR line endings to LF, then count bytes. Refuse a missing `AGENTS.md`, a missing source-command directory, or a directory containing zero markdown commands. A broken input must never pass by measuring as zero. The report prints every command file in deterministic size-descending, path-ascending order so an overage names where the weight actually sits. + +The checker constants are the one executable home for the exact ceilings. `AGENTS.md` keeps its approximate human explanation and points at the checker; the economics page explains why the gate exists and how to justify a future change without copying another authoritative number. ## Build steps 1. Add the economic record and correct the public claim. Create the proposed chain-economics page under `docs/wiki/engineering/`, add it to the wiki index, add one `See also` pointer from `docs/wiki/engineering/context-engineering.md`, and revise `README.md` plus `docs/light-path.md`. Include the measured positive, negative, and uncertain findings. Do not publish a savings percentage or duplicate the economic record across both wiki pages. -2. Put teeth behind context size. Add the read-only budget script and behavioral tests, integrate it into `npm run check`, and pin both missing-file and over-budget failures. The first slice is runnable here: the current tree passes, while a temporary oversized fixture fails with an exact overage. -3. Remove paid ritual from the voice gate. Put the selective rule in `AGENTS.md`; retain explicit live-gate calls in construct-the-plan, frame-it, quiz, wrap-up, and handoff; let the self-suspected dense-response clause cover other unusual prose. Remove the repeated `## Voice` blocks from build-it, start-up, test-it, security-scan, all five plan challenges, and all five implementation challenges. Regenerate the Claude and Codex shapes and keep the total source-command set below its new ceiling. -4. Port freshness-aware review reads. Change implementation rounds 2 through 4 only. Each round fully reads files changed in the preceding round and any files its angle newly requires. It fully rereads everything when prior context was compacted, a file changed since inspection, or the new angle needs unseen surrounding code. Keep rounds 1 and 5 unchanged. -5. Define the validation ladder. Update build-it, test-it, and wrap-up so targeted checks run during implementation and fixes; test-it runs one complete project check after its behavioral tests are green; wrap-up runs the final complete check after cumulative review. Extra complete checks require a red result or a substantive change after the last one. Remove the unconditional immediate rerun of an already green deterministic suite. +2. Put teeth behind context size. Add the read-only budget script and behavioral tests, integrate it into `npm run check`, and pin both missing-file and over-budget failures. Extend the installer test to prove a fresh consumer receives the checker while its package scripts remain untouched. The first slice is runnable here: the current tree passes, while a temporary oversized fixture fails with an exact overage. +3. Remove paid ritual from the voice gate. Put the selective rule in `AGENTS.md`; retain explicit live-gate calls in construct-the-plan, frame-it, quiz, wrap-up, and handoff; let the self-suspected dense-response clause cover other unusual prose. Remove the repeated `## Voice` blocks from build-it, start-up, test-it, security-scan, all five plan challenges, and all five implementation challenges. Add a semantic command-contract test that identifies the retained set by behavior rather than exact prose. Regenerate the Claude and Codex shapes and keep the total source-command set below its new ceiling. +4. Port freshness-aware review reads. Change implementation rounds 2 through 4 only. Each round fully reads files changed in the preceding round and any files its angle newly requires. It fully rereads everything when prior context was compacted, a file changed since inspection, or the new angle needs unseen surrounding code. Keep rounds 1 and 5 unchanged. Extend the command-contract test to prove all three escape concepts are present in rounds 2 through 4 and that rounds 1 and 5 still require full surrounding-context reads. +5. Define the validation ladder. Update build-it, test-it, and wrap-up so targeted checks run during implementation and fixes. Test-it discovers the canonical complete check from explicit project rules first, then the commands the CI workflow actually runs, then package scripts when neither stronger source exists. A CI workflow with several verification commands defines an ordered check set, not a guessed single script. Test-it runs the discovered check or set once after the behavioral tests are green; when no canonical check exists, it runs the broadest existing suite and names the missing project-level check in its report rather than inventing a command. Wrap-up repeats the same discovery after cumulative review. Extra complete checks require a red result or a substantive change after the last one. Remove the unconditional immediate rerun of an already green deterministic suite, and rewrite test-it's rationale so it preserves the narrower flake-canary reason instead of the retired run-it-twice rule. The command-contract test proves the two clean-chain complete-check stages, the targeted-check allowance, and the named rerun reasons without matching whole sentences. 6. Add coherent tool-use guidance without a runner. Build-it gets one compact paragraph: run independent reads together, keep dependent steps ordered, write large outputs to a file and return a short summary, use bounded reads when only one section is needed, and never combine unrelated edits to improve a metric. 7. Regenerate all command shapes, run `npm run check`, and inspect the final size report. The patch fails its own bar if it increases the source command total above 110 KiB or pushes `AGENTS.md` above 8 KiB. @@ -125,12 +131,15 @@ The command line supports `--check` for CI failure and a default human-readable - `docs/wiki/engineering/context-engineering.md` links to the economic page without repeating its evidence table or operational rules. - Current official vendor facts appear only as dated examples of why costs vary. No model name, price, cache duration, transcript path, billing threshold, or claim that every provider discounts cache use becomes Foundry policy. - `npm run check` fails when `AGENTS.md` exceeds 8,192 bytes or the nineteen source commands exceed 112,640 bytes, and passes at the current baseline. -- Budget tests cover pass, exact-boundary pass, one-byte-over failure, missing source directory, and deterministic file ordering. +- Budget tests cover pass, exact-boundary pass, one-byte-over failure, CRLF/LF parity, missing `AGENTS.md`, missing source directory, empty source directory, unreadable input, deterministic file ordering, the largest-contributor report, CLI exit codes, and import-without-execution. The existing installer suite proves the checker is copied to a fresh consumer without rewriting that consumer's package scripts. +- Exact byte limits live once in the checker. `AGENTS.md` and the economic page explain and route to that source without defining independent executable values. - Only construct-the-plan, frame-it, quiz, wrap-up, handoff, and the central self-suspected dense-response rule require live voice-gate calls. Generated command shapes remain byte-derived from `.cursor/commands/`. - Foundry's own `npm run check` enforces the context ceilings. The installer copies the checker, documents how a consumer opts in, and does not rewrite a consumer project's package scripts to force adoption. - Implementation rounds 2 through 4 carry all three full-reread escape conditions. Rounds 1 and 5 still require a full surrounding-context read. -- A clean chain requests one complete project check in test-it and one in wrap-up. Targeted checks, red-result reruns, and post-check substantive-change reruns remain allowed. A new test involving time, randomness, concurrency, or an outside process gets one targeted canary rerun even when the first result is green. +- A clean chain with a canonical complete check requests it once in test-it and once in wrap-up. Discovery follows explicit project rules, then CI commands, then package scripts; multiple CI verification commands remain an ordered set. If none names a complete check, both stages use the broadest existing suite and report the gap. Targeted checks, red-result reruns, and post-check substantive-change reruns remain allowed. A new test involving time, randomness, concurrency, or an outside process gets one targeted canary rerun even when the first result is green. +- Test-it's recorded rationale describes the narrow flake canary and no longer teaches an unconditional run-it-twice rule that contradicts the procedure. - Tool guidance never parallelizes work with a real data dependency and never encourages hiding output, combining unrelated edits, or optimizing a turn-count target. +- One semantic command-contract test pins the selective voice-gate set, freshness-aware reread escapes, rounds 1 and 5 full-read rule, and validation ladder. It must accept meaning-preserving rewording and reject a reintroduced universal gate, missing reread escape, or unconditional extra complete-suite run. - `npm run check` is green after regeneration, link checks, jargon checks, and the new budget gate. ## Demoted-claims tracking @@ -149,5 +158,24 @@ The command line supports `--check` for CI failure and a default human-readable | R5 | R2 blanket-rerun demotion | Confirmed against `/test-it`; narrow flake canaries preserve the reason the second run existed without repeating every complete suite. | No change. | | R5 | R3 quiz demotion | Confirmed in `/quiz`; it is opt-in and outside ordinary-chain cost. | No change. | | R5 | R4 cross-vendor cache demotion | Confirmed against the dated official sources; current examples agree, but the contract is vendor-owned. | No change. | +| R1 repeat | Every consumer project has a canonical complete check test-it can invoke. | Demoted. Some installed projects may expose only partial suites or no named check. | Added ordered discovery and an honest broadest-suite fallback with the gap reported. | +| R1 repeat | Copying the scripts directory is enough proof that consumers receive the checker. | Demoted. That is current implementation, not a pinned install contract. | Added an installer acceptance test for the checker and the no-package-rewrite boundary. | +| R2 repeat | Raw file bytes give a stable context-budget result on every supported checkout. | Demoted. CRLF checkouts inflate the same prose relative to LF checkouts. | Normalize line endings to LF before measuring and test parity. | +| R2 repeat | Missing or empty command input will naturally fail the budget check. | Demoted. Zero files can produce a zero-byte pass unless it is rejected explicitly. | Refuse missing `AGENTS.md`, a missing command directory, and an empty command set. | +| R3 repeat | A total-only budget failure is actionable enough for a future maintainer. | Demoted. It names the problem but not the files contributing most to it. | Add a stable per-command size breakdown, largest first. | +| R3 repeat | Repeating exact ceilings in the checker and explanatory prose is harmless. | Demoted. The numbers can drift and make the prose lie about enforcement. | Make checker constants authoritative; prose points at them and carries reasons only. | +| R4 repeat | The checker can leave exit behavior implicit. | Demoted. CI needs violations and broken inputs to fail, while a report-only invocation should remain informational. | Defined zero/nonzero behavior for report and check modes. | +| R4 repeat | Exported functions imply the module is safe to import in tests. | Demoted. A top-level CLI path could inspect the real checkout or exit during import. | Require a direct-run guard and an import-without-execution test. | +| R5 repeat | R1-repeat complete-check discovery | Escalated after rereading test-it, wrap-up, package scripts, and CI ownership. Package script names are weaker evidence than the commands CI actually runs. | Reordered discovery to project rules, CI, then package scripts; preserve multi-command CI verification as a set. | +| R5 repeat | R1-repeat installer contract | Confirmed in `scripts/install.js` and its fresh-install fixture. | No change. | +| R5 repeat | R2-repeat line-ending portability | Confirmed; the repository has no checkout-level line-ending pin. | No change. | +| R5 repeat | R2-repeat missing-input refusal | Confirmed; an empty sum needs an explicit invalid-input branch. | No change. | +| R5 repeat | R3-repeat actionable budget output | Confirmed; current totals alone would not identify the largest command contributors. | No change. | +| R5 repeat | R3-repeat single limit authority | Confirmed against the existing approximate `AGENTS.md` budget note. | No change. | +| R5 repeat | R4-repeat CLI exit contract | Confirmed by the `&&`-chained `npm run check` pipeline. | No change. | +| R5 repeat | R4-repeat import safety | Confirmed against the import-safe pattern and regression test in `scripts/rotate-sessions.js`. | No change. | +| R6 | Generated-shape parity and prose acceptance bars are enough to preserve the three command-policy changes. | Demoted. Shape generation proves copies agree, not that the source commands retain the intended semantics. | Add one behavior-based command-contract test covering voice routing, reread escapes, and the validation ladder. | +| R7 | R6 semantic command-policy contract | Confirmed after rereading the generated-shape suite. It proves copies are synchronized but never inspects the source commands' economic policies. | No change. | +| R8 | Replacing test-it's unconditional rerun in the procedure is enough to retire the old rule. | Demoted. Test-it's rationale explicitly preserves the run-it-twice rule and would contradict the new validation ladder. | Require the rationale to carry the narrower flake-canary reason and remove the retired rule. | ## Deviations From 65d6b5d463b414b31e72614f8b1cbb7741ec00f8 Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:20:00 -0400 Subject: [PATCH 03/13] Mark chain economics work ready for implementation --- docs/plans/chain-economics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/plans/chain-economics.md b/docs/plans/chain-economics.md index 441470b..03f2b11 100644 --- a/docs/plans/chain-economics.md +++ b/docs/plans/chain-economics.md @@ -1,6 +1,6 @@ --- id: chain-economics -status: PROPOSED +status: IN_PROGRESS created: 2026-08-05 --- From e332759cb3e4377927105472efb86ae47020eb1e Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:24:54 -0400 Subject: [PATCH 04/13] Enforce context limits where repeated input becomes costly --- README.md | 6 +- docs/light-path.md | 6 +- docs/wiki/INDEX.md | 1 + docs/wiki/engineering/chain-economics.md | 41 +++++ docs/wiki/engineering/context-engineering.md | 2 + package.json | 2 +- scripts/check-context-budgets.js | 113 ++++++++++++++ tests/check-context-budgets.test.js | 149 +++++++++++++++++++ tests/install.test.js | 13 ++ 9 files changed, 326 insertions(+), 7 deletions(-) create mode 100644 docs/wiki/engineering/chain-economics.md create mode 100644 scripts/check-context-budgets.js create mode 100644 tests/check-context-budgets.test.js diff --git a/README.md b/README.md index 7eaf630..40df39e 100644 --- a/README.md +++ b/README.md @@ -58,9 +58,9 @@ The wiki ships stocked. Three reference shelves come with the repo: engineering ## What it costs -The full chain is thorough and token-heavy. On a mid-size feature it plausibly lands in the low hundreds of thousands of tokens end to end, which is real money on metered plans. That's the honest price of ten review rounds and the bookkeeping around them. +The full chain is thorough and token-heavy. On a mid-size feature it plausibly lands in the low hundreds of thousands of tokens end to end, which is real money on metered plans. The review rounds are not automatically the expensive part: in the source project's measured runs, build volume, repeated context-heavy reads, duplicate checks, and late mechanical work were often larger contributors. The rounds still found defects late in the chain. -You don't have to pay it every time. [The light path](docs/light-path.md) is the documented cheaper shape: same stages, two challenge rounds instead of five on each side, a fraction of the cost. Keep frame it in either path; a five-question interview is the cheapest insurance in the whole chain. Spend on the full chain when the work touches money, user data, or anything you'll live with for months. +You don't have to run all of it every time. [The light path](docs/light-path.md) is a lower-assurance shape with two challenge rounds instead of five on each side. It usually does less work, but Foundry has no evidence for a fixed saving: the result depends on the model, tool, task, and context already carried. Keep frame it in either path; a five-question interview is cheap insurance. Spend on the full chain when the work touches money, user data, or anything you'll live with for months. The evidence and the limits on what it proves live in [chain economics](docs/wiki/engineering/chain-economics.md). ## A session, end to end @@ -78,7 +78,7 @@ Two ways in. Starting fresh? Clone this repo and build your project inside it. The commands are already wired: Cursor and Claude Code pick them up the moment you open the folder, and Codex reads each one as a skill you invoke by hand. -Have a project already? Copy the three tool folders (`.cursor/commands/`, `.claude/commands/`, `.agents/`), the two rules files (`AGENTS.md`, `CLAUDE.md`), and `scripts/` into it, plus `docs/wiki/` if you want the reference library. That's the whole install. Or let the installer do the same copy, run from this repo's root: `node scripts/install.js ` (add `--wiki` for the library, `--dry-run` to preview), and re-run it after pulling Foundry to refresh your copies; it reports created, updated, and unchanged. The commands are plain markdown and need nothing running on your machine; `scripts/` rides along because the commands lean on its voice gate, phrase list, and log rotation, and Node is needed only at the moments you run those checks. The sessions log creates itself at your first wrap-up. +Have a project already? Copy the three tool folders (`.cursor/commands/`, `.claude/commands/`, `.agents/`), the two rules files (`AGENTS.md`, `CLAUDE.md`), and `scripts/` into it, plus `docs/wiki/` if you want the reference library. That's the whole install. Or let the installer do the same copy, run from this repo's root: `node scripts/install.js ` (add `--wiki` for the library, `--dry-run` to preview), and re-run it after pulling Foundry to refresh your copies; it reports created, updated, and unchanged. The commands are plain markdown and need nothing running on your machine; `scripts/` rides along because the commands lean on its voice gate, phrase list, log rotation, and context-budget report. Node is needed only at the moments you run those checks. The installer does not rewrite your package scripts. To enforce Foundry's context ceilings in a consumer project, add `node scripts/check-context-budgets.js --check` to that project's own complete check. The sessions log creates itself at your first wrap-up. If your project keeps its own docs pointing at wiki pages in a local Foundry checkout instead of copying the library in, `scripts/check-wiki-pointers.js` re-verifies those pointers from your project's own check chain, so a page renamed here fails loudly there instead of rotting silently. Arguments and the surfaces format are in its header. diff --git a/docs/light-path.md b/docs/light-path.md index 998ccad..b6e8931 100644 --- a/docs/light-path.md +++ b/docs/light-path.md @@ -1,8 +1,8 @@ # The light path -The full chain is thorough and token-heavy: five plan challenges, a build, tests, a security check, five implementation challenges, and the bookkeeping around them. On a mid-size feature that plausibly lands in the low hundreds of thousands of tokens end to end. If you're paying per token, or working with a smaller model that gets less out of each round, that's real money for real diminishing returns. +The full chain is thorough and token-heavy: five plan challenges, a build, tests, a security check, five implementation challenges, and the bookkeeping around them. On a mid-size feature that plausibly lands in the low hundreds of thousands of tokens end to end. If you're paying per token, or working with a smaller model that gets less out of each round, that's real money. -So here's the honest cheaper shape. Same process, a fraction of the cost: +The light path is the lower-assurance shape. It asks for fewer review rounds, so it normally does less work, but Foundry has not measured a fixed saving and does not treat the rounds as the dominant cost. Build volume, repeated reads, duplicate checks, and late mechanical work can outweigh them. Use this shape when the lower assurance is an honest trade: 1. **start-up**, then **construct-the-plan** as normal. 2. **frame-it stays.** A five-question interview is the cheapest insurance in the whole chain; never cut it first. @@ -11,6 +11,6 @@ So here's the honest cheaper shape. Same process, a fraction of the cost: 5. **challenge-implementation twice**, not five times. 6. **wrap-up** and **handoff** as normal. -When to spend on the full chain instead: work you'll live with for months, work that touches money or user data, work where being subtly wrong is expensive. The extra rounds exist because "done" and "right" aren't the same thing, and each fresh angle closes more of that gap. +When to spend on the full chain instead: work you'll live with for months, work that touches money or user data, work where being subtly wrong is expensive. The extra rounds exist because "done" and "right" aren't the same thing. In the source project's measured runs, later rounds and independent graders still found consequential defects. [Chain economics](wiki/engineering/chain-economics.md) carries the evidence and its limits. Smaller models: the chain is plain instructions, so any capable model can follow it. Expect the challenge rounds to catch less per round; compensate with the angles that need the least cleverness (the sequence of steps, hostile inputs, config syntax) rather than more rounds. diff --git a/docs/wiki/INDEX.md b/docs/wiki/INDEX.md index dc725d6..830f269 100644 --- a/docs/wiki/INDEX.md +++ b/docs/wiki/INDEX.md @@ -13,6 +13,7 @@ Ideas about building software that have aged well across fifty years, for design - [Hide what's likely to change](engineering/information-hiding.md): how to split code into modules, and the wrong-abstraction trap. Reach for it before extracting shared code or designing an interface. - [Theory and documentation](engineering/theory-and-documentation.md): why the code alone can't carry a project's reasoning, and what to write down instead. Reach for it when deciding what a plan or record should say. - [Context engineering](engineering/context-engineering.md): keeping what the agent reads lean, honest, and re-tunable. Reach for it when a rules file grows, the model generation changes, or the process feels heavier than the work it governs. +- [Chain economics](engineering/chain-economics.md): what Foundry's source project actually measured about cost, what stayed uncertain, and which defaults the evidence supports. Reach for it before claiming a process change saves money. - [Clear technical writing](engineering/clear-technical-writing.md): how to explain technical work in plain English without deleting the terms the reader needs. Reach for it when writing for someone who doesn't share your context, or when a review flags unexplained language. ## Design fundamentals diff --git a/docs/wiki/engineering/chain-economics.md b/docs/wiki/engineering/chain-economics.md new file mode 100644 index 0000000..c885950 --- /dev/null +++ b/docs/wiki/engineering/chain-economics.md @@ -0,0 +1,41 @@ +# Chain economics + +Foundry grew out of a large working chain, so it carries a real temptation: call every bit of process expensive, cut the visible review rounds, and tell ourselves the shorter shape costs a fraction as much. The source project's evidence does not support that story. + +It supports a narrower one. Long builds repeatedly paid to carry a large conversation through small tool resumptions. Large file rereads, duplicate complete checks, manual polling, and mechanical close-out work were visible costs. The review rounds were usually short, and later rounds plus independent graders kept finding defects. The useful target is repeated context and ritual, not review depth by default. + +## What was measured + +The July 27 to August 3, 2026 cost program used flushed transcripts, usage exports when one was available, and an explicitly approximate transcript model when billing was not. The distinction matters: a transcript estimate is not an invoice, and an in-flight transcript can lag badly enough to miss the stage currently running. + +| Evidence | Result | What it supports | +|---|---|---| +| Session 734 billing correction | Ordinary chains in the measured sample cost roughly $176 to $293 under that tool and model setup. Cached input plus the tool fee dominated. Earlier transcript constants had understated cost. | Use an attributable billing or usage surface for money claims. Treat stage-start transcript meters as lower bounds. | +| Session 737 model reversal | A theoretically cheaper builder produced enough correction burden that the project restored the more capable builder and kept a separate reviewer. | Do not make a generic cheaper-model recommendation. Quality and rework belong in the economic calculation. | +| Sessions 740 to 755, three-build pilot | Median assistant turns moved from 175 to 172. Median tool calls moved from 195 to 236. The approximate transcript estimate moved from about $103 to about $116. Billing was unavailable. Quality held under selective live voice checks, coherent batching guidance, and freshness-aware rereads. | Keep the smaller defaults because their quality bar held. Do not claim they saved money. | +| The same three-build pilot, wrap-up stage | Wrap-up used 33 to 53 turns after the changes, compared with 62 to 67 in the baseline. A project-specific runner and batching guidance had both changed. | The stage moved in the useful direction, but no single change gets causal credit. | +| Session 757 administrative follow-up | Most administrative work was already batched or represented real decision points. The remaining avoidable ledger calls were modelled at roughly $4 per chain. | Do not add a generic administrative runner for a small, uncertain remnant. | +| Sessions 763 to 766 compaction pilot | Five measured summary boundaries lowered later per-event cost by 31.8 to 66.3 percent, but the summary's own full-price reread outweighed the recovery. Median net saving was -19.3 percent. Quality stayed flat. | Do not prompt compaction as a savings mechanism. Keep durable recovery checkpoints on correctness merit only. | + +One result sits outside the table because it is easy to misread. Session 744 was expensive, granular work: 284 assistant turns, 343 tool calls, repeated large reads, and an approximate cost near $300. An independent grader also found a real WebSocket authority defect that five review rounds missed. The avoidable part was execution granularity and late mechanical work. The grader paid for itself. + +## What is modelled or still uncertain + +Transcript-derived dollar figures use an approximate rate and reconstructed context. They are useful for locating where a session grew, not for stating what was billed. Compaction makes the in-chain meter less trustworthy because the visible transcript no longer represents everything the provider charged to reread. + +The three-build pilot was heterogeneous and changed three defaults together. It can say quality held. It cannot divide a saving among selective live checks, batching, and freshness-aware rereads, and its billed cost was unavailable. The wrap-up movement is descriptive for the same reason. + +Tool economics also move underneath the process. As checked on August 5, 2026, [Cursor's summarization documentation](https://docs.cursor.com/en/agent/chat/summarization) describes context management rather than a savings guarantee; [Cursor pricing](https://docs.cursor.com/account/pricing), the [Codex rate card](https://help.openai.com/en/articles/20001106-codex-rate-card), and [Anthropic pricing](https://docs.anthropic.com/en/docs/about-claude/pricing) expose different units and cache treatment. These are examples of variation, not Foundry policy. Recheck the live provider before doing money math. + +## Defaults this evidence earns + +- Keep five review rounds in the full path. Use the light path as a lower-assurance choice, never as a promised fraction of the cost. +- Keep live voice checks where prose quality has real room to drift: final plan prose, frame-it, quiz, wrap-up, handoff, and any draft the agent itself finds dense or awkward. Let fixed short reports rely on their shape, while committed prose still passes the blocking scan. +- Read every changed file in full during implementation rounds 1 and 5. In rounds 2 through 4, fully read files changed since the last inspection and whatever the new angle needs. Compaction, intervening edits, or unseen surrounding code force a full reread. +- Use targeted checks while working. Run one complete project check in test-it and one final complete check in wrap-up on a clean chain. A failure or a substantive later change earns another run. +- Batch independent reads, but keep real dependencies ordered. Large output can land in a file with a short chat summary. Never combine unrelated work to improve a turn count. +- Keep the always-loaded rules and source commands below their executable limits. `node scripts/check-context-budgets.js` reports the current weight; `--check` makes an overage fail. + +## How to revisit the call + +Start with an attributable usage or billing export and a flushed transcript. Predeclare which sessions count, which fields will be compared, and the exact window used for every metric. Record quality findings beside cost. If several defaults change together, judge the bundle and refuse per-lever attribution. A useful result can still be "quality held, saving not demonstrated." That is better than a neat percentage built on stale telemetry. diff --git a/docs/wiki/engineering/context-engineering.md b/docs/wiki/engineering/context-engineering.md index 18ab3a0..ce46df1 100644 --- a/docs/wiki/engineering/context-engineering.md +++ b/docs/wiki/engineering/context-engineering.md @@ -22,6 +22,8 @@ Models update on someone else's schedule, and a harness tuned to one generation' Before running two pieces of agent work at the same time, ask whether one's output feeds the other's input. No data crossing means no dependency: run them at once. Data crossing means a real edge: the later step waits. Most sequential chains contain steps that were only typed in order, not dependent in fact, and most parallel ambitions break on edges that turn out to be real (a review round that consumes the previous round's fixes is a real edge, however tempting the speedup). One verification pattern is recorded here for when the evidence arrives: if a single skeptical reviewer provably misses a class of problem, several reviewers with genuinely different lenses (correctness, security, reproduction) catch what identical duplicates never will. Adopt that on evidence of a real miss, not ahead of it. (Codez, "Graph Engineering with Claude," July 2026: https://x.com/0xcodez/status/2079165300625330317. The plan-shape counterparts, program design and vertical slices, are credited in construct-the-plan's rationale to Dex Horthy: https://x.com/dexhorthy/status/2081058573556306030) +See also [chain economics](chain-economics.md) for the measured cost record behind Foundry's context ceilings, selective live checks, and freshness-aware rereads. + ## Sources - **Thariq Shihipar, "The new rules of context engineering for Claude 5 models" (July 2026)** diff --git a/package.json b/package.json index 9836c7b..f080fc3 100644 --- a/package.json +++ b/package.json @@ -8,6 +8,6 @@ "scripts": { "shapes": "node scripts/generate-command-shapes.js", "test": "node --test", - "check": "node --test && node scripts/generate-command-shapes.js --confirm && node scripts/check-links.js && node scripts/check-jargon.js" + "check": "node scripts/check-context-budgets.js --check && node --test && node scripts/generate-command-shapes.js --confirm && node scripts/check-links.js && node scripts/check-jargon.js" } } diff --git a/scripts/check-context-budgets.js b/scripts/check-context-budgets.js new file mode 100644 index 0000000..296dfce --- /dev/null +++ b/scripts/check-context-budgets.js @@ -0,0 +1,113 @@ +#!/usr/bin/env node + +// Keeps Foundry's always-loaded rules and source commands inside the context +// budgets the project promises. Generated command shapes are derived from the +// source commands, so counting them again would charge the same prose twice. + +import { readFileSync, readdirSync } from 'node:fs'; +import { dirname, join, relative, resolve, sep } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; + +export const CONTEXT_LIMITS = Object.freeze({ + agentsBytes: 8_192, + commandBytes: 112_640, +}); + +function readNormalized(path, label) { + try { + return readFileSync(path, 'utf8').replace(/\r\n?|\n/g, '\n'); + } catch (error) { + const reason = error?.code ?? error?.message ?? 'unknown read error'; + throw new Error(`${label} could not be read (${reason})`); + } +} + +export function measureContextBudgets(rootDir) { + const root = resolve(rootDir); + const agentsPath = join(root, 'AGENTS.md'); + const commandsDir = join(root, '.cursor', 'commands'); + const agentsBytes = Buffer.byteLength(readNormalized(agentsPath, 'AGENTS.md'), 'utf8'); + + let entries; + try { + entries = readdirSync(commandsDir, { withFileTypes: true }); + } catch (error) { + const reason = error?.code ?? error?.message ?? 'unknown read error'; + throw new Error(`source command directory could not be read (${reason})`); + } + + const commandFiles = entries + .filter((entry) => entry.isFile() && entry.name.endsWith('.md')) + .map((entry) => { + const path = join(commandsDir, entry.name); + const portablePath = relative(root, path).split(sep).join('/'); + return { + path: portablePath, + bytes: Buffer.byteLength(readNormalized(path, portablePath), 'utf8'), + }; + }) + .sort((a, b) => b.bytes - a.bytes || (a.path < b.path ? -1 : a.path > b.path ? 1 : 0)); + + if (commandFiles.length === 0) { + throw new Error('source command directory contains no markdown commands'); + } + + return { + agentsBytes, + commandBytes: commandFiles.reduce((sum, file) => sum + file.bytes, 0), + commandCount: commandFiles.length, + commandFiles, + }; +} + +export function evaluateContextBudgets(measurement, limits = CONTEXT_LIMITS) { + const violations = []; + if (measurement.agentsBytes > limits.agentsBytes) { + violations.push({ name: 'AGENTS.md', actual: measurement.agentsBytes, limit: limits.agentsBytes }); + } + if (measurement.commandBytes > limits.commandBytes) { + violations.push({ name: 'source commands', actual: measurement.commandBytes, limit: limits.commandBytes }); + } + return { ok: violations.length === 0, violations }; +} + +function printReport(measurement, evaluation) { + console.log(`[context-budgets] AGENTS.md: ${measurement.agentsBytes} / ${CONTEXT_LIMITS.agentsBytes} bytes`); + console.log( + `[context-budgets] source commands: ${measurement.commandBytes} / ${CONTEXT_LIMITS.commandBytes} bytes ` + + `across ${measurement.commandCount} files` + ); + for (const file of measurement.commandFiles) { + console.log(`[context-budgets] ${String(file.bytes).padStart(6)} ${file.path}`); + } + for (const violation of evaluation.violations) { + const over = violation.actual - violation.limit; + console.error( + `[context-budgets] OVER: ${violation.name} is ${violation.actual} bytes, ` + + `${over} byte${over === 1 ? '' : 's'} above its ${violation.limit}-byte limit` + ); + } +} + +function main() { + const args = process.argv.slice(2); + if (args.some((arg) => arg !== '--check') || args.filter((arg) => arg === '--check').length > 1) { + console.error('[context-budgets] usage: node scripts/check-context-budgets.js [--check]'); + process.exitCode = 1; + return; + } + + try { + const root = process.env.CONTEXT_BUDGET_ROOT ?? join(dirname(fileURLToPath(import.meta.url)), '..'); + const measurement = measureContextBudgets(root); + const evaluation = evaluateContextBudgets(measurement); + printReport(measurement, evaluation); + if (args.includes('--check') && !evaluation.ok) process.exitCode = 1; + } catch (error) { + console.error(`[context-budgets] ERROR: ${error.message}`); + process.exitCode = 1; + } +} + +const invokedDirectly = process.argv[1] && import.meta.url === pathToFileURL(resolve(process.argv[1])).href; +if (invokedDirectly) main(); diff --git a/tests/check-context-budgets.test.js b/tests/check-context-budgets.test.js new file mode 100644 index 0000000..871dec8 --- /dev/null +++ b/tests/check-context-budgets.test.js @@ -0,0 +1,149 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { execFileSync } from 'node:child_process'; +import { chmodSync, mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; + +import { + CONTEXT_LIMITS, + evaluateContextBudgets, + measureContextBudgets, +} from '../scripts/check-context-budgets.js'; + +const repoRoot = join(dirname(fileURLToPath(import.meta.url)), '..'); +const script = join(repoRoot, 'scripts', 'check-context-budgets.js'); + +function makeFixture({ agents = 'rules\n', commands = { 'alpha.md': 'command\n' } } = {}) { + const root = mkdtempSync(join(tmpdir(), 'context-budgets-')); + writeFileSync(join(root, 'AGENTS.md'), agents); + if (commands !== null) { + mkdirSync(join(root, '.cursor', 'commands'), { recursive: true }); + for (const [name, body] of Object.entries(commands)) { + writeFileSync(join(root, '.cursor', 'commands', name), body); + } + } + return root; +} + +function run(root, args = []) { + try { + const out = execFileSync('node', [script, ...args], { + env: { ...process.env, CONTEXT_BUDGET_ROOT: root }, + encoding: 'utf8', + }); + return { code: 0, out }; + } catch (error) { + return { code: error.status, out: `${error.stdout ?? ''}${error.stderr ?? ''}` }; + } +} + +test('passes ordinary input and reports the measured command set', () => { + const root = makeFixture({ commands: { 'beta.md': '12345', 'alpha.md': '123' } }); + try { + const measurement = measureContextBudgets(root); + assert.deepEqual(measurement.commandFiles, [ + { path: '.cursor/commands/beta.md', bytes: 5 }, + { path: '.cursor/commands/alpha.md', bytes: 3 }, + ]); + assert.equal(measurement.commandBytes, 8); + assert.equal(measurement.commandCount, 2); + assert.deepEqual(evaluateContextBudgets(measurement).violations, []); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test('the exact boundaries pass and one byte over fails with an exact overage', () => { + const root = makeFixture({ + agents: 'a'.repeat(CONTEXT_LIMITS.agentsBytes), + commands: { 'alpha.md': 'b'.repeat(CONTEXT_LIMITS.commandBytes) }, + }); + try { + assert.equal(evaluateContextBudgets(measureContextBudgets(root)).ok, true); + writeFileSync( + join(root, '.cursor', 'commands', 'alpha.md'), + 'b'.repeat(CONTEXT_LIMITS.commandBytes + 1) + ); + const evaluation = evaluateContextBudgets(measureContextBudgets(root)); + assert.deepEqual(evaluation.violations, [ + { + name: 'source commands', + actual: CONTEXT_LIMITS.commandBytes + 1, + limit: CONTEXT_LIMITS.commandBytes, + }, + ]); + const checked = run(root, ['--check']); + assert.equal(checked.code, 1); + assert.ok(checked.out.includes('1 byte above')); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test('normalizes CRLF and lone CR before measuring', () => { + const lf = makeFixture({ agents: 'one\ntwo\n', commands: { 'alpha.md': 'a\nb\n' } }); + const mixed = makeFixture({ agents: 'one\r\ntwo\r', commands: { 'alpha.md': 'a\r\nb\r' } }); + try { + assert.deepEqual(measureContextBudgets(mixed), measureContextBudgets(lf)); + } finally { + rmSync(lf, { recursive: true, force: true }); + rmSync(mixed, { recursive: true, force: true }); + } +}); + +test('refuses missing AGENTS.md, a missing command directory, and an empty command directory', () => { + const missingAgents = makeFixture(); + const missingCommands = makeFixture({ commands: null }); + const emptyCommands = makeFixture({ commands: {} }); + rmSync(join(missingAgents, 'AGENTS.md')); + try { + assert.throws(() => measureContextBudgets(missingAgents), /AGENTS\.md could not be read/); + assert.throws(() => measureContextBudgets(missingCommands), /source command directory could not be read/); + assert.throws(() => measureContextBudgets(emptyCommands), /contains no markdown commands/); + } finally { + rmSync(missingAgents, { recursive: true, force: true }); + rmSync(missingCommands, { recursive: true, force: true }); + rmSync(emptyCommands, { recursive: true, force: true }); + } +}); + +test('refuses an unreadable command file', () => { + const root = makeFixture(); + const file = join(root, '.cursor', 'commands', 'alpha.md'); + chmodSync(file, 0o000); + try { + assert.throws(() => measureContextBudgets(root), /alpha\.md could not be read/); + } finally { + chmodSync(file, 0o600); + rmSync(root, { recursive: true, force: true }); + } +}); + +test('report mode stays informational while check mode enforces the ceiling', () => { + const root = makeFixture({ agents: 'a'.repeat(CONTEXT_LIMITS.agentsBytes + 1) }); + try { + const report = run(root); + assert.equal(report.code, 0); + assert.ok(report.out.includes('AGENTS.md')); + assert.ok(report.out.includes('.cursor/commands/alpha.md')); + assert.ok(report.out.indexOf('source commands') < report.out.indexOf('.cursor/commands/alpha.md')); + + const checked = run(root, ['--check']); + assert.equal(checked.code, 1); + assert.ok(checked.out.includes('OVER')); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test('importing the checker never measures the checkout or exits', () => { + const missingRoot = join(tmpdir(), 'context-budget-root-that-does-not-exist'); + const out = execFileSync( + 'node', + ['--input-type=module', '--eval', `await import('${pathToFileURL(script).href}'); console.log('imported')`], + { env: { ...process.env, CONTEXT_BUDGET_ROOT: missingRoot }, encoding: 'utf8' } + ); + assert.equal(out, 'imported\n'); +}); diff --git a/tests/install.test.js b/tests/install.test.js index 1641909..e8b7956 100644 --- a/tests/install.test.js +++ b/tests/install.test.js @@ -28,6 +28,7 @@ test('fresh install populates the documented set and only that set', () => { assert.ok(existsSync(join(target, 'AGENTS.md'))); assert.ok(existsSync(join(target, 'CLAUDE.md'))); assert.ok(existsSync(join(target, 'scripts', 'phrase-list.json'))); + assert.ok(existsSync(join(target, 'scripts', 'check-context-budgets.js'))); assert.ok(!existsSync(join(target, 'docs')), 'wiki must stay home without --wiki'); assert.ok(!existsSync(join(target, 'README.md')), 'Foundry\'s own README must not ride along'); assert.ok(!existsSync(join(target, 'tests')), 'Foundry\'s tests must not ride along'); @@ -35,6 +36,18 @@ test('fresh install populates the documented set and only that set', () => { rmSync(target, { recursive: true, force: true }); }); +test('install copies the budget checker without rewriting consumer package scripts', () => { + const target = mkdtempSync(join(tmpdir(), 'install-package-boundary-')); + const packageFile = join(target, 'package.json'); + const original = JSON.stringify({ scripts: { check: 'consumer-owned-check' } }, null, 2) + '\n'; + writeFileSync(packageFile, original); + const r = run([target]); + assert.equal(r.code, 0); + assert.ok(existsSync(join(target, 'scripts', 'check-context-budgets.js'))); + assert.equal(readFileSync(packageFile, 'utf8'), original); + rmSync(target, { recursive: true, force: true }); +}); + test('--wiki includes the reference library', () => { const target = mkdtempSync(join(tmpdir(), 'install-wiki-')); const r = run([target, '--wiki']); From 1520809efaa3c66ab08ced5b9f3833cadfd260f6 Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:25:30 -0400 Subject: [PATCH 05/13] Keep chain assurance while removing repeated ritual --- .agents/skills/build-it/SKILL.md | 8 +- .../challenge-implementation-1/SKILL.md | 4 - .../challenge-implementation-2/SKILL.md | 6 +- .../challenge-implementation-3/SKILL.md | 6 +- .../challenge-implementation-4/SKILL.md | 6 +- .../challenge-implementation-5/SKILL.md | 6 +- .agents/skills/challenge-plan-1/SKILL.md | 4 - .agents/skills/challenge-plan-2/SKILL.md | 4 - .agents/skills/challenge-plan-3/SKILL.md | 4 - .agents/skills/challenge-plan-4/SKILL.md | 4 - .agents/skills/challenge-plan-5/SKILL.md | 4 - .agents/skills/security-scan/SKILL.md | 4 - .agents/skills/start-up/SKILL.md | 4 - .agents/skills/test-it/SKILL.md | 10 +- .agents/skills/wrap-up/SKILL.md | 2 +- .claude/commands/build-it.md | 8 +- .../commands/challenge-implementation-1.md | 4 - .../commands/challenge-implementation-2.md | 6 +- .../commands/challenge-implementation-3.md | 6 +- .../commands/challenge-implementation-4.md | 6 +- .../commands/challenge-implementation-5.md | 6 +- .claude/commands/challenge-plan-1.md | 4 - .claude/commands/challenge-plan-2.md | 4 - .claude/commands/challenge-plan-3.md | 4 - .claude/commands/challenge-plan-4.md | 4 - .claude/commands/challenge-plan-5.md | 4 - .claude/commands/security-scan.md | 4 - .claude/commands/start-up.md | 4 - .claude/commands/test-it.md | 10 +- .claude/commands/wrap-up.md | 2 +- .cursor/commands/build-it.md | 8 +- .../commands/challenge-implementation-1.md | 4 - .../commands/challenge-implementation-2.md | 6 +- .../commands/challenge-implementation-3.md | 6 +- .../commands/challenge-implementation-4.md | 6 +- .../commands/challenge-implementation-5.md | 6 +- .cursor/commands/challenge-plan-1.md | 4 - .cursor/commands/challenge-plan-2.md | 4 - .cursor/commands/challenge-plan-3.md | 4 - .cursor/commands/challenge-plan-4.md | 4 - .cursor/commands/challenge-plan-5.md | 4 - .cursor/commands/security-scan.md | 4 - .cursor/commands/start-up.md | 4 - .cursor/commands/test-it.md | 10 +- .cursor/commands/wrap-up.md | 2 +- AGENTS.md | 4 +- docs/plans/chain-economics.md | 2 + tests/chain-economics-contract.test.js | 107 ++++++++++++++++++ 48 files changed, 156 insertions(+), 185 deletions(-) create mode 100644 tests/chain-economics-contract.test.js diff --git a/.agents/skills/build-it/SKILL.md b/.agents/skills/build-it/SKILL.md index f826e9b..672717f 100644 --- a/.agents/skills/build-it/SKILL.md +++ b/.agents/skills/build-it/SKILL.md @@ -7,10 +7,6 @@ description: 'When the human types `/build-it` (or "build it", "build this", "im When the human types `/build-it` (or "build it", "build this", "implement it", "go build"), the plan is settled. Start implementing per the challenged plan. From here to handoff the chain runs without questions to the human, per AGENTS.md § The flow guarantee; the recovery paths below are the only sanctioned stops. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## The unattended-chain rule (why the stops below look the way they do) The human may queue the entire remaining chain (challenge rounds, test-it, security-scan, wrap-up, handoff) and walk away. Each queued command fires the moment the previous turn ends. Three implications: @@ -37,6 +33,10 @@ No plan survives contact with the code untouched, and under an unattended chain The log lives in the plan file's `## Deviations` section, one bullet per departure with four parts: what the plan said, what the code forced, what was chosen, and a one-line lesson. Cite code locations. Implementation-challenge round 1 reads this section first and treats every entry as a prime target, because deviations are exactly where the work left the reviewed plan. +## During implementation: keep tool use coherent and checks targeted + +Run independent reads or inspections together, but keep steps ordered when one's output feeds the next. Use bounded reads when only one section is needed. Put large command output in a file and return the useful summary without hiding evidence. Run targeted checks for the files and behavior being changed; test-it owns the first complete project check. Never combine unrelated edits or skip output just to improve a turn count. + ## During implementation: commit as units land, no pull request Work commits and pushes to the feature branch as coherent units complete. Do NOT open a pull request during build-it or the review rounds that follow; that's wrap-up's job, once, at the end. No-git mode: edit in place, and keep a running list of what changed in the plan file. diff --git a/.agents/skills/challenge-implementation-1/SKILL.md b/.agents/skills/challenge-implementation-1/SKILL.md index 4bf2cd9..a7e19a1 100644 --- a/.agents/skills/challenge-implementation-1/SKILL.md +++ b/.agents/skills/challenge-implementation-1/SKILL.md @@ -7,10 +7,6 @@ description: 'Round 1 of implementation-challenge.' Round 1 of implementation-challenge. Trace through the changes you just made and look for bugs or edge cases that weren't obvious. Aim for a thorough audit, not a quick scan. Fix what you find; don't just surface it. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation (no code edits, no file writes by the agent), decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." Don't invent imaginary changes to challenge. diff --git a/.agents/skills/challenge-implementation-2/SKILL.md b/.agents/skills/challenge-implementation-2/SKILL.md index 5e78e6f..788f83d 100644 --- a/.agents/skills/challenge-implementation-2/SKILL.md +++ b/.agents/skills/challenge-implementation-2/SKILL.md @@ -7,10 +7,6 @@ description: 'Round 2 of implementation-challenge.' Round 2 of implementation-challenge. Given how much this work matters, trace through the changes again from a different angle. Look for bugs or edge cases that weren't obvious in round 1. Fix what you find. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation, decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." @@ -30,7 +26,7 @@ If a different angle is clearly more relevant, override the default and say why. 1. **Read back your prior turns.** List every angle used so far (original implementation + round 1). Format: `Angles used: [round 0: X], [round 1: Y]`. 2. **Confirm the round-2 angle**: either "using the default for this round: check whether the earlier rounds' fixes broke anything new" OR "overriding the default to [angle]: [reason]". The angle must be different from rounds 0 and 1. -Then re-read each modified file in full surrounding context, paying particular attention to changes made during round 1. +Read every file changed during round 1 in full surrounding context, plus whatever files the round-2 angle newly requires. A file untouched since the last inspection does not need reprinting unless the conversation was compacted, the file changed since inspection, or this angle needs surrounding code no earlier round inspected. Any of those conditions forces a full read. ## What to look for diff --git a/.agents/skills/challenge-implementation-3/SKILL.md b/.agents/skills/challenge-implementation-3/SKILL.md index 3c4bb54..2e08716 100644 --- a/.agents/skills/challenge-implementation-3/SKILL.md +++ b/.agents/skills/challenge-implementation-3/SKILL.md @@ -7,10 +7,6 @@ description: 'Round 3 of implementation-challenge.' Round 3 of implementation-challenge. Are you 200% sure you've covered this from every angle? Trace through the changes again from the round-3 angle. Fix what you find. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation, decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." @@ -30,7 +26,7 @@ If a different angle is clearly more relevant, override the default and say why. 1. **Read back your prior turns.** List every angle used so far (original implementation + rounds 1-2). Format: `Angles used: [round 0: X], [round 1: Y], [round 2: Z]`. 2. **Confirm the round-3 angle**: either "using the default for this round: how this code talks to its neighbours" OR "overriding the default to [angle]: [reason]". The angle must be different from rounds 0-2. -Then re-read each modified file in full surrounding context and trace every seam. +Read every file changed during round 2 in full surrounding context, plus callers, consumers, or other files the round-3 angle newly requires. A file untouched since the last inspection does not need reprinting unless the conversation was compacted, the file changed since inspection, or this angle needs surrounding code no earlier round inspected. Any of those conditions forces a full read. Then trace every seam. ## What to look for diff --git a/.agents/skills/challenge-implementation-4/SKILL.md b/.agents/skills/challenge-implementation-4/SKILL.md index 034ae45..69c8fc2 100644 --- a/.agents/skills/challenge-implementation-4/SKILL.md +++ b/.agents/skills/challenge-implementation-4/SKILL.md @@ -7,10 +7,6 @@ description: 'Round 4 of implementation-challenge.' Round 4 of implementation-challenge. Trace through the changes again from the round-4 angle. Look for bugs the prior three rounds missed. Fix what you find. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation, decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." @@ -34,6 +30,8 @@ If the work touches no external services AND has no peer precedent to compare ag 1. **Read back your prior turns.** List every angle used so far (original implementation + rounds 1-3). Format: `Angles used: [round 0: X], [round 1: Y], [round 2: Z], [round 3: W]`. 2. **Confirm the round-4 angle**: either "using the default for this round: external concerns" OR "overriding the default to [angle]: [reason]". The angle must be different from rounds 0-3. +Read every file changed during round 3 in full surrounding context, plus whatever files or sources the round-4 angle newly requires. A file untouched since the last inspection does not need reprinting unless the conversation was compacted, the file changed since inspection, or this angle needs surrounding code no earlier round inspected. Any of those conditions forces a full read. + ## Useful angles to rotate through The canonical list lives in AGENTS.md § The challenge rounds: implementation / where data flows and what happens when something fails / how this code talks to its neighbours / how this code talks to outside services / config-file syntax and quoting pitfalls / what unusual or hostile inputs would break this? / what happens when shell scripts fail mid-pipeline? / will this code be discoverable when it feels unfamiliar? / is the sequence of steps right? / where else does the same root cause apply? / check whether the earlier rounds' fixes broke anything new / does this convention reach every place the mechanism reaches? diff --git a/.agents/skills/challenge-implementation-5/SKILL.md b/.agents/skills/challenge-implementation-5/SKILL.md index de17a1b..e46604f 100644 --- a/.agents/skills/challenge-implementation-5/SKILL.md +++ b/.agents/skills/challenge-implementation-5/SKILL.md @@ -7,10 +7,6 @@ description: 'Round 5 of implementation-challenge.' Round 5 of implementation-challenge. Final-pass deep audit. Are you 200% sure you've covered this from every angle and fixed every bug? If not, keep tracing. Fix what you find. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation, decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." @@ -52,6 +48,8 @@ Look for: 1. **Read back your prior turns.** List every angle used so far (original implementation + rounds 1-4). Format: `Angles used: [round 0: V], [round 1: W], [round 2: X], [round 3: Y], [round 4: Z]`. 2. **Confirm the round-5 angle**: either "using the default for this round: over-engineering check and simpler solutions" OR "overriding the default to [angle]: [reason]". +Before hunting for cuts, re-read every file modified since implementation began in full surrounding context. Do not trust the diff; this round judges the accumulated implementation, including fixes from rounds 1 through 4. + ## Auto-fix bugs found (when K > 0) Same policy as round 1: fix now via a new commit on the feature branch (no pull request until wrap-up; in no-git mode fix in place and log it). Out-of-scope findings become observations, human-territory bugs defer with `[NEEDS HUMAN-R5: ]`. K counts high-confidence defects only; a cut that changes behavior needs the same confidence bar as a fix. diff --git a/.agents/skills/challenge-plan-1/SKILL.md b/.agents/skills/challenge-plan-1/SKILL.md index 0cd4918..ee81916 100644 --- a/.agents/skills/challenge-plan-1/SKILL.md +++ b/.agents/skills/challenge-plan-1/SKILL.md @@ -7,10 +7,6 @@ description: 'Round 1 of plan-challenge.' Round 1 of plan-challenge. Are you sure you've done all the research and followed every thread needed to reach this conclusion? If not, keep going. Look for opportunities you might be missing. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no plan was presented in this conversation (no construct-the-plan run, no inline plan content, no plan file referenced), decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." Don't invent an imaginary plan to challenge. diff --git a/.agents/skills/challenge-plan-2/SKILL.md b/.agents/skills/challenge-plan-2/SKILL.md index fc156b9..f96f955 100644 --- a/.agents/skills/challenge-plan-2/SKILL.md +++ b/.agents/skills/challenge-plan-2/SKILL.md @@ -7,10 +7,6 @@ description: 'Round 2 of plan-challenge.' Round 2 of plan-challenge. Given how much this matters, trace through the plan again from a different angle. Be sure you've done all the research and followed every thread. Look for opportunities you might be missing. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no plan was presented in this conversation (no construct-the-plan run, no inline plan content, no plan file referenced), decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." Don't invent an imaginary plan to challenge. diff --git a/.agents/skills/challenge-plan-3/SKILL.md b/.agents/skills/challenge-plan-3/SKILL.md index bf1b628..21fb99f 100644 --- a/.agents/skills/challenge-plan-3/SKILL.md +++ b/.agents/skills/challenge-plan-3/SKILL.md @@ -7,10 +7,6 @@ description: 'Round 3 of plan-challenge.' Round 3 of plan-challenge. Are you 200% sure you've covered this from every angle? If not, keep going. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no plan was presented in this conversation (no construct-the-plan run, no inline plan content, no plan file referenced), decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." Don't invent an imaginary plan to challenge. diff --git a/.agents/skills/challenge-plan-4/SKILL.md b/.agents/skills/challenge-plan-4/SKILL.md index 6cc6cbb..02cabb1 100644 --- a/.agents/skills/challenge-plan-4/SKILL.md +++ b/.agents/skills/challenge-plan-4/SKILL.md @@ -7,10 +7,6 @@ description: 'Round 4 of plan-challenge.' Round 4 of plan-challenge. Are you 200% sure you've covered this from every angle? If not, keep going. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no plan was presented in this conversation (no construct-the-plan run, no inline plan content, no plan file referenced), decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." Don't invent an imaginary plan to challenge. diff --git a/.agents/skills/challenge-plan-5/SKILL.md b/.agents/skills/challenge-plan-5/SKILL.md index 425637b..5940fd7 100644 --- a/.agents/skills/challenge-plan-5/SKILL.md +++ b/.agents/skills/challenge-plan-5/SKILL.md @@ -7,10 +7,6 @@ description: 'Round 5 of plan-challenge.' Round 5 of plan-challenge. The demotion-review round. Different shape from rounds 1-4: instead of picking a new angle and hunting new findings, round 5 deep-reads every claim the prior rounds parked as uncertain (the "demoted claims") and verifies whether each demotion was warranted. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. Say it plainly: "I went back through the claims I parked as uncertain; two were dismissed too quickly and go back in, three stay uncertain, one turned out to be a real problem and I've fixed the plan." - ## Guard: nothing to challenge If no plan was presented in this conversation, decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." diff --git a/.agents/skills/security-scan/SKILL.md b/.agents/skills/security-scan/SKILL.md index 367c36e..8d39941 100644 --- a/.agents/skills/security-scan/SKILL.md +++ b/.agents/skills/security-scan/SKILL.md @@ -7,10 +7,6 @@ description: 'Single-round adversarial security scan.' Single-round adversarial security scan. Scan a focused surface for findings anchored to named threat classes, argue against each candidate before believing it, and require an articulated reproduction path before anything gets reported. Run it after editing security-sensitive code (auth, payments, user data, anything that parses untrusted input) and BEFORE the implementation-challenge chain. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to scan The scan covers the files modified by the agent in this conversation. If none, decline with: "No scan scope detected; make code edits in this conversation first, or name file paths explicitly after the command." Don't invent imaginary changes to scan. diff --git a/.agents/skills/start-up/SKILL.md b/.agents/skills/start-up/SKILL.md index d9ce18c..73c789e 100644 --- a/.agents/skills/start-up/SKILL.md +++ b/.agents/skills/start-up/SKILL.md @@ -7,10 +7,6 @@ description: 'Run the session start-up.' Run the session start-up. This is the chain's opener and the reading half of the pair that makes sessions continuous: handoff writes the note, start-up reads it. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Step 1: Preflight Check the ground you're standing on and act on what you find: diff --git a/.agents/skills/test-it/SKILL.md b/.agents/skills/test-it/SKILL.md index 3c56637..847e69c 100644 --- a/.agents/skills/test-it/SKILL.md +++ b/.agents/skills/test-it/SKILL.md @@ -7,10 +7,6 @@ description: 'When the human types `/test-it` (or "test it", "cover it", "write When the human types `/test-it` (or "test it", "cover it", "write tests"), the implementation from build-it is in place and this stage proves it works. It sits between build-it and security-scan in the chain. It triages the change, writes new behavioral tests for what was just built, drives them to green, and stops the chain if it can't. No questions to the human except the halt case at the bottom (AGENTS.md § The flow guarantee). -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Build the check, don't punt verification If you're about to write "worth a manual check," stop and build the check instead: a test, a script, a measurement, an automated walk-through. Verification is the agent's job. Handing the human a manual test is a failure of this stage, not diligence. A tool you need but can't reach is an access request, stated plainly; it is never a manual-test handoff. @@ -52,7 +48,9 @@ Three cases when a test is red: Two fix-and-rerun tries is the cap before halting (Step 5). -When the suite reaches green, run it once more before trusting it: a pass that doesn't reproduce is a red (flaky tests are how broken work slips through), and it routes through the same three cases above. +When the behavioral tests reach green, discover the project's canonical complete check in this order: explicit project rules, the verification commands the CI workflow actually runs, then package scripts if neither stronger source names it. Several CI verification commands remain an ordered set; don't guess a single replacement. If none defines a complete check, run the broadest existing suite and name the missing project-level check in the report instead of inventing a command. Run that complete check or set once. A red result routes through the same three cases above, and a substantive change after the green result earns another run. + +Targeted checks remain unlimited while fixing. A new test involving time, randomness, concurrency, or an outside process gets one targeted canary rerun even when it first passes; deterministic tests do not need an unconditional second run. A fourth case sits outside the code entirely: when an outside service fails mid-stage (an API down, a platform change), that is the world breaking, not the work. Log it, name it in the report's Deferred field, and never "fix" working code in response. @@ -82,4 +80,4 @@ K counts real defects the tests revealed, fixed by changing the code. Report the ## Rationale (recorded so future edits don't drift it) -The stage exists because review rounds judging untested code argue about reasoning, while rounds judging tested code argue about evidence. The never-weaken rule and the plan-derived expectations both counter the same failure: an agent grading its own homework drifts toward whatever the code already does. The grader step brings in the one reviewer that can't recognize its own work, and the cited NeurIPS 2024 result is why that seat matters. The run-it-twice rule and the outside-service case adapt the acceptance-testing discipline in Mike Fishbein's infinite-headcount factory skills (flaky greens hide broken builds; a failure of the world is never a reason to edit working code). Recorded so the lineage survives edits. +The stage exists because review rounds judging untested code argue about reasoning, while rounds judging tested code argue about evidence. The never-weaken rule and the plan-derived expectations both counter the same failure: an agent grading its own homework drifts toward whatever the code already does. The grader step brings in the one reviewer that can't recognize its own work, and the cited NeurIPS 2024 result is why that seat matters. The narrow canary rule and the outside-service case adapt the acceptance-testing discipline in Mike Fishbein's infinite-headcount factory skills: risky tests must reproduce, while a failure of the world is never a reason to edit working code. Recorded so the lineage survives edits. diff --git a/.agents/skills/wrap-up/SKILL.md b/.agents/skills/wrap-up/SKILL.md index fa6e3c9..dd1d532 100644 --- a/.agents/skills/wrap-up/SKILL.md +++ b/.agents/skills/wrap-up/SKILL.md @@ -17,7 +17,7 @@ Nothing else starts until every background task this conversation spawned is res ## Step 1: Cleanup pass -Re-read every file the session modified, in place, not as a diff: dead code out, unused imports out, naming consistent, comments carrying intent rather than narration. Run the project's check or lint suite and fix what it finds. +Re-read every file the session modified, in place, not as a diff: dead code out, unused imports out, naming consistent, comments carrying intent rather than narration. Then discover the canonical complete check in the same order as test-it: explicit project rules, the verification commands CI actually runs, then package scripts. Preserve several CI commands as an ordered set. If no source names a complete check, run the broadest existing suite and report the missing project-level check instead of inventing one. Run the discovered check or set once after the cumulative review. A red result or a substantive change after that run earns another complete check; otherwise don't repeat it. ## Step 2: Review the cumulative session work diff --git a/.claude/commands/build-it.md b/.claude/commands/build-it.md index fd143f2..5b965e9 100644 --- a/.claude/commands/build-it.md +++ b/.claude/commands/build-it.md @@ -1,9 +1,5 @@ When the human types `/build-it` (or "build it", "build this", "implement it", "go build"), the plan is settled. Start implementing per the challenged plan. From here to handoff the chain runs without questions to the human, per AGENTS.md § The flow guarantee; the recovery paths below are the only sanctioned stops. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## The unattended-chain rule (why the stops below look the way they do) The human may queue the entire remaining chain (challenge rounds, test-it, security-scan, wrap-up, handoff) and walk away. Each queued command fires the moment the previous turn ends. Three implications: @@ -30,6 +26,10 @@ No plan survives contact with the code untouched, and under an unattended chain The log lives in the plan file's `## Deviations` section, one bullet per departure with four parts: what the plan said, what the code forced, what was chosen, and a one-line lesson. Cite code locations. Implementation-challenge round 1 reads this section first and treats every entry as a prime target, because deviations are exactly where the work left the reviewed plan. +## During implementation: keep tool use coherent and checks targeted + +Run independent reads or inspections together, but keep steps ordered when one's output feeds the next. Use bounded reads when only one section is needed. Put large command output in a file and return the useful summary without hiding evidence. Run targeted checks for the files and behavior being changed; test-it owns the first complete project check. Never combine unrelated edits or skip output just to improve a turn count. + ## During implementation: commit as units land, no pull request Work commits and pushes to the feature branch as coherent units complete. Do NOT open a pull request during build-it or the review rounds that follow; that's wrap-up's job, once, at the end. No-git mode: edit in place, and keep a running list of what changed in the plan file. diff --git a/.claude/commands/challenge-implementation-1.md b/.claude/commands/challenge-implementation-1.md index bcabcff..2e5ab29 100644 --- a/.claude/commands/challenge-implementation-1.md +++ b/.claude/commands/challenge-implementation-1.md @@ -1,9 +1,5 @@ Round 1 of implementation-challenge. Trace through the changes you just made and look for bugs or edge cases that weren't obvious. Aim for a thorough audit, not a quick scan. Fix what you find; don't just surface it. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation (no code edits, no file writes by the agent), decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." Don't invent imaginary changes to challenge. diff --git a/.claude/commands/challenge-implementation-2.md b/.claude/commands/challenge-implementation-2.md index 228a95c..50bfdb1 100644 --- a/.claude/commands/challenge-implementation-2.md +++ b/.claude/commands/challenge-implementation-2.md @@ -1,9 +1,5 @@ Round 2 of implementation-challenge. Given how much this work matters, trace through the changes again from a different angle. Look for bugs or edge cases that weren't obvious in round 1. Fix what you find. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation, decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." @@ -23,7 +19,7 @@ If a different angle is clearly more relevant, override the default and say why. 1. **Read back your prior turns.** List every angle used so far (original implementation + round 1). Format: `Angles used: [round 0: X], [round 1: Y]`. 2. **Confirm the round-2 angle**: either "using the default for this round: check whether the earlier rounds' fixes broke anything new" OR "overriding the default to [angle]: [reason]". The angle must be different from rounds 0 and 1. -Then re-read each modified file in full surrounding context, paying particular attention to changes made during round 1. +Read every file changed during round 1 in full surrounding context, plus whatever files the round-2 angle newly requires. A file untouched since the last inspection does not need reprinting unless the conversation was compacted, the file changed since inspection, or this angle needs surrounding code no earlier round inspected. Any of those conditions forces a full read. ## What to look for diff --git a/.claude/commands/challenge-implementation-3.md b/.claude/commands/challenge-implementation-3.md index b460641..313a00e 100644 --- a/.claude/commands/challenge-implementation-3.md +++ b/.claude/commands/challenge-implementation-3.md @@ -1,9 +1,5 @@ Round 3 of implementation-challenge. Are you 200% sure you've covered this from every angle? Trace through the changes again from the round-3 angle. Fix what you find. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation, decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." @@ -23,7 +19,7 @@ If a different angle is clearly more relevant, override the default and say why. 1. **Read back your prior turns.** List every angle used so far (original implementation + rounds 1-2). Format: `Angles used: [round 0: X], [round 1: Y], [round 2: Z]`. 2. **Confirm the round-3 angle**: either "using the default for this round: how this code talks to its neighbours" OR "overriding the default to [angle]: [reason]". The angle must be different from rounds 0-2. -Then re-read each modified file in full surrounding context and trace every seam. +Read every file changed during round 2 in full surrounding context, plus callers, consumers, or other files the round-3 angle newly requires. A file untouched since the last inspection does not need reprinting unless the conversation was compacted, the file changed since inspection, or this angle needs surrounding code no earlier round inspected. Any of those conditions forces a full read. Then trace every seam. ## What to look for diff --git a/.claude/commands/challenge-implementation-4.md b/.claude/commands/challenge-implementation-4.md index 0d05e35..903b64e 100644 --- a/.claude/commands/challenge-implementation-4.md +++ b/.claude/commands/challenge-implementation-4.md @@ -1,9 +1,5 @@ Round 4 of implementation-challenge. Trace through the changes again from the round-4 angle. Look for bugs the prior three rounds missed. Fix what you find. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation, decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." @@ -27,6 +23,8 @@ If the work touches no external services AND has no peer precedent to compare ag 1. **Read back your prior turns.** List every angle used so far (original implementation + rounds 1-3). Format: `Angles used: [round 0: X], [round 1: Y], [round 2: Z], [round 3: W]`. 2. **Confirm the round-4 angle**: either "using the default for this round: external concerns" OR "overriding the default to [angle]: [reason]". The angle must be different from rounds 0-3. +Read every file changed during round 3 in full surrounding context, plus whatever files or sources the round-4 angle newly requires. A file untouched since the last inspection does not need reprinting unless the conversation was compacted, the file changed since inspection, or this angle needs surrounding code no earlier round inspected. Any of those conditions forces a full read. + ## Useful angles to rotate through The canonical list lives in AGENTS.md § The challenge rounds: implementation / where data flows and what happens when something fails / how this code talks to its neighbours / how this code talks to outside services / config-file syntax and quoting pitfalls / what unusual or hostile inputs would break this? / what happens when shell scripts fail mid-pipeline? / will this code be discoverable when it feels unfamiliar? / is the sequence of steps right? / where else does the same root cause apply? / check whether the earlier rounds' fixes broke anything new / does this convention reach every place the mechanism reaches? diff --git a/.claude/commands/challenge-implementation-5.md b/.claude/commands/challenge-implementation-5.md index bcbdef4..d3bc2c0 100644 --- a/.claude/commands/challenge-implementation-5.md +++ b/.claude/commands/challenge-implementation-5.md @@ -1,9 +1,5 @@ Round 5 of implementation-challenge. Final-pass deep audit. Are you 200% sure you've covered this from every angle and fixed every bug? If not, keep tracing. Fix what you find. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation, decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." @@ -45,6 +41,8 @@ Look for: 1. **Read back your prior turns.** List every angle used so far (original implementation + rounds 1-4). Format: `Angles used: [round 0: V], [round 1: W], [round 2: X], [round 3: Y], [round 4: Z]`. 2. **Confirm the round-5 angle**: either "using the default for this round: over-engineering check and simpler solutions" OR "overriding the default to [angle]: [reason]". +Before hunting for cuts, re-read every file modified since implementation began in full surrounding context. Do not trust the diff; this round judges the accumulated implementation, including fixes from rounds 1 through 4. + ## Auto-fix bugs found (when K > 0) Same policy as round 1: fix now via a new commit on the feature branch (no pull request until wrap-up; in no-git mode fix in place and log it). Out-of-scope findings become observations, human-territory bugs defer with `[NEEDS HUMAN-R5: ]`. K counts high-confidence defects only; a cut that changes behavior needs the same confidence bar as a fix. diff --git a/.claude/commands/challenge-plan-1.md b/.claude/commands/challenge-plan-1.md index 84538a6..d86a611 100644 --- a/.claude/commands/challenge-plan-1.md +++ b/.claude/commands/challenge-plan-1.md @@ -1,9 +1,5 @@ Round 1 of plan-challenge. Are you sure you've done all the research and followed every thread needed to reach this conclusion? If not, keep going. Look for opportunities you might be missing. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no plan was presented in this conversation (no construct-the-plan run, no inline plan content, no plan file referenced), decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." Don't invent an imaginary plan to challenge. diff --git a/.claude/commands/challenge-plan-2.md b/.claude/commands/challenge-plan-2.md index 95ef0f7..52a8809 100644 --- a/.claude/commands/challenge-plan-2.md +++ b/.claude/commands/challenge-plan-2.md @@ -1,9 +1,5 @@ Round 2 of plan-challenge. Given how much this matters, trace through the plan again from a different angle. Be sure you've done all the research and followed every thread. Look for opportunities you might be missing. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no plan was presented in this conversation (no construct-the-plan run, no inline plan content, no plan file referenced), decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." Don't invent an imaginary plan to challenge. diff --git a/.claude/commands/challenge-plan-3.md b/.claude/commands/challenge-plan-3.md index 8606e84..9f2e61d 100644 --- a/.claude/commands/challenge-plan-3.md +++ b/.claude/commands/challenge-plan-3.md @@ -1,9 +1,5 @@ Round 3 of plan-challenge. Are you 200% sure you've covered this from every angle? If not, keep going. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no plan was presented in this conversation (no construct-the-plan run, no inline plan content, no plan file referenced), decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." Don't invent an imaginary plan to challenge. diff --git a/.claude/commands/challenge-plan-4.md b/.claude/commands/challenge-plan-4.md index 032ad54..137ebb8 100644 --- a/.claude/commands/challenge-plan-4.md +++ b/.claude/commands/challenge-plan-4.md @@ -1,9 +1,5 @@ Round 4 of plan-challenge. Are you 200% sure you've covered this from every angle? If not, keep going. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no plan was presented in this conversation (no construct-the-plan run, no inline plan content, no plan file referenced), decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." Don't invent an imaginary plan to challenge. diff --git a/.claude/commands/challenge-plan-5.md b/.claude/commands/challenge-plan-5.md index b0c28e3..77245f4 100644 --- a/.claude/commands/challenge-plan-5.md +++ b/.claude/commands/challenge-plan-5.md @@ -1,9 +1,5 @@ Round 5 of plan-challenge. The demotion-review round. Different shape from rounds 1-4: instead of picking a new angle and hunting new findings, round 5 deep-reads every claim the prior rounds parked as uncertain (the "demoted claims") and verifies whether each demotion was warranted. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. Say it plainly: "I went back through the claims I parked as uncertain; two were dismissed too quickly and go back in, three stay uncertain, one turned out to be a real problem and I've fixed the plan." - ## Guard: nothing to challenge If no plan was presented in this conversation, decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." diff --git a/.claude/commands/security-scan.md b/.claude/commands/security-scan.md index 5c32b73..22bdd28 100644 --- a/.claude/commands/security-scan.md +++ b/.claude/commands/security-scan.md @@ -1,9 +1,5 @@ Single-round adversarial security scan. Scan a focused surface for findings anchored to named threat classes, argue against each candidate before believing it, and require an articulated reproduction path before anything gets reported. Run it after editing security-sensitive code (auth, payments, user data, anything that parses untrusted input) and BEFORE the implementation-challenge chain. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to scan The scan covers the files modified by the agent in this conversation. If none, decline with: "No scan scope detected; make code edits in this conversation first, or name file paths explicitly after the command." Don't invent imaginary changes to scan. diff --git a/.claude/commands/start-up.md b/.claude/commands/start-up.md index d134766..316adae 100644 --- a/.claude/commands/start-up.md +++ b/.claude/commands/start-up.md @@ -1,9 +1,5 @@ Run the session start-up. This is the chain's opener and the reading half of the pair that makes sessions continuous: handoff writes the note, start-up reads it. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Step 1: Preflight Check the ground you're standing on and act on what you find: diff --git a/.claude/commands/test-it.md b/.claude/commands/test-it.md index 0b9944d..854a3c6 100644 --- a/.claude/commands/test-it.md +++ b/.claude/commands/test-it.md @@ -1,9 +1,5 @@ When the human types `/test-it` (or "test it", "cover it", "write tests"), the implementation from build-it is in place and this stage proves it works. It sits between build-it and security-scan in the chain. It triages the change, writes new behavioral tests for what was just built, drives them to green, and stops the chain if it can't. No questions to the human except the halt case at the bottom (AGENTS.md § The flow guarantee). -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Build the check, don't punt verification If you're about to write "worth a manual check," stop and build the check instead: a test, a script, a measurement, an automated walk-through. Verification is the agent's job. Handing the human a manual test is a failure of this stage, not diligence. A tool you need but can't reach is an access request, stated plainly; it is never a manual-test handoff. @@ -45,7 +41,9 @@ Three cases when a test is red: Two fix-and-rerun tries is the cap before halting (Step 5). -When the suite reaches green, run it once more before trusting it: a pass that doesn't reproduce is a red (flaky tests are how broken work slips through), and it routes through the same three cases above. +When the behavioral tests reach green, discover the project's canonical complete check in this order: explicit project rules, the verification commands the CI workflow actually runs, then package scripts if neither stronger source names it. Several CI verification commands remain an ordered set; don't guess a single replacement. If none defines a complete check, run the broadest existing suite and name the missing project-level check in the report instead of inventing a command. Run that complete check or set once. A red result routes through the same three cases above, and a substantive change after the green result earns another run. + +Targeted checks remain unlimited while fixing. A new test involving time, randomness, concurrency, or an outside process gets one targeted canary rerun even when it first passes; deterministic tests do not need an unconditional second run. A fourth case sits outside the code entirely: when an outside service fails mid-stage (an API down, a platform change), that is the world breaking, not the work. Log it, name it in the report's Deferred field, and never "fix" working code in response. @@ -75,4 +73,4 @@ K counts real defects the tests revealed, fixed by changing the code. Report the ## Rationale (recorded so future edits don't drift it) -The stage exists because review rounds judging untested code argue about reasoning, while rounds judging tested code argue about evidence. The never-weaken rule and the plan-derived expectations both counter the same failure: an agent grading its own homework drifts toward whatever the code already does. The grader step brings in the one reviewer that can't recognize its own work, and the cited NeurIPS 2024 result is why that seat matters. The run-it-twice rule and the outside-service case adapt the acceptance-testing discipline in Mike Fishbein's infinite-headcount factory skills (flaky greens hide broken builds; a failure of the world is never a reason to edit working code). Recorded so the lineage survives edits. +The stage exists because review rounds judging untested code argue about reasoning, while rounds judging tested code argue about evidence. The never-weaken rule and the plan-derived expectations both counter the same failure: an agent grading its own homework drifts toward whatever the code already does. The grader step brings in the one reviewer that can't recognize its own work, and the cited NeurIPS 2024 result is why that seat matters. The narrow canary rule and the outside-service case adapt the acceptance-testing discipline in Mike Fishbein's infinite-headcount factory skills: risky tests must reproduce, while a failure of the world is never a reason to edit working code. Recorded so the lineage survives edits. diff --git a/.claude/commands/wrap-up.md b/.claude/commands/wrap-up.md index e6e482c..70d1bc8 100644 --- a/.claude/commands/wrap-up.md +++ b/.claude/commands/wrap-up.md @@ -10,7 +10,7 @@ Nothing else starts until every background task this conversation spawned is res ## Step 1: Cleanup pass -Re-read every file the session modified, in place, not as a diff: dead code out, unused imports out, naming consistent, comments carrying intent rather than narration. Run the project's check or lint suite and fix what it finds. +Re-read every file the session modified, in place, not as a diff: dead code out, unused imports out, naming consistent, comments carrying intent rather than narration. Then discover the canonical complete check in the same order as test-it: explicit project rules, the verification commands CI actually runs, then package scripts. Preserve several CI commands as an ordered set. If no source names a complete check, run the broadest existing suite and report the missing project-level check instead of inventing one. Run the discovered check or set once after the cumulative review. A red result or a substantive change after that run earns another complete check; otherwise don't repeat it. ## Step 2: Review the cumulative session work diff --git a/.cursor/commands/build-it.md b/.cursor/commands/build-it.md index fd143f2..5b965e9 100644 --- a/.cursor/commands/build-it.md +++ b/.cursor/commands/build-it.md @@ -1,9 +1,5 @@ When the human types `/build-it` (or "build it", "build this", "implement it", "go build"), the plan is settled. Start implementing per the challenged plan. From here to handoff the chain runs without questions to the human, per AGENTS.md § The flow guarantee; the recovery paths below are the only sanctioned stops. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## The unattended-chain rule (why the stops below look the way they do) The human may queue the entire remaining chain (challenge rounds, test-it, security-scan, wrap-up, handoff) and walk away. Each queued command fires the moment the previous turn ends. Three implications: @@ -30,6 +26,10 @@ No plan survives contact with the code untouched, and under an unattended chain The log lives in the plan file's `## Deviations` section, one bullet per departure with four parts: what the plan said, what the code forced, what was chosen, and a one-line lesson. Cite code locations. Implementation-challenge round 1 reads this section first and treats every entry as a prime target, because deviations are exactly where the work left the reviewed plan. +## During implementation: keep tool use coherent and checks targeted + +Run independent reads or inspections together, but keep steps ordered when one's output feeds the next. Use bounded reads when only one section is needed. Put large command output in a file and return the useful summary without hiding evidence. Run targeted checks for the files and behavior being changed; test-it owns the first complete project check. Never combine unrelated edits or skip output just to improve a turn count. + ## During implementation: commit as units land, no pull request Work commits and pushes to the feature branch as coherent units complete. Do NOT open a pull request during build-it or the review rounds that follow; that's wrap-up's job, once, at the end. No-git mode: edit in place, and keep a running list of what changed in the plan file. diff --git a/.cursor/commands/challenge-implementation-1.md b/.cursor/commands/challenge-implementation-1.md index bcabcff..2e5ab29 100644 --- a/.cursor/commands/challenge-implementation-1.md +++ b/.cursor/commands/challenge-implementation-1.md @@ -1,9 +1,5 @@ Round 1 of implementation-challenge. Trace through the changes you just made and look for bugs or edge cases that weren't obvious. Aim for a thorough audit, not a quick scan. Fix what you find; don't just surface it. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation (no code edits, no file writes by the agent), decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." Don't invent imaginary changes to challenge. diff --git a/.cursor/commands/challenge-implementation-2.md b/.cursor/commands/challenge-implementation-2.md index 228a95c..50bfdb1 100644 --- a/.cursor/commands/challenge-implementation-2.md +++ b/.cursor/commands/challenge-implementation-2.md @@ -1,9 +1,5 @@ Round 2 of implementation-challenge. Given how much this work matters, trace through the changes again from a different angle. Look for bugs or edge cases that weren't obvious in round 1. Fix what you find. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation, decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." @@ -23,7 +19,7 @@ If a different angle is clearly more relevant, override the default and say why. 1. **Read back your prior turns.** List every angle used so far (original implementation + round 1). Format: `Angles used: [round 0: X], [round 1: Y]`. 2. **Confirm the round-2 angle**: either "using the default for this round: check whether the earlier rounds' fixes broke anything new" OR "overriding the default to [angle]: [reason]". The angle must be different from rounds 0 and 1. -Then re-read each modified file in full surrounding context, paying particular attention to changes made during round 1. +Read every file changed during round 1 in full surrounding context, plus whatever files the round-2 angle newly requires. A file untouched since the last inspection does not need reprinting unless the conversation was compacted, the file changed since inspection, or this angle needs surrounding code no earlier round inspected. Any of those conditions forces a full read. ## What to look for diff --git a/.cursor/commands/challenge-implementation-3.md b/.cursor/commands/challenge-implementation-3.md index b460641..313a00e 100644 --- a/.cursor/commands/challenge-implementation-3.md +++ b/.cursor/commands/challenge-implementation-3.md @@ -1,9 +1,5 @@ Round 3 of implementation-challenge. Are you 200% sure you've covered this from every angle? Trace through the changes again from the round-3 angle. Fix what you find. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation, decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." @@ -23,7 +19,7 @@ If a different angle is clearly more relevant, override the default and say why. 1. **Read back your prior turns.** List every angle used so far (original implementation + rounds 1-2). Format: `Angles used: [round 0: X], [round 1: Y], [round 2: Z]`. 2. **Confirm the round-3 angle**: either "using the default for this round: how this code talks to its neighbours" OR "overriding the default to [angle]: [reason]". The angle must be different from rounds 0-2. -Then re-read each modified file in full surrounding context and trace every seam. +Read every file changed during round 2 in full surrounding context, plus callers, consumers, or other files the round-3 angle newly requires. A file untouched since the last inspection does not need reprinting unless the conversation was compacted, the file changed since inspection, or this angle needs surrounding code no earlier round inspected. Any of those conditions forces a full read. Then trace every seam. ## What to look for diff --git a/.cursor/commands/challenge-implementation-4.md b/.cursor/commands/challenge-implementation-4.md index 0d05e35..903b64e 100644 --- a/.cursor/commands/challenge-implementation-4.md +++ b/.cursor/commands/challenge-implementation-4.md @@ -1,9 +1,5 @@ Round 4 of implementation-challenge. Trace through the changes again from the round-4 angle. Look for bugs the prior three rounds missed. Fix what you find. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation, decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." @@ -27,6 +23,8 @@ If the work touches no external services AND has no peer precedent to compare ag 1. **Read back your prior turns.** List every angle used so far (original implementation + rounds 1-3). Format: `Angles used: [round 0: X], [round 1: Y], [round 2: Z], [round 3: W]`. 2. **Confirm the round-4 angle**: either "using the default for this round: external concerns" OR "overriding the default to [angle]: [reason]". The angle must be different from rounds 0-3. +Read every file changed during round 3 in full surrounding context, plus whatever files or sources the round-4 angle newly requires. A file untouched since the last inspection does not need reprinting unless the conversation was compacted, the file changed since inspection, or this angle needs surrounding code no earlier round inspected. Any of those conditions forces a full read. + ## Useful angles to rotate through The canonical list lives in AGENTS.md § The challenge rounds: implementation / where data flows and what happens when something fails / how this code talks to its neighbours / how this code talks to outside services / config-file syntax and quoting pitfalls / what unusual or hostile inputs would break this? / what happens when shell scripts fail mid-pipeline? / will this code be discoverable when it feels unfamiliar? / is the sequence of steps right? / where else does the same root cause apply? / check whether the earlier rounds' fixes broke anything new / does this convention reach every place the mechanism reaches? diff --git a/.cursor/commands/challenge-implementation-5.md b/.cursor/commands/challenge-implementation-5.md index bcbdef4..d3bc2c0 100644 --- a/.cursor/commands/challenge-implementation-5.md +++ b/.cursor/commands/challenge-implementation-5.md @@ -1,9 +1,5 @@ Round 5 of implementation-challenge. Final-pass deep audit. Are you 200% sure you've covered this from every angle and fixed every bug? If not, keep tracing. Fix what you find. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no implementation was made in this conversation, decline with: "No implementation to challenge in this conversation. Make changes first, then run the challenge chain starting with round 1." @@ -45,6 +41,8 @@ Look for: 1. **Read back your prior turns.** List every angle used so far (original implementation + rounds 1-4). Format: `Angles used: [round 0: V], [round 1: W], [round 2: X], [round 3: Y], [round 4: Z]`. 2. **Confirm the round-5 angle**: either "using the default for this round: over-engineering check and simpler solutions" OR "overriding the default to [angle]: [reason]". +Before hunting for cuts, re-read every file modified since implementation began in full surrounding context. Do not trust the diff; this round judges the accumulated implementation, including fixes from rounds 1 through 4. + ## Auto-fix bugs found (when K > 0) Same policy as round 1: fix now via a new commit on the feature branch (no pull request until wrap-up; in no-git mode fix in place and log it). Out-of-scope findings become observations, human-territory bugs defer with `[NEEDS HUMAN-R5: ]`. K counts high-confidence defects only; a cut that changes behavior needs the same confidence bar as a fix. diff --git a/.cursor/commands/challenge-plan-1.md b/.cursor/commands/challenge-plan-1.md index 84538a6..d86a611 100644 --- a/.cursor/commands/challenge-plan-1.md +++ b/.cursor/commands/challenge-plan-1.md @@ -1,9 +1,5 @@ Round 1 of plan-challenge. Are you sure you've done all the research and followed every thread needed to reach this conclusion? If not, keep going. Look for opportunities you might be missing. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no plan was presented in this conversation (no construct-the-plan run, no inline plan content, no plan file referenced), decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." Don't invent an imaginary plan to challenge. diff --git a/.cursor/commands/challenge-plan-2.md b/.cursor/commands/challenge-plan-2.md index 95ef0f7..52a8809 100644 --- a/.cursor/commands/challenge-plan-2.md +++ b/.cursor/commands/challenge-plan-2.md @@ -1,9 +1,5 @@ Round 2 of plan-challenge. Given how much this matters, trace through the plan again from a different angle. Be sure you've done all the research and followed every thread. Look for opportunities you might be missing. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no plan was presented in this conversation (no construct-the-plan run, no inline plan content, no plan file referenced), decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." Don't invent an imaginary plan to challenge. diff --git a/.cursor/commands/challenge-plan-3.md b/.cursor/commands/challenge-plan-3.md index 8606e84..9f2e61d 100644 --- a/.cursor/commands/challenge-plan-3.md +++ b/.cursor/commands/challenge-plan-3.md @@ -1,9 +1,5 @@ Round 3 of plan-challenge. Are you 200% sure you've covered this from every angle? If not, keep going. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no plan was presented in this conversation (no construct-the-plan run, no inline plan content, no plan file referenced), decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." Don't invent an imaginary plan to challenge. diff --git a/.cursor/commands/challenge-plan-4.md b/.cursor/commands/challenge-plan-4.md index 032ad54..137ebb8 100644 --- a/.cursor/commands/challenge-plan-4.md +++ b/.cursor/commands/challenge-plan-4.md @@ -1,9 +1,5 @@ Round 4 of plan-challenge. Are you 200% sure you've covered this from every angle? If not, keep going. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to challenge If no plan was presented in this conversation (no construct-the-plan run, no inline plan content, no plan file referenced), decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." Don't invent an imaginary plan to challenge. diff --git a/.cursor/commands/challenge-plan-5.md b/.cursor/commands/challenge-plan-5.md index b0c28e3..77245f4 100644 --- a/.cursor/commands/challenge-plan-5.md +++ b/.cursor/commands/challenge-plan-5.md @@ -1,9 +1,5 @@ Round 5 of plan-challenge. The demotion-review round. Different shape from rounds 1-4: instead of picking a new angle and hunting new findings, round 5 deep-reads every claim the prior rounds parked as uncertain (the "demoted claims") and verifies whether each demotion was warranted. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. Say it plainly: "I went back through the claims I parked as uncertain; two were dismissed too quickly and go back in, three stay uncertain, one turned out to be a real problem and I've fixed the plan." - ## Guard: nothing to challenge If no plan was presented in this conversation, decline with: "No plan to challenge in this conversation. Draft one first with construct-the-plan, write the plan inline above this command, or name a plan file." diff --git a/.cursor/commands/security-scan.md b/.cursor/commands/security-scan.md index 5c32b73..22bdd28 100644 --- a/.cursor/commands/security-scan.md +++ b/.cursor/commands/security-scan.md @@ -1,9 +1,5 @@ Single-round adversarial security scan. Scan a focused surface for findings anchored to named threat classes, argue against each candidate before believing it, and require an articulated reproduction path before anything gets reported. Run it after editing security-sensitive code (auth, payments, user data, anything that parses untrusted input) and BEFORE the implementation-challenge chain. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Guard: nothing to scan The scan covers the files modified by the agent in this conversation. If none, decline with: "No scan scope detected; make code edits in this conversation first, or name file paths explicitly after the command." Don't invent imaginary changes to scan. diff --git a/.cursor/commands/start-up.md b/.cursor/commands/start-up.md index d134766..316adae 100644 --- a/.cursor/commands/start-up.md +++ b/.cursor/commands/start-up.md @@ -1,9 +1,5 @@ Run the session start-up. This is the chain's opener and the reading half of the pair that makes sessions continuous: handoff writes the note, start-up reads it. -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Step 1: Preflight Check the ground you're standing on and act on what you find: diff --git a/.cursor/commands/test-it.md b/.cursor/commands/test-it.md index 0b9944d..854a3c6 100644 --- a/.cursor/commands/test-it.md +++ b/.cursor/commands/test-it.md @@ -1,9 +1,5 @@ When the human types `/test-it` (or "test it", "cover it", "write tests"), the implementation from build-it is in place and this stage proves it works. It sits between build-it and security-scan in the chain. It triages the change, writes new behavioral tests for what was just built, drives them to green, and stops the chain if it can't. No questions to the human except the halt case at the bottom (AGENTS.md § The flow guarantee). -## Voice - -Output prints to the human's chat: follow AGENTS.md § Voice. Pipe substantive drafts through `node scripts/voice-gate.js` and rewrite what it flags. - ## Build the check, don't punt verification If you're about to write "worth a manual check," stop and build the check instead: a test, a script, a measurement, an automated walk-through. Verification is the agent's job. Handing the human a manual test is a failure of this stage, not diligence. A tool you need but can't reach is an access request, stated plainly; it is never a manual-test handoff. @@ -45,7 +41,9 @@ Three cases when a test is red: Two fix-and-rerun tries is the cap before halting (Step 5). -When the suite reaches green, run it once more before trusting it: a pass that doesn't reproduce is a red (flaky tests are how broken work slips through), and it routes through the same three cases above. +When the behavioral tests reach green, discover the project's canonical complete check in this order: explicit project rules, the verification commands the CI workflow actually runs, then package scripts if neither stronger source names it. Several CI verification commands remain an ordered set; don't guess a single replacement. If none defines a complete check, run the broadest existing suite and name the missing project-level check in the report instead of inventing a command. Run that complete check or set once. A red result routes through the same three cases above, and a substantive change after the green result earns another run. + +Targeted checks remain unlimited while fixing. A new test involving time, randomness, concurrency, or an outside process gets one targeted canary rerun even when it first passes; deterministic tests do not need an unconditional second run. A fourth case sits outside the code entirely: when an outside service fails mid-stage (an API down, a platform change), that is the world breaking, not the work. Log it, name it in the report's Deferred field, and never "fix" working code in response. @@ -75,4 +73,4 @@ K counts real defects the tests revealed, fixed by changing the code. Report the ## Rationale (recorded so future edits don't drift it) -The stage exists because review rounds judging untested code argue about reasoning, while rounds judging tested code argue about evidence. The never-weaken rule and the plan-derived expectations both counter the same failure: an agent grading its own homework drifts toward whatever the code already does. The grader step brings in the one reviewer that can't recognize its own work, and the cited NeurIPS 2024 result is why that seat matters. The run-it-twice rule and the outside-service case adapt the acceptance-testing discipline in Mike Fishbein's infinite-headcount factory skills (flaky greens hide broken builds; a failure of the world is never a reason to edit working code). Recorded so the lineage survives edits. +The stage exists because review rounds judging untested code argue about reasoning, while rounds judging tested code argue about evidence. The never-weaken rule and the plan-derived expectations both counter the same failure: an agent grading its own homework drifts toward whatever the code already does. The grader step brings in the one reviewer that can't recognize its own work, and the cited NeurIPS 2024 result is why that seat matters. The narrow canary rule and the outside-service case adapt the acceptance-testing discipline in Mike Fishbein's infinite-headcount factory skills: risky tests must reproduce, while a failure of the world is never a reason to edit working code. Recorded so the lineage survives edits. diff --git a/.cursor/commands/wrap-up.md b/.cursor/commands/wrap-up.md index e6e482c..70d1bc8 100644 --- a/.cursor/commands/wrap-up.md +++ b/.cursor/commands/wrap-up.md @@ -10,7 +10,7 @@ Nothing else starts until every background task this conversation spawned is res ## Step 1: Cleanup pass -Re-read every file the session modified, in place, not as a diff: dead code out, unused imports out, naming consistent, comments carrying intent rather than narration. Run the project's check or lint suite and fix what it finds. +Re-read every file the session modified, in place, not as a diff: dead code out, unused imports out, naming consistent, comments carrying intent rather than narration. Then discover the canonical complete check in the same order as test-it: explicit project rules, the verification commands CI actually runs, then package scripts. Preserve several CI commands as an ordered set. If no source names a complete check, run the broadest existing suite and report the missing project-level check instead of inventing one. Run the discovered check or set once after the cumulative review. A red result or a substantive change after that run earns another complete check; otherwise don't repeat it. ## Step 2: Review the cumulative session work diff --git a/AGENTS.md b/AGENTS.md index af8d3e4..cb9f216 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,4 +1,4 @@ - + # Foundry: the working agreement @@ -55,4 +55,4 @@ No session closes while anything it spawned is still running or unread: a subage ## Voice -Everything written for a human reads like a person wrote it: plain words, short sentences, no jargon stacks, no em dashes. Before sending a substantive response, pipe the draft through `node scripts/voice-gate.js` and rewrite what it flags. The same list also has teeth: `npm run check` fails when a listed phrase lands in committed prose (docs, command files, code comments), so the voice bar holds without anyone remembering to hold it. The gate only knows the phrases in `scripts/phrase-list.json`; wrap-up's jargon step grows that list every session, so it keeps pace with the jargon you actually produce. For vocabulary the list doesn't know yet, the test is: would someone outside the codebase follow this sentence? If not, rewrite it. +Everything written for a human reads like a person wrote it: plain words, short sentences, no jargon stacks, no em dashes. Final plan prose, frame-it, quiz, wrap-up, and handoff run through `node scripts/voice-gate.js`. Other responses use the live gate when the agent itself suspects the draft is dense or awkward; short structured stage reports rely on their fixed shape. The same list also has teeth: `npm run check` fails when a listed phrase lands in committed prose (docs, command files, code comments), so the voice bar holds without anyone remembering to hold it. The gate only knows the phrases in `scripts/phrase-list.json`; wrap-up's jargon step grows that list every session, so it keeps pace with the jargon you actually produce. For vocabulary the list doesn't know yet, the test is: would someone outside the codebase follow this sentence? If not, rewrite it. diff --git a/docs/plans/chain-economics.md b/docs/plans/chain-economics.md index 03f2b11..005adbd 100644 --- a/docs/plans/chain-economics.md +++ b/docs/plans/chain-economics.md @@ -179,3 +179,5 @@ The checker constants are the one executable home for the exact ceilings. `AGENT | R8 | Replacing test-it's unconditional rerun in the procedure is enough to retire the old rule. | Demoted. Test-it's rationale explicitly preserves the run-it-twice rule and would contradict the new validation ladder. | Require the rationale to carry the narrower flake-canary reason and remove the retired rule. | ## Deviations + +- The plan said implementation round 5 already required a full surrounding-context read. The command had only a narrow reminder to reread guards. Added the full accumulated-file read to `.cursor/commands/challenge-implementation-5.md` so the acceptance bar wins over the mistaken baseline. Lesson: verify an asserted existing safeguard before planning to leave it unchanged. diff --git a/tests/chain-economics-contract.test.js b/tests/chain-economics-contract.test.js new file mode 100644 index 0000000..293d4d2 --- /dev/null +++ b/tests/chain-economics-contract.test.js @@ -0,0 +1,107 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { readdirSync, readFileSync } from 'node:fs'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const repoRoot = join(dirname(fileURLToPath(import.meta.url)), '..'); +const commandsDir = join(repoRoot, '.cursor', 'commands'); + +function command(name) { + return readFileSync(join(commandsDir, `${name}.md`), 'utf8'); +} + +function commandsWithLiveGate(overrides = {}) { + return readdirSync(commandsDir) + .filter((file) => file.endsWith('.md')) + .filter((file) => { + const name = file.replace(/\.md$/, ''); + const body = overrides[name] ?? readFileSync(join(commandsDir, file), 'utf8'); + return body.includes('voice-gate.js'); + }) + .map((file) => file.replace(/\.md$/, '')) + .sort(); +} + +function hasFreshnessPolicy(text) { + const lower = text.toLowerCase(); + const priorRoundChanges = /changed (?:during|in) (?:the )?(?:prior|preceding|round \d)/.test(lower); + const newAngleNeeds = /(?:angle.*(?:newly )?requires|new angle.*requires)/.test(lower); + const compacted = /compact|summari[sz]/.test(lower); + const changedAfterRead = /changed (?:since|after) inspection/.test(lower); + const unseenContext = /no earlier round inspected|unseen (?:surrounding )?context|context .*not.*(?:seen|inspected)/.test(lower); + const fullRead = /full surrounding context|fully read|re-?read the whole|full read/.test(lower); + return priorRoundChanges && newAngleNeeds && compacted && changedAfterRead && unseenContext && fullRead; +} + +function hasValidationDiscovery(text) { + const lower = text.toLowerCase(); + const rules = lower.indexOf('explicit project rules'); + const ci = lower.indexOf('ci', rules + 1); + const scripts = lower.indexOf('package scripts', ci + 1); + return ( + rules >= 0 && + ci > rules && + scripts > ci && + /ordered (?:check )?set|ordered set/.test(lower) && + /broadest existing suite/.test(lower) && + /missing project-level check/.test(lower) + ); +} + +function hasUniversalGreenRerun(text) { + return /(?:suite|complete check).{0,40}(?:reaches|is) green.{0,100}(?:run|execute).{0,30}(?:again|once more)/is.test(text); +} + +test('only the five prose-heavy stages require an explicit live voice gate', () => { + const expected = ['construct-the-plan', 'frame-it', 'handoff', 'quiz', 'wrap-up']; + assert.deepEqual(commandsWithLiveGate(), expected); + + const agreement = readFileSync(join(repoRoot, 'AGENTS.md'), 'utf8'); + assert.match(agreement, /suspects? the draft is dense or awkward/i); + + const universalMutation = command('build-it') + '\nRun `node scripts/voice-gate.js` before every reply.\n'; + assert.notDeepEqual(commandsWithLiveGate({ 'build-it': universalMutation }), expected); +}); + +test('middle review rounds use freshness escapes while rounds 1 and 5 require full reads', () => { + for (const round of [2, 3, 4]) { + assert.equal(hasFreshnessPolicy(command(`challenge-implementation-${round}`)), true, `round ${round}`); + } + for (const round of [1, 5]) { + assert.match(command(`challenge-implementation-${round}`), /re-read every file|re-read each modified file/i); + assert.match(command(`challenge-implementation-${round}`), /full surrounding context/i); + } + + const reworded = [ + 'Fully read files changed in the preceding round and anything the new angle requires.', + 'Re-read the whole file if the chat was summarized, if it changed after inspection,', + 'or if the angle needs unseen context.', + ].join(' '); + assert.equal(hasFreshnessPolicy(reworded), true); + assert.equal(hasFreshnessPolicy(reworded.replace('if the chat was summarized, ', '')), false); +}); + +test('test-it and wrap-up share the validation ladder without a universal green rerun', () => { + const testIt = command('test-it'); + const wrapUp = command('wrap-up'); + + assert.equal(hasValidationDiscovery(testIt), true); + assert.equal(hasValidationDiscovery(wrapUp), true); + assert.match(testIt, /targeted checks remain unlimited/i); + assert.match(testIt, /time, randomness, concurrency, or an outside process/i); + assert.match(testIt, /run that complete check or set once/i); + assert.match(wrapUp, /once after the cumulative review/i); + assert.match(testIt, /red result.*substantive change/is); + assert.match(wrapUp, /red result.*substantive change/is); + assert.equal(hasUniversalGreenRerun(testIt), false); + assert.equal(hasUniversalGreenRerun(wrapUp), false); + + const reworded = [ + 'Choose the full verification path from explicit project rules, then CI as an ordered set,', + 'then package scripts. If none exists, use the broadest existing suite and report the', + 'missing project-level check.', + ].join(' '); + assert.equal(hasValidationDiscovery(reworded), true); + assert.equal(hasUniversalGreenRerun('When the complete check is green, run it once more.'), true); +}); From 7df2c7b4b81b2166ceaa693ef9913ca1a3721093 Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:26:52 -0400 Subject: [PATCH 06/13] Keep the context checker executable through every supported spelling --- scripts/check-context-budgets.js | 15 +++++++++++++-- tests/check-context-budgets.test.js | 23 ++++++++++++++++++++--- 2 files changed, 33 insertions(+), 5 deletions(-) diff --git a/scripts/check-context-budgets.js b/scripts/check-context-budgets.js index 296dfce..1b11572 100644 --- a/scripts/check-context-budgets.js +++ b/scripts/check-context-budgets.js @@ -4,7 +4,7 @@ // budgets the project promises. Generated command shapes are derived from the // source commands, so counting them again would charge the same prose twice. -import { readFileSync, readdirSync } from 'node:fs'; +import { readFileSync, readdirSync, realpathSync } from 'node:fs'; import { dirname, join, relative, resolve, sep } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; @@ -109,5 +109,16 @@ function main() { } } -const invokedDirectly = process.argv[1] && import.meta.url === pathToFileURL(resolve(process.argv[1])).href; +const invokedDirectly = process.argv[1] && import.meta.url === pathToFileURL(resolveArgvPath(process.argv[1])).href; if (invokedDirectly) main(); + +function resolveArgvPath(argvPath) { + for (const candidate of [argvPath, `${argvPath}.js`]) { + try { + return realpathSync(candidate); + } catch { + // Try the next supported spelling. + } + } + return resolve(argvPath); +} diff --git a/tests/check-context-budgets.test.js b/tests/check-context-budgets.test.js index 871dec8..ee7218f 100644 --- a/tests/check-context-budgets.test.js +++ b/tests/check-context-budgets.test.js @@ -14,6 +14,7 @@ import { const repoRoot = join(dirname(fileURLToPath(import.meta.url)), '..'); const script = join(repoRoot, 'scripts', 'check-context-budgets.js'); +const extensionlessScript = script.replace(/\.js$/, ''); function makeFixture({ agents = 'rules\n', commands = { 'alpha.md': 'command\n' } } = {}) { const root = mkdtempSync(join(tmpdir(), 'context-budgets-')); @@ -122,13 +123,16 @@ test('refuses an unreadable command file', () => { }); test('report mode stays informational while check mode enforces the ceiling', () => { - const root = makeFixture({ agents: 'a'.repeat(CONTEXT_LIMITS.agentsBytes + 1) }); + const root = makeFixture({ + agents: 'a'.repeat(CONTEXT_LIMITS.agentsBytes + 1), + commands: { 'small.md': 'x', 'largest.md': 'xxx' }, + }); try { const report = run(root); assert.equal(report.code, 0); assert.ok(report.out.includes('AGENTS.md')); - assert.ok(report.out.includes('.cursor/commands/alpha.md')); - assert.ok(report.out.indexOf('source commands') < report.out.indexOf('.cursor/commands/alpha.md')); + assert.ok(report.out.indexOf('source commands') < report.out.indexOf('.cursor/commands/largest.md')); + assert.ok(report.out.indexOf('.cursor/commands/largest.md') < report.out.indexOf('.cursor/commands/small.md')); const checked = run(root, ['--check']); assert.equal(checked.code, 1); @@ -138,6 +142,19 @@ test('report mode stays informational while check mode enforces the ceiling', () } }); +test('the extensionless CLI spelling still executes the checker', () => { + const root = makeFixture(); + try { + const out = execFileSync('node', [extensionlessScript, '--check'], { + env: { ...process.env, CONTEXT_BUDGET_ROOT: root }, + encoding: 'utf8', + }); + assert.ok(out.includes('[context-budgets] AGENTS.md')); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + test('importing the checker never measures the checkout or exits', () => { const missingRoot = join(tmpdir(), 'context-budget-root-that-does-not-exist'); const out = execFileSync( From 408bea0611b061a2e88b60f3fde4638c7237f107 Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:28:18 -0400 Subject: [PATCH 07/13] Guard selective voice routing at its central policy --- tests/chain-economics-contract.test.js | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/tests/chain-economics-contract.test.js b/tests/chain-economics-contract.test.js index 293d4d2..695282d 100644 --- a/tests/chain-economics-contract.test.js +++ b/tests/chain-economics-contract.test.js @@ -23,6 +23,16 @@ function commandsWithLiveGate(overrides = {}) { .sort(); } +function hasSelectiveCentralVoicePolicy(text) { + const lower = text.toLowerCase(); + const namedStages = /final plan|construct-the-plan/.test(lower) && + ['frame-it', 'quiz', 'wrap-up', 'handoff'].every((stage) => lower.includes(stage)); + const judgmentEscape = /suspects?.{0,50}dense or awkward/.test(lower); + const fixedReportEscape = /short structured stage reports.{0,60}(?:fixed shape|their fixed shape)/.test(lower); + const universal = /every response.{0,80}(?:voice|gate)|all responses.{0,80}(?:voice|gate)|before sending a substantive response.{0,80}(?:voice|gate)/.test(lower); + return namedStages && judgmentEscape && fixedReportEscape && !universal; +} + function hasFreshnessPolicy(text) { const lower = text.toLowerCase(); const priorRoundChanges = /changed (?:during|in) (?:the )?(?:prior|preceding|round \d)/.test(lower); @@ -58,10 +68,18 @@ test('only the five prose-heavy stages require an explicit live voice gate', () assert.deepEqual(commandsWithLiveGate(), expected); const agreement = readFileSync(join(repoRoot, 'AGENTS.md'), 'utf8'); - assert.match(agreement, /suspects? the draft is dense or awkward/i); + assert.equal(hasSelectiveCentralVoicePolicy(agreement), true); const universalMutation = command('build-it') + '\nRun `node scripts/voice-gate.js` before every reply.\n'; assert.notDeepEqual(commandsWithLiveGate({ 'build-it': universalMutation }), expected); + + const reworded = [ + 'Use the live checker on final plan writing, frame-it, quiz, wrap-up, and handoff.', + 'Use it elsewhere when the agent suspects the draft is dense or awkward.', + 'Short structured stage reports rely on their fixed shape.', + ].join(' '); + assert.equal(hasSelectiveCentralVoicePolicy(reworded), true); + assert.equal(hasSelectiveCentralVoicePolicy(`${reworded} Every response must run the voice gate.`), false); }); test('middle review rounds use freshness escapes while rounds 1 and 5 require full reads', () => { From adc53c56088f114aaf5fcc7ec00616cbc1c45a7b Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:28:48 -0400 Subject: [PATCH 08/13] Reject universal voice rules regardless of modifier --- tests/chain-economics-contract.test.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/chain-economics-contract.test.js b/tests/chain-economics-contract.test.js index 695282d..37a8182 100644 --- a/tests/chain-economics-contract.test.js +++ b/tests/chain-economics-contract.test.js @@ -29,7 +29,7 @@ function hasSelectiveCentralVoicePolicy(text) { ['frame-it', 'quiz', 'wrap-up', 'handoff'].every((stage) => lower.includes(stage)); const judgmentEscape = /suspects?.{0,50}dense or awkward/.test(lower); const fixedReportEscape = /short structured stage reports.{0,60}(?:fixed shape|their fixed shape)/.test(lower); - const universal = /every response.{0,80}(?:voice|gate)|all responses.{0,80}(?:voice|gate)|before sending a substantive response.{0,80}(?:voice|gate)/.test(lower); + const universal = /(?:every|all)(?:\s+\w+){0,2}\s+responses?.{0,80}(?:voice|gate)|before sending a substantive response.{0,80}(?:voice|gate)/.test(lower); return namedStages && judgmentEscape && fixedReportEscape && !universal; } @@ -79,7 +79,7 @@ test('only the five prose-heavy stages require an explicit live voice gate', () 'Short structured stage reports rely on their fixed shape.', ].join(' '); assert.equal(hasSelectiveCentralVoicePolicy(reworded), true); - assert.equal(hasSelectiveCentralVoicePolicy(`${reworded} Every response must run the voice gate.`), false); + assert.equal(hasSelectiveCentralVoicePolicy(`${reworded} Every substantive response must run the voice gate.`), false); }); test('middle review rounds use freshness escapes while rounds 1 and 5 require full reads', () => { From dfc8baedeee484aa305f20ee363ddd8fb85221ab Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:29:18 -0400 Subject: [PATCH 09/13] Make validation-order tests read the command they guard --- tests/chain-economics-contract.test.js | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/tests/chain-economics-contract.test.js b/tests/chain-economics-contract.test.js index 37a8182..0a60851 100644 --- a/tests/chain-economics-contract.test.js +++ b/tests/chain-economics-contract.test.js @@ -46,13 +46,15 @@ function hasFreshnessPolicy(text) { function hasValidationDiscovery(text) { const lower = text.toLowerCase(); - const rules = lower.indexOf('explicit project rules'); - const ci = lower.indexOf('ci', rules + 1); - const scripts = lower.indexOf('package scripts', ci + 1); + const rules = lower.match(/explicit project rules/); + const afterRules = rules ? lower.slice(rules.index + rules[0].length) : ''; + const ci = afterRules.match(/\bci\b/); + const afterCi = ci ? afterRules.slice(ci.index + ci[0].length) : ''; + const scripts = afterCi.match(/package scripts/); return ( - rules >= 0 && - ci > rules && - scripts > ci && + rules !== null && + ci !== null && + scripts !== null && /ordered (?:check )?set|ordered set/.test(lower) && /broadest existing suite/.test(lower) && /missing project-level check/.test(lower) @@ -121,5 +123,6 @@ test('test-it and wrap-up share the validation ladder without a universal green 'missing project-level check.', ].join(' '); assert.equal(hasValidationDiscovery(reworded), true); + assert.equal(hasValidationDiscovery(reworded.replace('then CI as an ordered set, ', '')), false); assert.equal(hasUniversalGreenRerun('When the complete check is green, run it once more.'), true); }); From bc5f442f8ee6f21b7e97ca3f7f8ec12ca2a29eef Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:30:50 -0400 Subject: [PATCH 10/13] Keep economics evidence inspectable as vendors move docs --- docs/wiki/engineering/chain-economics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/wiki/engineering/chain-economics.md b/docs/wiki/engineering/chain-economics.md index c885950..e1e361e 100644 --- a/docs/wiki/engineering/chain-economics.md +++ b/docs/wiki/engineering/chain-economics.md @@ -25,7 +25,7 @@ Transcript-derived dollar figures use an approximate rate and reconstructed cont The three-build pilot was heterogeneous and changed three defaults together. It can say quality held. It cannot divide a saving among selective live checks, batching, and freshness-aware rereads, and its billed cost was unavailable. The wrap-up movement is descriptive for the same reason. -Tool economics also move underneath the process. As checked on August 5, 2026, [Cursor's summarization documentation](https://docs.cursor.com/en/agent/chat/summarization) describes context management rather than a savings guarantee; [Cursor pricing](https://docs.cursor.com/account/pricing), the [Codex rate card](https://help.openai.com/en/articles/20001106-codex-rate-card), and [Anthropic pricing](https://docs.anthropic.com/en/docs/about-claude/pricing) expose different units and cache treatment. These are examples of variation, not Foundry policy. Recheck the live provider before doing money math. +Tool economics also move underneath the process. As checked on August 5, 2026, [Cursor's summarization release note](https://cursor.com/changelog/1-6) describes context management rather than a savings guarantee; [Cursor's pricing policy](https://cursor.com/terms/pricing/2026-03-25), the [Codex rate card](https://help.openai.com/en/articles/20001106-codex-rate-card), and [Anthropic pricing](https://platform.claude.com/docs/en/about-claude/pricing) expose different units and cache treatment. These are examples of variation, not Foundry policy. Recheck the live provider before doing money math. ## Defaults this evidence earns From c68dcefc194a63b1d14bd15722abd304690c0b8a Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:31:47 -0400 Subject: [PATCH 11/13] Keep the implementation record aligned with live evidence --- docs/plans/chain-economics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/plans/chain-economics.md b/docs/plans/chain-economics.md index 005adbd..e370916 100644 --- a/docs/plans/chain-economics.md +++ b/docs/plans/chain-economics.md @@ -44,7 +44,7 @@ The selective voice gate could let an awkward short chat response through. The m - Foundry's current context guidance and budgets: `AGENTS.md` and `docs/wiki/engineering/context-engineering.md`. - Foundry's current stage behavior: `.cursor/commands/build-it.md`, `.cursor/commands/test-it.md`, `.cursor/commands/wrap-up.md`, and `.cursor/commands/challenge-implementation-{1..5}.md`. - Current size baseline: `AGENTS.md` 6,343 bytes; nineteen `.cursor/commands/*.md` files 105,823 bytes total. -- Current vendor contracts, used only to keep the prose neutral: [Cursor summarization](https://docs.cursor.com/en/agent/chat/summarization) documents context management, not a savings guarantee; [Cursor pricing](https://docs.cursor.com/account/pricing) and the [Codex rate card](https://help.openai.com/en/articles/20001106-codex-rate-card) expose tool-specific token economics; [Anthropic pricing](https://docs.anthropic.com/en/docs/about-claude/pricing) exposes separate cache-write and cache-hit rates. These are dated examples checked on 2026-08-05, not Foundry policy. +- Current vendor contracts, used only to keep the prose neutral: [Cursor's summarization release note](https://cursor.com/changelog/1-6) documents context management, not a savings guarantee; [Cursor's pricing policy](https://cursor.com/terms/pricing/2026-03-25) and the [Codex rate card](https://help.openai.com/en/articles/20001106-codex-rate-card) expose tool-specific token economics; [Anthropic pricing](https://platform.claude.com/docs/en/about-claude/pricing) exposes separate cache-write and cache-hit rates. These are dated examples checked on 2026-08-05, not Foundry policy. ## File-tree change From 6a9b289ca1e4076b3f9e0637e5736aa07cb3f577 Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:32:38 -0400 Subject: [PATCH 12/13] Count linked commands so context limits cannot be bypassed --- scripts/check-context-budgets.js | 2 +- tests/check-context-budgets.test.js | 16 +++++++++++++++- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/scripts/check-context-budgets.js b/scripts/check-context-budgets.js index 1b11572..a98851d 100644 --- a/scripts/check-context-budgets.js +++ b/scripts/check-context-budgets.js @@ -37,7 +37,7 @@ export function measureContextBudgets(rootDir) { } const commandFiles = entries - .filter((entry) => entry.isFile() && entry.name.endsWith('.md')) + .filter((entry) => (entry.isFile() || entry.isSymbolicLink()) && entry.name.endsWith('.md')) .map((entry) => { const path = join(commandsDir, entry.name); const portablePath = relative(root, path).split(sep).join('/'); diff --git a/tests/check-context-budgets.test.js b/tests/check-context-budgets.test.js index ee7218f..1605809 100644 --- a/tests/check-context-budgets.test.js +++ b/tests/check-context-budgets.test.js @@ -1,7 +1,7 @@ import { test } from 'node:test'; import assert from 'node:assert/strict'; import { execFileSync } from 'node:child_process'; -import { chmodSync, mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'node:fs'; +import { chmodSync, mkdtempSync, mkdirSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; @@ -122,6 +122,20 @@ test('refuses an unreadable command file', () => { } }); +test('counts symlinked markdown commands instead of permitting a budget bypass', () => { + const root = makeFixture({ commands: {} }); + const target = join(root, 'shared-command.md'); + writeFileSync(target, 'linked command\n'); + symlinkSync(target, join(root, '.cursor', 'commands', 'linked.md')); + try { + const measurement = measureContextBudgets(root); + assert.equal(measurement.commandCount, 1); + assert.equal(measurement.commandBytes, 15); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + test('report mode stays informational while check mode enforces the ceiling', () => { const root = makeFixture({ agents: 'a'.repeat(CONTEXT_LIMITS.agentsBytes + 1), From 63650784f1718884ab6a82d72eb70600bfde1fe6 Mon Sep 17 00:00:00 2001 From: Simon Corry Date: Wed, 5 Aug 2026 16:35:30 -0400 Subject: [PATCH 13/13] Leave the economics work with an honest durable record --- docs/plans/chain-economics.md | 2 +- docs/sessions/LOG.md | 6 ++++-- docs/sessions/history/2026-W29.md | 5 +++++ 3 files changed, 10 insertions(+), 3 deletions(-) create mode 100644 docs/sessions/history/2026-W29.md diff --git a/docs/plans/chain-economics.md b/docs/plans/chain-economics.md index e370916..c81b547 100644 --- a/docs/plans/chain-economics.md +++ b/docs/plans/chain-economics.md @@ -1,6 +1,6 @@ --- id: chain-economics -status: IN_PROGRESS +status: SHIPPED created: 2026-08-05 --- diff --git a/docs/sessions/LOG.md b/docs/sessions/LOG.md index 7722fc5..1213047 100644 --- a/docs/sessions/LOG.md +++ b/docs/sessions/LOG.md @@ -2,6 +2,8 @@ One entry per session, newest first, in plain English. Wrap-up writes an entry here at every close and start-up reads this file at every open; that pair is what makes sessions continuous even when a handoff note goes missing. Entries older than the current week move to a history folder beside this file, one file per week, so this file never outgrows a sitting. Entry shape: `## YYYY-MM-DD: Title`, one short paragraph, an optional `Friction:` line when something fought back. -## 2026-07-15: The memory system and the reference library +## 2026-08-05: Honest chain economics -Foundry gained its memory and its bookshelf. This log now exists, with weekly rotation keeping it readable (`scripts/rotate-sessions.js`, loss-checked so rotation can never eat an entry). The wiki opened at `docs/wiki/` with an enforced index: a page nobody links from the index fails the check, so the map stays true. And the three general reference libraries moved in as the wiki's first residents: engineering fundamentals (Brooks, Parnas, Naur, and company), design fundamentals (the Norman-to-Rams canon), and motion fundamentals (adapted with credit from Emil Kowalski's and Meng To's MIT-licensed work). Wrap-up now grows this log and the wiki the same way it already grows the phrase list. +Foundry's cost guidance now follows the evidence instead of treating review rounds as the obvious expense. The full path keeps five rounds; the light path is described honestly as lower assurance. Always-loaded rules and source commands now have enforced size ceilings, the middle review rounds only reread fresh material, live voice checks stay on prose-heavy stages, and test-it plus wrap-up each own one complete check. The implementation challenge chain found six real defects before two clean rounds closed it, including loose policy tests, dead vendor links, and a symlink route around the new budget. + +Friction: All five required implementation rounds found defects, so the chain followed its own confidence rule through round eight; rounds seven and eight were clean. diff --git a/docs/sessions/history/2026-W29.md b/docs/sessions/history/2026-W29.md new file mode 100644 index 0000000..52fb27f --- /dev/null +++ b/docs/sessions/history/2026-W29.md @@ -0,0 +1,5 @@ +# Sessions, 2026-W29 + +## 2026-07-15: The memory system and the reference library + +Foundry gained its memory and its bookshelf. This log now exists, with weekly rotation keeping it readable (`scripts/rotate-sessions.js`, loss-checked so rotation can never eat an entry). The wiki opened at `docs/wiki/` with an enforced index: a page nobody links from the index fails the check, so the map stays true. And the three general reference libraries moved in as the wiki's first residents: engineering fundamentals (Brooks, Parnas, Naur, and company), design fundamentals (the Norman-to-Rams canon), and motion fundamentals (adapted with credit from Emil Kowalski's and Meng To's MIT-licensed work). Wrap-up now grows this log and the wiki the same way it already grows the phrase list.