From c02c709098ab3e77bf8f50a41e7b36898d1420d9 Mon Sep 17 00:00:00 2001
From: Jakob Heuser <jakob@codedrift.com>
Date: Wed, 2 Sep 2026 11:50:11 -0700
Subject: [PATCH 1/5] docs(openspec): propose the runtime fixture runner

`taskless test` reports a runtime rule as passing when its fixtures never ran.
`testOneRule` returns `ok: true, errors: [], ran: false`, and the renderer reads
only `ok`, so the command prints a tick and "1 rule(s) tested" about a rule it
did not test.

This is an unimplemented requirement rather than a missing feature.
`cli-rule-validation` already says `test` executes a rule against "ast-grep test
cases, Vale pass/fail fixture buckets, or the runtime harness". Two of the three
are implemented and have scenarios; the third has neither, and its absence
reports as success.

The decision the change turns on is what to do with a run that did not happen.
Returning `ok: false` is the obvious correction and is also wrong: a rule that
cannot run because nothing blessed it is not defective, and failing it would
turn `test` red for every project holding a runtime rule with nothing available
to make it green. So there is a third outcome, reported as itself.

The gate is unchanged. A fixture run executes the same check.ts under the same
signature as a scan, and that the input is test data is a statement about the
input rather than about what the program may do.

The Vale runner is the model, including its strictness: buckets read
independently so an unreadable one cannot present as empty, coverage classified
four ways so only `both` can pass, and a non-directory entry refused by name
rather than skipped. Runtime differs in one way that matters, since a case is a
directory and that directory is the harness root.
---
 .../runtime-fixture-runner/.openspec.yaml     |   2 +
 .../changes/runtime-fixture-runner/design.md  | 125 ++++++++++++++++++
 .../runtime-fixture-runner/proposal.md        |  85 ++++++++++++
 .../specs/cli-rule-validation/spec.md         |  52 ++++++++
 .../specs/cli-runtime-rule-execution/spec.md  |  62 +++++++++
 .../changes/runtime-fixture-runner/tasks.md   |  53 ++++++++
 6 files changed, 379 insertions(+)
 create mode 100644 openspec/changes/runtime-fixture-runner/.openspec.yaml
 create mode 100644 openspec/changes/runtime-fixture-runner/design.md
 create mode 100644 openspec/changes/runtime-fixture-runner/proposal.md
 create mode 100644 openspec/changes/runtime-fixture-runner/specs/cli-rule-validation/spec.md
 create mode 100644 openspec/changes/runtime-fixture-runner/specs/cli-runtime-rule-execution/spec.md
 create mode 100644 openspec/changes/runtime-fixture-runner/tasks.md

diff --git a/openspec/changes/runtime-fixture-runner/.openspec.yaml b/openspec/changes/runtime-fixture-runner/.openspec.yaml
new file mode 100644
index 00000000..032461ff
--- /dev/null
+++ b/openspec/changes/runtime-fixture-runner/.openspec.yaml
@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-09-02
diff --git a/openspec/changes/runtime-fixture-runner/design.md b/openspec/changes/runtime-fixture-runner/design.md
new file mode 100644
index 00000000..703b0b7c
--- /dev/null
+++ b/openspec/changes/runtime-fixture-runner/design.md
@@ -0,0 +1,125 @@
+## Context
+
+Three engines, three fixture stories, and one of them is missing. `sg` runs
+`ast-grep test` over `.tests/`; `vale` runs its rule over `pass/` and `fail/`
+buckets and checks both directions; `runtime` returns `ran: false, ok: true` and
+prints a tick.
+
+The Vale runner is the model, and it is worth reading before writing this one:
+`rules/vale/verify.ts` already carries the lessons this tier will otherwise
+relearn. Buckets are read independently, so a swallowed `EACCES` on `pass/`
+cannot present as "no pass fixtures were written". A nested directory is
+rejected rather than ignored, because the reader is flat while the engine
+recurses, and silently skipping an entry fails in the dangerous direction.
+Coverage is a four-way classification rather than a boolean, because
+`fail-only` is more misleading than `none`.
+
+## Goals / Non-Goals
+
+**Goals.** Run a runtime rule's fixtures. Report honestly when they did not run.
+Make `.tests/` mean something for the one tier that ships them and reads none.
+
+**Non-Goals.** Changing the execution gate. Adding a runtime-specific bypass.
+Making `check` run fixtures — `check` scans a repository, `test` runs fixtures,
+and that separation is why the gate can be shared without the commands merging.
+
+## Decisions
+
+### D1 — The gate is `check`'s gate, unchanged
+
+A fixture run executes `check.ts`. That is the same code, from the same
+delivery, with the same signature, as the code `check` runs against a
+repository. So it runs under the same policy: blessed by an authenticated
+reconcile, or `--dangerously-run-scripts`.
+
+Nothing about a fixture makes the code safer. The bytes do not know what
+directory they are pointed at, and "it is only running against test data" is a
+statement about the input, not about what the program may do. A separate,
+softer gate for fixtures would be the client-side bypass the runtime spec
+already forbids, arrived at by a different route.
+
+### D2 — A run that did not happen is a third state, not a pass and not a failure
+
+This is the decision the whole change turns on.
+
+Today the runtime branch returns `ok: true`, which is wrong. The obvious
+correction is `ok: false`, and it is also wrong: a rule that cannot run because
+nothing blessed it is not a defective rule, and failing it would turn `test` red
+for every project holding a runtime rule, with no action available that makes it
+green. That trades a silent wrong answer for a loud useless one.
+
+So `test` reports three outcomes for a runtime rule: it ran and the fixtures
+behaved, it ran and they did not, or it did not run and here is why. The third
+prints its own marker rather than a tick, names the reason, and does not count
+toward "N rule(s) tested". `ran` stops being advisory metadata in the `--json`
+envelope and becomes the field a caller branches on.
+
+The precedent cuts the other way for the other engines and that is consistent
+rather than contradictory. `verify.ts` says "skips are errors, never a pass",
+and for `sg` a skip means something went wrong — the runner exists, so declining
+to use it is a fault. For `runtime` the run is refused by a security policy
+working as designed. Same word, two situations, opposite correct handling. That
+distinction is the one this codebase got wrong once already with `unsafe` versus
+`missing`.
+
+### D3 — A fixture case is a directory, because the harness takes a root
+
+Vale's buckets hold documents. Runtime's hold **directories**, one per case, and
+the case directory is the `root` passed to the check. `types/runtime-rule.ts`
+defines a check as a function over `(root, matches)` that reads files under
+`root`, and `create-runtime-rule.txt` already documents `.tests/pass/case-1/`
+with that meaning.
+
+This is not a stylistic difference. A runtime rule exists because its evidence
+spans more than one file, so a layout allowing one file per case could not
+express the rules the tier is for. It also settles the recursion question Vale
+had to answer: a bucket is read one level deep, and each entry must be a
+directory. A loose file directly in `pass/` is an error naming the path, not a
+case with an implicit root.
+
+### D4 — Coverage is classified, and only `both` can pass
+
+Copied from `ValeFixtureCoverage` deliberately, including the reasoning. A rule
+with only `fail/` cases has shown it fires and not that it stays quiet; a rule
+with only `pass/` cases has shown the opposite; and a rule with neither has
+shown nothing while exiting zero. `none`, `pass-only`, `fail-only` and `both`
+are kept apart because a caller says different things about them.
+
+### D5 — Reading the buckets cannot be lenient
+
+The Vale reader rethrows anything that is not a missing directory, because
+swallowing an `EACCES` on one bucket makes a two-sided rule look one-sided and a
+one-sided rule look complete. The same applies here and is worth stating rather
+than inheriting by imitation: a fixture bucket that could not be read is an
+error, never an empty bucket.
+
+### D6 — Authoring a runtime rule locally now requires the flag
+
+A locally authored rule has no signature and never will, because blessing is
+recording and nothing recorded it. So its author must pass
+`--dangerously-run-scripts` to test their own rule, which `sg` and `vale`
+authors do not have to do.
+
+That is friction and it is the correct friction. The alternative is a rule that
+executes unblessed code because it happens to live in the working tree, which is
+exactly the property the gate exists to deny — a delivered rule and an authored
+one are indistinguishable on disk. The warning the flag prints is accurate in
+both cases.
+
+Worth stating in the recipe rather than discovered: an author who runs `test`
+and sees "did not run" should be told the flag in that message.
+
+## Risks / Trade-offs
+
+**A rule with no fixtures.** The other two engines fail it, on the grounds that
+nothing shows it fires or stays quiet. Runtime rules have not been carrying
+fixtures, because nothing ran them, so applying the same rule immediately would
+fail rules that were correct under the old behaviour. The proposal is to report
+it as unverified in the same third state as a gated-out run, and to revisit once
+delivered rules carry fixtures as a matter of course. Recorded here rather than
+decided quietly.
+
+**The runner is a second execution path.** It reuses `invoke.ts` rather than
+reimplementing invocation, so a divergence between how a fixture runs and how
+`check` runs is a shared-code question rather than a drift question. If the two
+ever need different behaviour, that is the signal to stop and ask why.
diff --git a/openspec/changes/runtime-fixture-runner/proposal.md b/openspec/changes/runtime-fixture-runner/proposal.md
new file mode 100644
index 00000000..e265f542
--- /dev/null
+++ b/openspec/changes/runtime-fixture-runner/proposal.md
@@ -0,0 +1,85 @@
+## Why
+
+`taskless test` reports a runtime rule as passing when its fixtures never ran.
+Measured against a scaffolded project holding one runtime rule:
+
+```
+$ taskless test                    $ taskless test --json
+✓ runtime/demo-probe               {"ok":true,"rules":[{"engine":"runtime",
+                                     "ruleId":"demo-probe","ok":true,
+1 rule(s) tested.                    "errors":[],"ran":false}]}
+exit 0
+```
+
+`testOneRule` returns `ok: true, errors: [], ran: false` for the runtime engine,
+and the human renderer reads only `ok` — `ran` exists in the `--json` schema and
+nothing prints it. So the command whose job is to prove a rule fires says
+"1 rule(s) tested" about a rule it did not test.
+
+**This is an unimplemented requirement, not a missing feature.**
+`cli-rule-validation` already says `test` executes a rule against "ast-grep test
+cases, Vale `pass`/`fail` fixture buckets, or the runtime harness". Two of those
+three are implemented and have scenarios. The third has neither, and its absence
+is reported as success.
+
+Three things make it worth doing now rather than later.
+
+**Nothing else consumes a runtime rule's fixtures.** Delivery accepts `.tests/`
+paths, `PRESERVED_SUBTREES` keeps them through a purge, and `strayModules`
+exempts them at any depth so a TypeScript fixture does not trip the
+one-executable-file rule. Every piece of support exists except the thing that
+runs them.
+
+**The demonstration depends on it.** The generator team withdrew the findings
+the demo was going to serve, on the grounds that they are implied by delivery.
+What a reader gets instead is a rule they can run — so a `test` that reports
+green without running is, in their words, the difference between a
+demonstration and a claim. They have recorded it as a limit on what their side
+can assert.
+
+**It is this repository's own named failure mode, in the command built to catch
+it.** The same requirement says an engine "SHALL NOT be trusted to report this
+itself", because `ast-grep test` calls an empty `invalid:` bucket a pass. The
+runtime tier does the same thing one level up.
+
+## What Changes
+
+**A runtime fixture runner.** `test` executes a rule's `check.ts` against each
+case under `.tests/pass/` and `.tests/fail/`, with the case directory as the
+`root` the harness hands the check. A `fail/` case SHALL produce at least one
+finding; a `pass/` case SHALL produce none.
+
+**Gated by the gate that already exists.** Running a fixture executes delivered
+code, so it runs under the same policy `check` uses: an authenticated reconcile
+that returns the rule's signature in `run`, or `--dangerously-run-scripts`.
+Neither is a new mechanism and no demo rule id is special-cased.
+
+**A run that did not happen is reported as such.** Not as a pass, and not as a
+failure. A rule that cannot run because nothing blessed it and no flag was
+passed is not a defect in the rule, and failing it would turn `test` red for
+every project holding a runtime rule with no way to make it green.
+
+**Fixture coverage is classified, not counted.** Only a rule populating both
+buckets can pass, which is what the requirement already demands of the other
+two engines.
+
+## Capabilities
+
+### Modified Capabilities
+
+- `cli-rule-validation`: the `test` requirement gains the runtime scenarios it
+  names an engine for and never specified, plus the reporting rule for a run
+  that was gated out.
+- `cli-runtime-rule-execution`: gains the fixture runner and its gating.
+
+## Impact
+
+- `packages/cli/src/rules/runtime/` — a fixture runner beside the existing
+  harness, reusing `invoke.ts` rather than a second execution path.
+- `packages/cli/src/rules/inspect.ts` — `testOneRule`'s runtime branch stops
+  returning `ok: true` for a run that did not happen.
+- `packages/cli/src/commands/verify.ts` — the renderer distinguishes a skipped
+  rule from a passing one, and the summary line stops counting it as tested.
+- `packages/cli/src/schemas/verify-test.ts` — `ran` becomes load-bearing rather
+  than advisory.
+- No change to `check`, to reconcile, or to the execution gate itself.
diff --git a/openspec/changes/runtime-fixture-runner/specs/cli-rule-validation/spec.md b/openspec/changes/runtime-fixture-runner/specs/cli-rule-validation/spec.md
new file mode 100644
index 00000000..9dd63b3d
--- /dev/null
+++ b/openspec/changes/runtime-fixture-runner/specs/cli-rule-validation/spec.md
@@ -0,0 +1,52 @@
+## MODIFIED Requirements
+
+### Requirement: Test runs a rule's fixtures and runs verify first
+
+`test` SHALL execute a rule against its test material — ast-grep test cases, Vale `pass`/`fail` fixture buckets, or the runtime harness — and SHALL run `verify` first, stopping on a verify failure without running the fixtures.
+
+Ordering is the point. When a rule is both malformed and under-fixtured, the fixture complaint is the less useful of the two errors and is the one that surfaces first if the checks run in the other order — so the author is told their fixtures are incomplete while the reason the rule could never have run goes unmentioned.
+
+A rule that populates only one bucket has proved only half of what a rule claims, whatever its engine. An engine SHALL NOT be trusted to report this itself: `ast-grep test` reports an empty `invalid:` bucket as `1 passed; 0 failed` and exits zero, so a rule that has never matched anything is indistinguishable from one that passed.
+
+A runtime rule's fixtures execute delivered code, so they SHALL run only under the policy `check` already applies: an authenticated reconcile that returns the rule's signature in `run`, or `--dangerously-run-scripts`. A run refused by that policy SHALL be reported as not run, and SHALL be reported as neither a pass nor a failure: the rule is not defective, and no action available to its holder would make a failure green.
+
+#### Scenario: A malformed rule reports the malformation, not the fixtures
+
+- **WHEN** `test` runs against a rule that is both invalid and missing a fixture bucket
+- **THEN** it SHALL report the validation error
+- **AND** it SHALL NOT report the fixture coverage as the failure
+
+#### Scenario: Vale fixtures are tested per bucket
+
+- **WHEN** `test` runs against a Vale rule
+- **THEN** every `fail/` document SHALL produce at least one finding for that rule
+- **AND** every `pass/` document SHALL produce none
+- **AND** a rule populating only one bucket SHALL be reported as unverified rather than passing
+
+#### Scenario: ast-grep fixtures are counted per bucket
+
+- **WHEN** `test` runs against an ast-grep rule
+- **THEN** the `valid:` and `invalid:` entries SHALL be counted across every `-test.yml` file the rule owns
+- **AND** a rule populating only one bucket SHALL be reported as unverified rather than passing
+- **AND** a rule whose buckets are all empty or absent SHALL be reported as unverified rather than passing
+- **AND** a green `ast-grep test` run SHALL NOT on its own be sufficient to report the rule as passing
+
+#### Scenario: Runtime fixtures are run per case
+
+- **WHEN** `test` runs against a runtime rule and the execution policy permits it
+- **THEN** each directory under `.tests/fail/` SHALL be passed to the check as its `root` and SHALL produce at least one finding
+- **AND** each directory under `.tests/pass/` SHALL be passed as its `root` and SHALL produce none
+- **AND** a rule populating only one bucket SHALL be reported as unverified rather than passing
+
+#### Scenario: A runtime rule the policy refuses is reported as not run
+
+- **WHEN** `test` runs against a runtime rule with no blessed signature and no `--dangerously-run-scripts`
+- **THEN** the rule SHALL NOT be reported as passing
+- **AND** the output SHALL say the fixtures did not run and why
+- **AND** the rule SHALL NOT be counted among the rules tested
+- **AND** the refusal alone SHALL NOT fail the command
+
+#### Scenario: A check that throws is distinguished from one that finds nothing
+
+- **WHEN** a runtime rule's check raises while running a fixture case
+- **THEN** that SHALL be reported as the check failing, not as the case producing no findings
diff --git a/openspec/changes/runtime-fixture-runner/specs/cli-runtime-rule-execution/spec.md b/openspec/changes/runtime-fixture-runner/specs/cli-runtime-rule-execution/spec.md
new file mode 100644
index 00000000..57af3e8b
--- /dev/null
+++ b/openspec/changes/runtime-fixture-runner/specs/cli-runtime-rule-execution/spec.md
@@ -0,0 +1,62 @@
+## ADDED Requirements
+
+### Requirement: A runtime rule's fixtures are executed through the harness
+
+The CLI SHALL execute a runtime rule against the fixture cases it holds under
+`.tests/pass/` and `.tests/fail/`. A case SHALL be a **directory**, and that
+directory SHALL be the `root` the harness passes to the check.
+
+Each bucket SHALL be read one level deep. An entry that is not a directory SHALL
+be an error naming the path, not an ignored entry. Buckets SHALL be read
+independently, and a bucket that cannot be read SHALL be an error rather than an
+empty bucket.
+
+**Rationale.** A check is a function over a root and its matches, reading
+whatever files under that root it needs, so a case has to be a directory: a
+runtime rule exists because its evidence spans more than one file, and a
+one-file-per-case layout could not express the rules the tier is for. The
+strictness on reading is the Vale runner's, for its reason — a swallowed
+permission error on one bucket makes a two-sided rule look one-sided, and a
+one-sided rule look complete.
+
+#### Scenario: A case directory is the harness root
+
+- **WHEN** a runtime rule's fixture case is executed
+- **THEN** the check SHALL receive that case's directory as its `root`
+- **AND** SHALL resolve the files it reads beneath that root
+
+#### Scenario: A loose file in a bucket is refused
+
+- **WHEN** a fixture bucket contains an entry that is not a directory
+- **THEN** the CLI SHALL report an error naming that path
+- **AND** SHALL NOT silently skip it
+
+#### Scenario: An unreadable bucket is not an empty bucket
+
+- **WHEN** a fixture bucket exists but cannot be read
+- **THEN** the CLI SHALL report the failure
+- **AND** SHALL NOT treat the bucket as holding no cases
+
+### Requirement: Fixture execution obeys the runtime execution gate
+
+Executing a fixture case SHALL be permitted only where executing the rule itself
+would be: when an authenticated reconcile returns the rule's signature in `run`,
+or when `--dangerously-run-scripts` is passed. No rule identifier SHALL be
+exempt, and the fixture path SHALL NOT constitute a separate or softer gate.
+
+**Rationale.** A fixture run executes the same `check.ts`, from the same
+delivery, under the same signature as a scan. That the input is test data is a
+statement about the input, not about what the program may do. A softer gate for
+fixtures would be the client-side bypass this capability already forbids,
+reached by another route.
+
+#### Scenario: Fixtures do not run for an unblessed rule
+
+- **WHEN** `test` runs against a runtime rule whose signature no reconcile has blessed, without the escape flag
+- **THEN** no fixture case SHALL be executed
+
+#### Scenario: The documented escape runs fixtures
+
+- **WHEN** `test` runs with `--dangerously-run-scripts`
+- **THEN** fixture cases SHALL execute under that flag's existing warning
+- **AND** under no other mechanism
diff --git a/openspec/changes/runtime-fixture-runner/tasks.md b/openspec/changes/runtime-fixture-runner/tasks.md
new file mode 100644
index 00000000..483da32e
--- /dev/null
+++ b/openspec/changes/runtime-fixture-runner/tasks.md
@@ -0,0 +1,53 @@
+# Tasks
+
+One PR. The runner and the reporting change are the same defect from two sides:
+running the fixtures without fixing the report leaves a tick on a gated-out
+rule, and fixing the report without the runner leaves every runtime rule
+permanently unverified.
+
+## 1. Read the fixtures
+
+- [ ] 1.1 Enumerate `.tests/pass/` and `.tests/fail/` one level deep, requiring each entry to be a directory (D3). A loose file names its own path in the error rather than being skipped
+- [ ] 1.2 Read the two buckets independently and rethrow anything that is not a missing directory, so an unreadable bucket cannot present as an empty one (D5)
+- [ ] 1.3 Classify coverage as `both` / `pass-only` / `fail-only` / `none`, and let only `both` reach a pass (D4)
+
+## 2. Run them
+
+- [ ] 2.1 Execute the rule's `check.ts` once per case, with the case directory as the harness `root` (D3), through the existing `invoke.ts` rather than a second invocation path
+- [ ] 2.2 Require every `fail/` case to produce at least one finding, and every `pass/` case to produce none. Name the cases that broke either direction, the way the Vale runner names `missingFailures` and `unexpectedFindings`
+- [ ] 2.3 Treat a check that throws as a distinct outcome from a check that returned no findings. Both are zero findings downstream, and only one of them is the rule's fault
+
+## 3. Gate it exactly as `check` does
+
+- [ ] 3.1 Run fixtures only when an authenticated reconcile returns the rule's signature in `run`, or when `--dangerously-run-scripts` is passed (D1)
+- [ ] 3.2 Add `--dangerously-run-scripts` to `test`, printing the same warning `check` prints
+- [ ] 3.3 Test that no rule id is special-cased, and that a gated-out fixture run executes nothing
+- [ ] 3.4 Test that the flag is the only mechanism besides a blessed signature. A fixture path is not a softer gate because its input is test data
+
+## 4. Report a run that did not happen
+
+- [ ] 4.1 Stop returning `ok: true` from `testOneRule`'s runtime branch for a run that did not occur
+- [ ] 4.2 Render a skipped rule with its own marker and the reason, never a tick (D2)
+- [ ] 4.3 Stop counting a skipped rule in "N rule(s) tested"
+- [ ] 4.4 Make `ran` load-bearing in the `--json` envelope rather than advisory, and confirm the exit code does not fail on a gated-out run alone
+- [ ] 4.5 Say what to do about it: a rule that did not run because nothing blessed it should name `--dangerously-run-scripts` in the message (D6)
+
+## 5. Prove it bites
+
+- [ ] 5.1 Reproduce the current defect first, as a failing test: a runtime rule whose fixtures never ran must not report `ok: true` and must not print a tick
+- [ ] 5.2 Test a rule whose `fail/` case produces no findings, which is the silent regression this exists to catch
+- [ ] 5.3 Test a rule whose `pass/` case produces findings, which is the indiscriminate rule
+- [ ] 5.4 Test each coverage class, asserting only `both` can pass
+- [ ] 5.5 Test that an unreadable bucket is an error and not an empty bucket (D5)
+- [ ] 5.6 Revert the runner and watch the suite fail before believing it
+
+## 6. Say so where an author reads it
+
+- [ ] 6.1 `create-runtime-rule` states that testing a locally authored rule needs the flag, and why (D6)
+- [ ] 6.2 `verify-rule` stops reporting runtime tests as "not run" without saying what was not run
+
+## 7. Close out
+
+- [ ] 7.1 Decide the no-fixtures case: unverified in the third state now, or a failure matching the other engines. Recorded as an open risk rather than settled quietly
+- [ ] 7.2 Tell the generator team, who have recorded this as a limit on what the demonstration can assert
+- [ ] 7.3 Archive the change

From 98b1d559837f0204412876bfcdf7615645b91a29 Mon Sep 17 00:00:00 2001
From: Jakob Heuser <jakob@codedrift.com>
Date: Wed, 2 Sep 2026 11:56:52 -0700
Subject: [PATCH 2/5] docs(openspec): a rule needs a fixture, and the runner is
 a caller not a sibling

Settles the open question in 7.1: a runtime rule with no fixture cases fails,
matching ast-grep and Vale. Making the tier that executes arbitrary code the one
whose rules need not prove anything would be the wrong exception.

Depth is not policed. A trivial case that exercises little and passes is fine,
but the latitude reaches one bucket only: a `fail/` case producing no findings
is the silent regression this runner exists to catch, so a case that tests
nothing can only be a `pass/` case. The `fail/` case is the one that has to do
real work, and the one worth having.

Also corrects the change's own framing, which understated what already exists.
Runtime rules execute today: `check` reaches `executeRuntimeRule` through
`dispatch.ts:205` with the repository root, gated by signature planning. The
gap is fixtures, and only in `test`.

That makes D3 an observation rather than a decision. `executeRuntimeRule`
already takes a root, so a case directory is a different root handed to the same
function, and the runner is a loop plus an assertion over machinery `check`
already proves on every authenticated run.
---
 .../changes/runtime-fixture-runner/design.md  | 63 ++++++++++++++-----
 .../runtime-fixture-runner/proposal.md        |  7 ++-
 .../specs/cli-rule-validation/spec.md         |  1 +
 .../changes/runtime-fixture-runner/tasks.md   |  2 +-
 4 files changed, 53 insertions(+), 20 deletions(-)

diff --git a/openspec/changes/runtime-fixture-runner/design.md b/openspec/changes/runtime-fixture-runner/design.md
index 703b0b7c..e1a850a2 100644
--- a/openspec/changes/runtime-fixture-runner/design.md
+++ b/openspec/changes/runtime-fixture-runner/design.md
@@ -1,9 +1,16 @@
 ## Context
 
-Three engines, three fixture stories, and one of them is missing. `sg` runs
-`ast-grep test` over `.tests/`; `vale` runs its rule over `pass/` and `fail/`
-buckets and checks both directions; `runtime` returns `ran: false, ok: true` and
-prints a tick.
+**Runtime rules do execute today.** `check` runs them against the repository
+through `dispatch.ts:205` into `executeRuntimeRule`, gated by the signature
+planning that happens before dispatch. The executor, the process spawn, the
+timeout and the result shape all exist and are exercised on every authenticated
+`check`. Nothing here is about making runtime rules runnable.
+
+What is missing is running them against **fixtures**. Three engines, three
+fixture stories, and one of them is absent. `sg` runs `ast-grep test` over
+`.tests/`; `vale` runs its rule over `pass/` and `fail/` buckets and checks both
+directions; `runtime` returns `ran: false, ok: true` and prints a tick. Two
+commands, one engine, one of them wired up.
 
 The Vale runner is the model, and it is worth reading before writing this one:
 `rules/vale/verify.ts` already carries the lessons this tier will otherwise
@@ -62,13 +69,24 @@ working as designed. Same word, two situations, opposite correct handling. That
 distinction is the one this codebase got wrong once already with `unsafe` versus
 `missing`.
 
-### D3 — A fixture case is a directory, because the harness takes a root
+### D3 — A fixture case is a directory, because the harness already takes a root
 
-Vale's buckets hold documents. Runtime's hold **directories**, one per case, and
-the case directory is the `root` passed to the check. `types/runtime-rule.ts`
-defines a check as a function over `(root, matches)` that reads files under
-`root`, and `create-runtime-rule.txt` already documents `.tests/pass/case-1/`
-with that meaning.
+This is less a choice than an observation. `executeRuntimeRule(root, rule,
+options)` in `runtime/harness.ts` already takes a root, and `check` reaches it
+through `dispatch.ts:205` with the repository root. A fixture case directory is
+simply a different root handed to the same function.
+
+So the runner is a caller of the existing executor rather than a sibling of it:
+a loop over case directories plus the pass/fail assertion. Process spawn,
+timeout, narrowing, capture discovery and the result shape are all shared with
+`check` and already proven there. Any other case layout would need new plumbing
+instead of reusing what exists, which is the argument for directories rather
+than a preference for them.
+
+Vale's buckets hold documents; runtime's hold **directories**, one per case.
+`types/runtime-rule.ts` defines a check as a function over `(root, matches)`
+reading files under `root`, and `create-runtime-rule.txt` already documents
+`.tests/pass/case-1/` with that meaning.
 
 This is not a stylistic difference. A runtime rule exists because its evidence
 spans more than one file, so a layout allowing one file per case could not
@@ -111,13 +129,24 @@ and sees "did not run" should be told the flag in that message.
 
 ## Risks / Trade-offs
 
-**A rule with no fixtures.** The other two engines fail it, on the grounds that
-nothing shows it fires or stays quiet. Runtime rules have not been carrying
-fixtures, because nothing ran them, so applying the same rule immediately would
-fail rules that were correct under the old behaviour. The proposal is to report
-it as unverified in the same third state as a gated-out run, and to revisit once
-delivered rules carry fixtures as a matter of course. Recorded here rather than
-decided quietly.
+**A rule with no fixtures fails, and that is the decision.** The other two
+engines fail it, on the grounds that nothing shows the rule fires or stays
+quiet, and making runtime the exception would leave the tier that executes
+arbitrary code as the one whose rules need not prove anything.
+
+Depth is not policed. A trivial case that exercises little and passes is
+acceptable; the requirement is that a rule has fixtures, not that they are
+thorough. But that latitude only reaches one bucket. A `fail/` case producing no
+findings is the silent regression this runner exists to catch, so a case that
+tests nothing can only be a `pass/` case, and the `fail/` case is the one that
+has to do real work. That is also the one worth having, since it is what proves
+the rule fires at all.
+
+The cost is real and worth naming: runtime rules have not been carrying
+fixtures, because nothing ran them, so this fails rules that were correct under
+the old behaviour. That is the same trade this CLI already made when `check`,
+`verify` and `test` stopped migrating silently — a wall met once, in exchange
+for a report that means something.
 
 **The runner is a second execution path.** It reuses `invoke.ts` rather than
 reimplementing invocation, so a divergence between how a fixture runs and how
diff --git a/openspec/changes/runtime-fixture-runner/proposal.md b/openspec/changes/runtime-fixture-runner/proposal.md
index e265f542..9ec7270a 100644
--- a/openspec/changes/runtime-fixture-runner/proposal.md
+++ b/openspec/changes/runtime-fixture-runner/proposal.md
@@ -74,8 +74,11 @@ two engines.
 
 ## Impact
 
-- `packages/cli/src/rules/runtime/` — a fixture runner beside the existing
-  harness, reusing `invoke.ts` rather than a second execution path.
+- `packages/cli/src/rules/runtime/` — a fixture runner that CALLS the existing
+  executor rather than sitting beside it. `executeRuntimeRule` already takes a
+  root, and `check` already passes the repository root through `dispatch.ts`;
+  a case directory is a different root handed to the same function. What is new
+  is the loop over cases and the pass/fail assertion, not the execution.
 - `packages/cli/src/rules/inspect.ts` — `testOneRule`'s runtime branch stops
   returning `ok: true` for a run that did not happen.
 - `packages/cli/src/commands/verify.ts` — the renderer distinguishes a skipped
diff --git a/openspec/changes/runtime-fixture-runner/specs/cli-rule-validation/spec.md b/openspec/changes/runtime-fixture-runner/specs/cli-rule-validation/spec.md
index 9dd63b3d..0e064d91 100644
--- a/openspec/changes/runtime-fixture-runner/specs/cli-rule-validation/spec.md
+++ b/openspec/changes/runtime-fixture-runner/specs/cli-rule-validation/spec.md
@@ -37,6 +37,7 @@ A runtime rule's fixtures execute delivered code, so they SHALL run only under t
 - **THEN** each directory under `.tests/fail/` SHALL be passed to the check as its `root` and SHALL produce at least one finding
 - **AND** each directory under `.tests/pass/` SHALL be passed as its `root` and SHALL produce none
 - **AND** a rule populating only one bucket SHALL be reported as unverified rather than passing
+- **AND** a rule holding no fixture cases at all SHALL be reported as unverified rather than passing
 
 #### Scenario: A runtime rule the policy refuses is reported as not run
 
diff --git a/openspec/changes/runtime-fixture-runner/tasks.md b/openspec/changes/runtime-fixture-runner/tasks.md
index 483da32e..0db6ce02 100644
--- a/openspec/changes/runtime-fixture-runner/tasks.md
+++ b/openspec/changes/runtime-fixture-runner/tasks.md
@@ -48,6 +48,6 @@ permanently unverified.
 
 ## 7. Close out
 
-- [ ] 7.1 Decide the no-fixtures case: unverified in the third state now, or a failure matching the other engines. Recorded as an open risk rather than settled quietly
+- [x] 7.1 Decide the no-fixtures case. **Settled: it fails, matching the other engines.** Every rule needs a fixture. Depth is not policed, so a trivial case that exercises little and passes is acceptable, but it can only live in `pass/` — a `fail/` case that produces nothing is the silent regression this runner exists to catch
 - [ ] 7.2 Tell the generator team, who have recorded this as a limit on what the demonstration can assert
 - [ ] 7.3 Archive the change

From 13553d931188c8155b7ffb6560a6e1ded548fd5d Mon Sep 17 00:00:00 2001
From: Jakob Heuser <jakob@codedrift.com>
Date: Wed, 2 Sep 2026 11:59:50 -0700
Subject: [PATCH 3/5] docs(openspec): nothing to test is an incomplete
 delivery, sequenced not taken

A rule with no fixtures proves nothing, which makes it incomplete by the same
standard as a rule with no captures. Delivery already answers that question and
answers it before writing, so a refused set leaves no directory behind. Catching
it in `test` instead means the rule is already on disk and the failure reads as
the holder's fault for a file the service sent.

That is the styleguide's rule about build output applied to a payload: an
invariant enforced where the artifact is produced cannot be violated, while one
enforced afterwards can only be detected.

It is not taken here, because it is a contract change. A runtime delivery with
no `.tests/` is valid today, and the published spec names `ENGINE_LAYOUTS` as
the completeness authority, so requiring fixtures refuses payloads that are
correct under the agreed contract. If the service is not already sending them,
every runtime delivery breaks with nothing written.

So the halves land in order. `test` failing an authored rule is ours alone and
ships with this change. Delivery completeness follows once the generator team
has confirmed fixtures ride in `files[]` as a matter of course, as its own
change.
---
 .../changes/runtime-fixture-runner/design.md  | 29 +++++++++++++++++++
 .../changes/runtime-fixture-runner/tasks.md   |  4 ++-
 2 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/openspec/changes/runtime-fixture-runner/design.md b/openspec/changes/runtime-fixture-runner/design.md
index e1a850a2..76c0d6f7 100644
--- a/openspec/changes/runtime-fixture-runner/design.md
+++ b/openspec/changes/runtime-fixture-runner/design.md
@@ -127,6 +127,35 @@ both cases.
 Worth stating in the recipe rather than discovered: an author who runs `test`
 and sees "did not run" should be told the flag in that message.
 
+### D7 — Nothing to test means the delivery was incomplete, and that belongs in delivery
+
+`test` failing a fixtureless rule is the right answer for a rule someone wrote.
+It is the wrong place to catch a rule the service sent, because by then the rule
+is on disk and the failure reads as the holder's fault for a file they never
+authored.
+
+Delivery already answers "is this a complete rule": `describeIncompleteSet`
+requires the rule file, the per-engine config where one exists, and at least one
+capture, and the whole set is assessed before anything is written so a refused
+delivery leaves no directory behind. A rule with nothing to test is incomplete
+by the same standard as a rule with no captures. Both are rules that cannot
+demonstrate anything, and both are cheaper refused than written.
+
+This is the styleguide's own rule about build output applied to a payload: an
+invariant enforced where the artifact is produced cannot be violated, while one
+enforced afterwards can only be detected.
+
+**It is a contract change, so it is sequenced rather than taken.** Today a
+runtime delivery carrying no `.tests/` is valid, and `ENGINE_LAYOUTS` is what
+the published spec names as the completeness authority. Requiring fixtures means
+deliveries that are correct under the agreed contract start being refused with
+nothing written, which is a hard break if the service is not already sending
+them. So it needs their agreement first, and this change does not enforce it.
+
+The two halves land in the right order: `test` failing an authored rule is ours
+alone and ships now, delivery completeness follows once the service has
+committed to shipping fixtures.
+
 ## Risks / Trade-offs
 
 **A rule with no fixtures fails, and that is the decision.** The other two
diff --git a/openspec/changes/runtime-fixture-runner/tasks.md b/openspec/changes/runtime-fixture-runner/tasks.md
index 0db6ce02..7f94d7e0 100644
--- a/openspec/changes/runtime-fixture-runner/tasks.md
+++ b/openspec/changes/runtime-fixture-runner/tasks.md
@@ -50,4 +50,6 @@ permanently unverified.
 
 - [x] 7.1 Decide the no-fixtures case. **Settled: it fails, matching the other engines.** Every rule needs a fixture. Depth is not policed, so a trivial case that exercises little and passes is acceptable, but it can only live in `pass/` — a `fail/` case that produces nothing is the silent regression this runner exists to catch
 - [ ] 7.2 Tell the generator team, who have recorded this as a limit on what the demonstration can assert
-- [ ] 7.3 Archive the change
+- [ ] 7.3 Ask whether runtime deliveries carry `.tests/` today, and whether they will commit to it. A rule with nothing to test is an incomplete delivery (D7), but requiring it in `describeIncompleteSet` refuses payloads that are valid under the current contract, so it is theirs to agree before it is ours to enforce
+- [ ] 7.4 Once agreed, add fixtures to delivery completeness as its own change. Not here: this one must not turn a service omission into a refused write without warning
+- [ ] 7.5 Archive the change

From ff3551f48c3924b74a181d3764761c7dd6dacc11 Mon Sep 17 00:00:00 2001
From: Jakob Heuser <jakob@codedrift.com>
Date: Wed, 2 Sep 2026 12:03:44 -0700
Subject: [PATCH 4/5] docs(openspec): a case that never reaches the check is
 not zero findings

`executeRuntimeRule` gates on the narrow and returns an empty array without
invoking `check.ts` when nothing matches. So a fixture case whose narrow finds
nothing in its small directory is indistinguishable downstream from a check that
ran and found nothing, and the scenarios as drafted would fail a `fail/` case
for it while blaming the check for a fixture that never reached it.

That is this change's own failure mode reintroduced inside the fix, which is
worth fixing in a proposal rather than after implementation.

The runner therefore needs the invocation signal, not only the findings, and
reports three outcomes per case. A case producing no narrow matches is a fixture
defect in BOTH buckets: the `pass/` side is the quieter half, because such a
case reads as a clean pass while proving only that the narrow did not match. It
cannot show the check stays quiet, so counting it as evidence is the same
empty-scan-reports-success shape one level down.

Found by review.
---
 .../changes/runtime-fixture-runner/design.md  | 30 +++++++++++++++++++
 .../specs/cli-rule-validation/spec.md         |  7 +++++
 .../changes/runtime-fixture-runner/tasks.md   |  2 ++
 3 files changed, 39 insertions(+)

diff --git a/openspec/changes/runtime-fixture-runner/design.md b/openspec/changes/runtime-fixture-runner/design.md
index 76c0d6f7..40a5e80c 100644
--- a/openspec/changes/runtime-fixture-runner/design.md
+++ b/openspec/changes/runtime-fixture-runner/design.md
@@ -156,6 +156,36 @@ The two halves land in the right order: `test` failing an authored rule is ours
 alone and ships now, delivery completeness follows once the service has
 committed to shipping fixtures.
 
+### D8 — A case that never reaches the check is its own outcome, not zero findings
+
+`executeRuntimeRule` runs the narrow first and gates on it:
+
+```ts
+if (matches.length === 0) return []; // gate: no matches, no check
+```
+
+So a fixture case whose narrow matches nothing returns an empty array having
+never invoked `check.ts`, which is indistinguishable downstream from a check
+that ran and found nothing. Under the scenarios as first drafted, a `fail/` case
+in that state fails, and the message blames the check for a fixture that never
+reached it.
+
+That is this change's own failure mode, reintroduced inside the fix. The runner
+therefore has to know whether the check was **invoked**, not only what it
+returned, and report three outcomes per case rather than two.
+
+It matters in both buckets, and the `pass/` side is the quieter half. A `pass/`
+case with no narrow matches looks like a clean pass and proves nothing about the
+check: it demonstrates that the narrow did not match, which is a fact about the
+fixture. A case that never invokes the check cannot show the check stays quiet,
+so counting it as evidence is the same empty-scan-reports-success shape one
+level down.
+
+So a case producing no narrow matches is reported as a **fixture defect** in
+either bucket, naming the case and saying the check never ran. It is actionable
+in a way "expected a finding, got none" is not, and it points at the file the
+author has to change.
+
 ## Risks / Trade-offs
 
 **A rule with no fixtures fails, and that is the decision.** The other two
diff --git a/openspec/changes/runtime-fixture-runner/specs/cli-rule-validation/spec.md b/openspec/changes/runtime-fixture-runner/specs/cli-rule-validation/spec.md
index 0e064d91..1e478376 100644
--- a/openspec/changes/runtime-fixture-runner/specs/cli-rule-validation/spec.md
+++ b/openspec/changes/runtime-fixture-runner/specs/cli-rule-validation/spec.md
@@ -47,6 +47,13 @@ A runtime rule's fixtures execute delivered code, so they SHALL run only under t
 - **AND** the rule SHALL NOT be counted among the rules tested
 - **AND** the refusal alone SHALL NOT fail the command
 
+#### Scenario: A case that never reaches the check is reported as a fixture defect
+
+- **WHEN** a fixture case produces no narrow matches, so the check is never invoked
+- **THEN** the CLI SHALL report that case as a fixture defect naming the case
+- **AND** SHALL say the check did not run
+- **AND** SHALL do so whether the case is in `pass/` or `fail/`, since a case that never invokes the check is evidence about the fixture rather than about the rule
+
 #### Scenario: A check that throws is distinguished from one that finds nothing
 
 - **WHEN** a runtime rule's check raises while running a fixture case
diff --git a/openspec/changes/runtime-fixture-runner/tasks.md b/openspec/changes/runtime-fixture-runner/tasks.md
index 7f94d7e0..51ebe71b 100644
--- a/openspec/changes/runtime-fixture-runner/tasks.md
+++ b/openspec/changes/runtime-fixture-runner/tasks.md
@@ -16,6 +16,8 @@ permanently unverified.
 - [ ] 2.1 Execute the rule's `check.ts` once per case, with the case directory as the harness `root` (D3), through the existing `invoke.ts` rather than a second invocation path
 - [ ] 2.2 Require every `fail/` case to produce at least one finding, and every `pass/` case to produce none. Name the cases that broke either direction, the way the Vale runner names `missingFailures` and `unexpectedFindings`
 - [ ] 2.3 Treat a check that throws as a distinct outcome from a check that returned no findings. Both are zero findings downstream, and only one of them is the rule's fault
+- [ ] 2.4 Distinguish a case that never reached the check from one where the check found nothing (D8). `executeRuntimeRule` gates on the narrow and returns `[]` without invoking `check.ts`, so the runner needs the invocation signal rather than only the findings
+- [ ] 2.5 Report a case producing no narrow matches as a fixture defect in BOTH buckets, naming the case and saying the check never ran. A `pass/` case that never invokes the check proves nothing about the check staying quiet
 
 ## 3. Gate it exactly as `check` does
 

From d6a5c4b2261092ce47f5ad4a22484e233b9d0e5f Mon Sep 17 00:00:00 2001
From: Jakob Heuser <jakob@codedrift.com>
Date: Wed, 2 Sep 2026 12:13:12 -0700
Subject: [PATCH 5/5] docs(openspec): a runtime rule ships with fixtures, and
 that is not a proposal

D7 framed requiring `.tests/` as a contract change needing the generator team's
agreement. That is wrong. A runtime rule ships with a fixtures directory, and a
delivery without one is a defect on their side rather than a payload shape we
are asking them to adopt. They already hold the material, since their
verification gate executes the generated check against failing and passing
examples before accepting the rule, and they have already established that
fixtures are files in the collection rather than a field beside it.

What is sequenced is enforcement, and for a reason unrelated to agreement.
Being right about whose bug it is does not stop `describeIncompleteSet` from
turning that bug into a refused write for every user, with nothing written and
nothing they can do. So: confirm deliveries carry them, file anything that does
not as the defect it is, and enforce once the fix has shipped.
---
 .../changes/runtime-fixture-runner/design.md  | 29 ++++++++++++-------
 .../changes/runtime-fixture-runner/tasks.md   |  4 +--
 2 files changed, 21 insertions(+), 12 deletions(-)

diff --git a/openspec/changes/runtime-fixture-runner/design.md b/openspec/changes/runtime-fixture-runner/design.md
index 40a5e80c..3e6ffa2d 100644
--- a/openspec/changes/runtime-fixture-runner/design.md
+++ b/openspec/changes/runtime-fixture-runner/design.md
@@ -145,16 +145,25 @@ This is the styleguide's own rule about build output applied to a payload: an
 invariant enforced where the artifact is produced cannot be violated, while one
 enforced afterwards can only be detected.
 
-**It is a contract change, so it is sequenced rather than taken.** Today a
-runtime delivery carrying no `.tests/` is valid, and `ENGINE_LAYOUTS` is what
-the published spec names as the completeness authority. Requiring fixtures means
-deliveries that are correct under the agreed contract start being refused with
-nothing written, which is a hard break if the service is not already sending
-them. So it needs their agreement first, and this change does not enforce it.
-
-The two halves land in the right order: `test` failing an authored rule is ours
-alone and ships now, delivery completeness follows once the service has
-committed to shipping fixtures.
+**This is not a negotiation.** A runtime rule ships with a `.tests/` directory;
+a delivery without one is a defect on the service side, not a payload shape we
+are asking them to adopt. They already hold the material — their verification
+gate executes the generated check against failing and passing examples on its
+way to accepting the rule — and they have already established that fixtures are
+files in the collection rather than a field beside it.
+
+**What is sequenced is enforcement, not the expectation.** Being right about
+whose bug it is does not stop `describeIncompleteSet` from turning that bug into
+a refused write for every user, with nothing written and no action available to
+them. So the order is: confirm deliveries carry `.tests/`, and if they do not,
+file it as the defect it is rather than proposing it as a change. Enforce once
+the fix has shipped.
+
+That leaves the two halves landing in the right order for a reason that has
+nothing to do with agreement. `test` failing an authored rule is ours alone and
+ships now. Delivery completeness follows once we know we are enforcing a rule
+the service already satisfies, rather than discovering it does not through an
+outage.
 
 ### D8 — A case that never reaches the check is its own outcome, not zero findings
 
diff --git a/openspec/changes/runtime-fixture-runner/tasks.md b/openspec/changes/runtime-fixture-runner/tasks.md
index 51ebe71b..1d519425 100644
--- a/openspec/changes/runtime-fixture-runner/tasks.md
+++ b/openspec/changes/runtime-fixture-runner/tasks.md
@@ -52,6 +52,6 @@ permanently unverified.
 
 - [x] 7.1 Decide the no-fixtures case. **Settled: it fails, matching the other engines.** Every rule needs a fixture. Depth is not policed, so a trivial case that exercises little and passes is acceptable, but it can only live in `pass/` — a `fail/` case that produces nothing is the silent regression this runner exists to catch
 - [ ] 7.2 Tell the generator team, who have recorded this as a limit on what the demonstration can assert
-- [ ] 7.3 Ask whether runtime deliveries carry `.tests/` today, and whether they will commit to it. A rule with nothing to test is an incomplete delivery (D7), but requiring it in `describeIncompleteSet` refuses payloads that are valid under the current contract, so it is theirs to agree before it is ours to enforce
-- [ ] 7.4 Once agreed, add fixtures to delivery completeness as its own change. Not here: this one must not turn a service omission into a refused write without warning
+- [ ] 7.3 Confirm runtime deliveries carry `.tests/`. Not a request: a runtime rule ships with fixtures, and a delivery without them is a defect on their side (D7). If any do not, file it as a bug rather than proposing it as a contract change
+- [ ] 7.4 Once 7.3 confirms deliveries carry them, add fixtures to delivery completeness as its own change. Sequenced only so a service defect does not reach users as a refused write with nothing they can do about it
 - [ ] 7.5 Archive the change