diff --git a/CLAUDE.md b/CLAUDE.md index d0e70f9a..f91ebe95 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -270,6 +270,37 @@ This happens when the **parent** PR is merged with `--delete-branch`: deleting t - **No OpenSpec archive check at all.** An unarchived change directory is expected on a pull request AND on `main` while a forward-merging stack drains. Neither is a failure, and nothing reports it. Archive the change on the tip slice; a replacement that measures abandoned work rather than in-progress work is a separate piece of design. - **Clean up local branches** once the stack lands: `git fetch --prune`, then delete the branches that merged (`git branch --merged main`). +## A spec delta REPLACES a requirement, it does not patch it + +**A `## MODIFIED Requirements` block must restate the requirement in full, including every scenario you are not changing.** `openspec archive` rewrites the standing requirement to exactly what the delta contains. Anything you leave out is deleted from the spec, silently, with nothing reporting it. + +Measured on `signature-coverage-statement`, whose proposal described itself as strictly additive and whose delta added one scenario while carrying one of the three that already existed: + +``` +BEFORE AFTER ARCHIVE +Envelope is emitted for algoVersion 1 -> Envelope is emitted for algoVersion 1 +Version is read before parameters -> (gone) +Signatures compare as whole strings -> (gone) + A v1 signature covers the engine rule file +``` + +Two normative scenarios, about parameter-parsing order and whole-string comparison, would have left the spec as a side effect of documenting something unrelated. + +**Do not rely on a preserve-on-sync guardrail.** `openspec-sync-specs` advises retaining content a delta does not mention, and that advice does not reach `openspec archive`, which is what actually runs. A delta that omits a scenario, and a comment saying the omission is deliberate, produce the same result: the scenario is gone. + +**Check it the way it was found**, since nothing else will. `openspec validate --strict` passes on a delta that drops scenarios: + +```bash +# commit everything first: the revert below is a hard reset +pnpm openspec archive -y +grep -n "^#### Scenario" openspec/specs//spec.md +git reset --hard HEAD && git clean -fd openspec/ +``` + +Every scenario present before must still be present, plus whatever you added. Two cautions from doing it: commit first, and restore uncommitted edits by copying files **over** the change directory rather than copying the directory onto itself, which nests a stray copy inside it. + +**The paranoia is warranted because the failure is invisible.** Nothing fails, no check reports it, and the requirement still reads coherently afterwards. It just no longer says the thing it used to say. + ## OpenSpec Apply When implementing changes via `/opsx:apply`, **pause after each task group** for user review before continuing. Commit between groups and wait for confirmation. diff --git a/openspec/changes/runtime-demo-path/design.md b/openspec/changes/runtime-demo-path/design.md index 5f7ec9f4..d40d7aea 100644 --- a/openspec/changes/runtime-demo-path/design.md +++ b/openspec/changes/runtime-demo-path/design.md @@ -209,48 +209,37 @@ reason over its own text for `unsupported`, so this needs nothing new. A demo that invents a rule when generation fails would be worse than a demo that fails, because it would look like success. -### D6 — The findings ship as `Finding[]`, grouped by the example that produced them - -**This shape is under revision and is not yet agreed with the service.** Task -0.4 — telling the service what findings shape we want — is open, so what follows -is the ask rather than the contract. The retrieval shape the two sides have -settled is D1's, and it does not include this. - -The service offered either its harness's `Finding[]` or something narrower, and -left the shape to us since we render it. - -`Finding[]` as-is, because it is not the service's internal type: `Finding` is -declared in `types/runtime-rule.ts` as part of the runtime-rule contract, and is -what a check returns on this side. Choosing a narrower demo-only object would be -introducing a payload only the demo consumes, which is the thing D1 exists to -forbid. The same reasoning that reuses `submitRule`, `pollRuleStatus` and -`writeRuleFile` applies to the type a check's results already have. - -What the demo does add is a wrapper, and the wrapper is the part carrying the -demonstration: - -``` -examples: [ { name, expectation: "fails" | "passes", findings: Finding[] } ] -``` - -A flat list of findings cannot distinguish a rule that catches the failing -examples from one that fires on everything it is shown, and those two render -identically as "3 findings". The second is the recurring defect in this area -wearing a success costume — an empty or indiscriminate scan reporting as a pass. -Attributing each finding to an example, and stating what that example was -expected to do, makes the asymmetry the thing a reader sees: findings on the -examples that should fail, none on the examples that should pass. - -That also makes the demo checkable rather than merely viewable. The CLI can -assert the asymmetry instead of printing whatever arrives, so a demo that -silently stops finding anything fails rather than looking clean. +### D6 — There are no findings. Withdrawn, and the reason is worth keeping. + +**Withdrawn.** The demonstration serves the rule alone; the retrieval response +is the delivery response and carries nothing beside it. + +Three rounds went into this. The service offered its harness's `Finding[]`, we +worked out a wrapper attributing each finding to the fixture example that +produced it, and the service then withdrew the whole idea with the argument that +settles it: **the verification gate already refuses a rule whose asymmetry does +not hold.** A rule only reaches delivery having flagged the failing example and +ignored the passing one. So findings alongside the rule are a constant implied +by delivery — a field whose content a reader could infer from having received a +rule at all. + +Our wrapper was the right fix to the wrong problem. It was aimed at the case +where a flat list renders identically for a rule that catches the bad examples +and one that fires on everything; that concern is real, and it is already +answered upstream of us rather than needing to be answered in the payload. + +**What survives is the concern, relocated.** The demonstration's value is a rule +a reader can run, not a result they can read. That makes the local run the whole +demonstration — and `taskless test` currently reports a runtime rule as passing +when its fixtures never ran, which turns the demonstration back into a claim. +Fixing that is what D6 was reaching for, in the place it actually lives. ### D7 — Each thing the demo cannot do is the behaviour we want, not a shortfall -D3 and D6 describe the demo negatively: it cannot reach the gate, and it must -fail rather than render an indiscriminate rule. Read together they invite the -wrong conclusion, that the demo is a reduced version of something better. It is -not. Every limit is a correct behaviour arriving through the ordinary path. +D3 and D6 describe the demo negatively: it cannot reach the gate, and it serves +no result beside the rule. Read together they invite the wrong conclusion, that +the demo is a reduced version of something better. It is not. Every limit is a +correct behaviour arriving through the ordinary path. **The rule is real and inspectable.** A developer gets `check.ts` and its captures on disk and can read what a runtime rule actually is. That is the diff --git a/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md b/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md index 7e9268b6..17671613 100644 --- a/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md +++ b/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md @@ -67,39 +67,25 @@ drift without anything detecting it. - **THEN** nothing SHALL be written for that rule - **AND** the failure SHALL name what was wrong with the payload -### Requirement: The demo shows what the rule found, per example +### Requirement: The demo serves the rule and nothing beside it -The retrieval response SHALL carry, for each example in the service's fixture, -that example's name, whether it is expected to fail or pass, and the findings -the generated check produced against it. Findings SHALL use the published -`Finding` shape rather than a demo-only object. +The retrieval response SHALL be the ordinary delivery response. The CLI SHALL +NOT require, read, or render any field that exists only for the demo. -The CLI SHALL render findings grouped by example, and SHALL report the demo as -failed when the expected asymmetry does not hold: findings against an example -expected to pass, or no findings against an example expected to fail. +**Rationale.** The service's verification gate already refuses a rule whose +fixture asymmetry does not hold, so a rule that reaches delivery has necessarily +flagged the failing example and ignored the passing one. Results served +alongside it would be a constant implied by delivery rather than a finding, and +a demo-only field is the bespoke shape the well-known-format requirement exists +to forbid. What the demonstration offers is a rule a reader can run, not a +result they can read. -**Rationale.** A flat list of findings renders identically for a rule that -catches the failing examples and a rule that fires on everything it is shown. -The second is the failure this area keeps producing — an indiscriminate or empty -scan reporting as a pass — so the demonstration is the asymmetry, not the count. -Asserting it makes the demo a check rather than a picture. +#### Scenario: The demo consumes an ordinary delivery -#### Scenario: Findings are attributed to the example that produced them - -- **WHEN** the demo renders its result -- **THEN** each finding SHALL be shown under the example it came from -- **AND** each example SHALL state whether it was expected to fail or pass - -#### Scenario: A rule that fires on a passing example fails the demo - -- **WHEN** the response carries a finding against an example expected to pass -- **THEN** the demo SHALL report failure rather than rendering the findings as - a successful demonstration - -#### Scenario: A rule that finds nothing fails the demo - -- **WHEN** the response carries no findings against an example expected to fail -- **THEN** the demo SHALL report failure +- **WHEN** the demo retrieves its rule +- **THEN** the response SHALL be handled by the same code that handles a + generated rule's delivery +- **AND** no demo-only field SHALL be required for the demo to succeed ### Requirement: The demo reports a terminal failure rather than inventing a rule diff --git a/openspec/changes/runtime-demo-path/tasks.md b/openspec/changes/runtime-demo-path/tasks.md index 9350aa3b..c201cf82 100644 --- a/openspec/changes/runtime-demo-path/tasks.md +++ b/openspec/changes/runtime-demo-path/tasks.md @@ -21,8 +21,8 @@ what design D2 rules out. The mainline took the noun first; the demo follows it. - [x] 0.1 Confirm the endpoint shapes with the generator team (**N9**): `POST /cli/api/demo/request` returning `{ requestId, status }`, `GET /cli/api/demo/request/{requestId}` returning `{ requestId, status, rules[] }` with the published file-set variant. Agreed both sides - [x] 0.2 Confirm the service can serve the demo endpoints unauthenticated, and get their answer on whether reconcile can bless the sample signature for an anonymous caller. **Answered, both halves.** The endpoints take no authentication at all — public, like `rule-hash-vectors` — because nothing in the demo is scoped to a caller: no ticket, no corpus entry, no bill, and one Taskless-owned installation behind every request. And no blessing, for the reason D3 now records: blessing is recording, and a shared fixed rule is never recorded for the caller, so authenticating would not reach execution either -- [ ] 0.3 Agree what the demo rule is for — the scenario the generated rule addresses — so the demo shows something a person recognises rather than an arbitrary rule. The fixture now has a second job: its examples are what the served findings are attributed to, so it needs examples that are expected to fail AND examples that are expected to pass -- [ ] 0.4 Tell the service the findings shape we want (D6): `Finding[]` verbatim, grouped per fixture example with that example's name and whether it is expected to fail or pass. `Finding` is our published runtime-rule type, so this is the well-known format rather than a demo-only one +- [ ] 0.3 Agree what the demo rule is for — the scenario the generated rule addresses — so the demo shows something a person recognises rather than an arbitrary rule +- [x] 0.4 Tell the service the findings shape we want. **Void, not done**: the service withdrew the findings entirely (D6). The verification gate already refuses a rule whose asymmetry does not hold, so findings alongside the rule are a constant implied by delivery. There is no shape to agree - [ ] 0.5 Ask the service which status `iterate` returns for a request outside the caller's organization: 404 `request_not_found` or 403 `access_denied`. It decides what `rule improve` against the demo rule reports — `iterateRule` maps the 404 to a `CLIError` carrying `RULE_NOT_FOUND` and the 403 to a plain `Error`, which `improveCommand` reports as `NETWORK_ERROR`, telling an agent to retry an id that will never resolve. The demo request exists under a Taskless-owned installation and is not the caller's, which is 403-shaped. D7 and the spec state the behaviour rather than a code until this is answered **Nothing below starts until 0.0 and 0.1 land.** Building against a guessed shape is @@ -37,8 +37,7 @@ what produced the seam this change is closing. - [ ] 1.3 Write through `writeRuleFile`. If the demo needs its own writer, stop: the shapes have diverged and that is the finding - [ ] 1.4 Report a terminal status by surfacing the service's reason, which `unsupportedMessage` already prefers over our own text - [ ] 1.5 Regenerate `src/generated/api.d.ts` once the endpoints are published -- [ ] 1.6 Render the served findings grouped by example, showing each example's name, whether it was expected to fail or pass, and what the check returned against it -- [ ] 1.7 Assert the asymmetry rather than printing whatever arrives: findings on the examples expected to fail, none on the examples expected to pass. A demo whose rule has stopped finding anything, or that fires on everything, SHALL fail rather than render cleanly (D6) +- [ ] 1.6 Point the reader at running the rule, since that is the whole demonstration once nothing is served beside it. `taskless test` reporting a runtime rule as passing when its fixtures never ran turns the demonstration into a claim, so this task is blocked on that being fixed ## 2. The hidden topic @@ -61,8 +60,7 @@ what produced the seam this change is closing. - [ ] 4.1 Test that a delivered demo rule missing `check.ts` or its captures is refused and nothing is written - [ ] 4.2 Test that a terminal `unsupported` surfaces the service's reason and writes nothing - [ ] 4.3 Test the whole path against a mock serving both endpoints, asserting the rule lands and `verify` reports it valid -- [ ] 4.4 Test that a payload whose passing examples carry findings is reported as a failed demonstration, not rendered as success — the indiscriminate-rule case D6 names -- [ ] 4.5 Test that a payload whose failing examples carry no findings is likewise reported as failed, since an empty scan is the shape a silent regression takes +- [ ] 4.4 Test that the retrieval response is consumed as an ordinary delivery, with no demo-only field read from it and none required to be present ## 5. Prove it can be undone diff --git a/openspec/changes/signature-coverage-statement/.openspec.yaml b/openspec/changes/signature-coverage-statement/.openspec.yaml new file mode 100644 index 00000000..032461ff --- /dev/null +++ b/openspec/changes/signature-coverage-statement/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-09-02 diff --git a/openspec/changes/signature-coverage-statement/proposal.md b/openspec/changes/signature-coverage-statement/proposal.md new file mode 100644 index 00000000..fc474c19 --- /dev/null +++ b/openspec/changes/signature-coverage-statement/proposal.md @@ -0,0 +1,47 @@ +## Why + +The signature envelope is self-describing about **how** a signature was +computed and silent about **what** it covers. `cli-rule-reconciliation`'s +`Canonical rule signature envelope` requirement says the algoVersion determines +"the normalization procedure and hash algorithm" and stops there. Which file a +rule's signature is over is agreed between the two teams and written down in +neither specification. + +That gap surfaced when the generator team proposed moving `signature` from the +rule onto each file, on the grounds that "nothing in the payload says which file +the signature covers". The observation was right; the remedy was not. The +binding is a property of the signature scheme rather than of any particular +delivery, so it belongs in the envelope's definition, not distributed across the +collection. The proposal was declined and this statement is what we owe instead. + +Stating it also answers the question the proposal was really reaching for: what +happens when an engine needs to sign more than one file. A new algoVersion is +the mechanism that already exists for changing what a signature means, so a +multi-file signature is a v2 signature rather than a new payload shape. + +## What Changes + +**The envelope's version states its coverage, not only its computation.** A v1 +signature covers exactly one file, the engine's `ruleFile` from the layout table +— `check.ts` for runtime. An engine that needs to sign more than one file does +so under a later algoVersion. + +This is a documentation-level change to a requirement, not a behavioural one. +No signature changes, no payload changes, and nothing recomputes. The CLI +already signs exactly the rule file and already reads the algoVersion before +parsing parameters; this says so where a reader looks for it. + +## Capabilities + +### Modified Capabilities + +- `cli-rule-reconciliation`: The canonical signature envelope requirement gains + the coverage statement and the versioning rule for changing it. + +## Impact + +- `openspec/specs/cli-rule-reconciliation/spec.md` — one requirement, amended. +- No source changes. `ALGO_VERSION` stays 1 and `signRuleFile` keeps signing the + rule file it already signs. +- Recorded with the generator team in the cross-team document (round thirteen), + where the per-file proposal was declined and this was offered in its place. diff --git a/openspec/changes/signature-coverage-statement/specs/cli-rule-reconciliation/spec.md b/openspec/changes/signature-coverage-statement/specs/cli-rule-reconciliation/spec.md new file mode 100644 index 00000000..20b8c211 --- /dev/null +++ b/openspec/changes/signature-coverage-statement/specs/cli-rule-reconciliation/spec.md @@ -0,0 +1,42 @@ +## MODIFIED Requirements + +### Requirement: Canonical rule signature envelope + +The CLI SHALL represent a rule file's canonical signature as a single self-describing +string of the form `;h=;d=`. For algoVersion `1` this is +`1;h=sha-256;d=`, where `` is the digest as lowercase hexadecimal. The token +before the **first** `;` is the algoVersion and SHALL be read up to that one delimiter to +detect the version (and therefore the normalization procedure and hash algorithm) before +any `key=value` parameters are parsed. Signatures SHALL be compared as whole strings. + +The algoVersion SHALL also determine **what the signature covers**. A signature at +algoVersion `1` covers exactly one file: the engine's `ruleFile` from the rule layout +table, which is `check.ts` for the runtime engine. An engine that requires more than one +file to be signed SHALL do so under a later algoVersion. + +**Rationale.** Coverage is a property of the signature scheme, not of the delivery that +carries a signature. Stating it on the version keeps a payload from having to say which +of its files is the signed one, and gives a future multi-file scheme a mechanism that +already exists rather than a new payload shape. Leaving it unstated is what let both +teams hold the same binding as a private assumption. + +#### Scenario: Envelope is emitted for algoVersion 1 + +- **WHEN** the CLI computes a signature for a rule file's bytes using algoVersion 1 +- **THEN** the signature SHALL be the string `1;h=sha-256;d=` for that file's normalized bytes + +#### Scenario: Version is read before parameters + +- **WHEN** the CLI parses a signature string +- **THEN** it SHALL read the algoVersion as the substring before the first `;` +- **AND** SHALL NOT rely on the `key=value` parameter syntax to determine the version + +#### Scenario: Signatures compare as whole strings + +- **WHEN** the CLI compares two signatures for equality +- **THEN** it SHALL compare the full envelope strings, not the bare digests + +#### Scenario: A v1 signature covers the engine's rule file + +- **WHEN** a runtime rule carries a signature at algoVersion 1 +- **THEN** that signature SHALL be over `check.ts` and over no other file in the rule directory diff --git a/openspec/changes/signature-coverage-statement/tasks.md b/openspec/changes/signature-coverage-statement/tasks.md new file mode 100644 index 00000000..ff1f0433 --- /dev/null +++ b/openspec/changes/signature-coverage-statement/tasks.md @@ -0,0 +1,20 @@ +# Tasks + +One PR. A single requirement gains two sentences. + +## 1. State it + +- [x] 1.1 Amend `Canonical rule signature envelope` so the algoVersion governs coverage as well as normalization and algorithm +- [x] 1.2 Say that a v1 signature covers exactly the engine's `ruleFile`, naming `check.ts` for runtime as the instance rather than as the rule +- [x] 1.3 Say that signing more than one file requires a later algoVersion, so the mechanism for changing coverage is the one that already exists + +## 2. Keep it honest + +- [x] 2.1 Confirm no source change is implied. **Verified**: `ALGO_VERSION` is 1 (`rule-hash.ts:14`), `signRuleFile` reads one path and hashes it (`rule-hash.ts:116`), `run-set.ts:41` signs `rule.checkFile` once per rule, and `ENGINE_LAYOUTS` gives every engine exactly one `ruleFile` (`.yml` for sg and vale, `check.ts` for runtime) so the statement generalises rather than special-casing runtime +- [x] 2.2 Confirm the statement matches what the service does. **Confirmed by the generator team**, who declined-and-accepted in the same round: "a v1 signature covers the engine's ruleFile, multi-file signing would be v2" is recorded in their half of the cross-team document as settled, not proposed + +- [x] 2.3 Carry every existing scenario into the MODIFIED delta. **A delta is a replacement, not a patch**: measured by archiving this change and diffing the result, `openspec archive` rewrites the requirement to exactly what the delta contains, so `Version is read before parameters` and `Signatures compare as whole strings` were dropped from the standing spec until they were restated here + +## 3. Close out + +- [ ] 3.1 Archive the change