Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,37 @@ This happens when the **parent** PR is merged with `--delete-branch`: deleting t
- **No OpenSpec archive check at all.** An unarchived change directory is expected on a pull request AND on `main` while a forward-merging stack drains. Neither is a failure, and nothing reports it. Archive the change on the tip slice; a replacement that measures abandoned work rather than in-progress work is a separate piece of design.
- **Clean up local branches** once the stack lands: `git fetch --prune`, then delete the branches that merged (`git branch --merged main`).

## A spec delta REPLACES a requirement, it does not patch it

**A `## MODIFIED Requirements` block must restate the requirement in full, including every scenario you are not changing.** `openspec archive` rewrites the standing requirement to exactly what the delta contains. Anything you leave out is deleted from the spec, silently, with nothing reporting it.

Measured on `signature-coverage-statement`, whose proposal described itself as strictly additive and whose delta added one scenario while carrying one of the three that already existed:

```
BEFORE AFTER ARCHIVE
Envelope is emitted for algoVersion 1 -> Envelope is emitted for algoVersion 1
Version is read before parameters -> (gone)
Signatures compare as whole strings -> (gone)
A v1 signature covers the engine rule file
```

Two normative scenarios, about parameter-parsing order and whole-string comparison, would have left the spec as a side effect of documenting something unrelated.

**Do not rely on a preserve-on-sync guardrail.** `openspec-sync-specs` advises retaining content a delta does not mention, and that advice does not reach `openspec archive`, which is what actually runs. A delta that omits a scenario, and a comment saying the omission is deliberate, produce the same result: the scenario is gone.

**Check it the way it was found**, since nothing else will. `openspec validate --strict` passes on a delta that drops scenarios:

```bash
# commit everything first: the revert below is a hard reset
pnpm openspec archive <change-name> -y
grep -n "^#### Scenario" openspec/specs/<capability>/spec.md
git reset --hard HEAD && git clean -fd openspec/
```

Every scenario present before must still be present, plus whatever you added. Two cautions from doing it: commit first, and restore uncommitted edits by copying files **over** the change directory rather than copying the directory onto itself, which nests a stray copy inside it.

**The paranoia is warranted because the failure is invisible.** Nothing fails, no check reports it, and the requirement still reads coherently afterwards. It just no longer says the thing it used to say.

## OpenSpec Apply

When implementing changes via `/opsx:apply`, **pause after each task group** for user review before continuing. Commit between groups and wait for confirmation.
67 changes: 28 additions & 39 deletions openspec/changes/runtime-demo-path/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,48 +209,37 @@ reason over its own text for `unsupported`, so this needs nothing new.
A demo that invents a rule when generation fails would be worse than a demo
that fails, because it would look like success.

### D6 — The findings ship as `Finding[]`, grouped by the example that produced them

**This shape is under revision and is not yet agreed with the service.** Task
0.4 — telling the service what findings shape we want — is open, so what follows
is the ask rather than the contract. The retrieval shape the two sides have
settled is D1's, and it does not include this.

The service offered either its harness's `Finding[]` or something narrower, and
left the shape to us since we render it.

`Finding[]` as-is, because it is not the service's internal type: `Finding` is
declared in `types/runtime-rule.ts` as part of the runtime-rule contract, and is
what a check returns on this side. Choosing a narrower demo-only object would be
introducing a payload only the demo consumes, which is the thing D1 exists to
forbid. The same reasoning that reuses `submitRule`, `pollRuleStatus` and
`writeRuleFile` applies to the type a check's results already have.

What the demo does add is a wrapper, and the wrapper is the part carrying the
demonstration:

```
examples: [ { name, expectation: "fails" | "passes", findings: Finding[] } ]
```

A flat list of findings cannot distinguish a rule that catches the failing
examples from one that fires on everything it is shown, and those two render
identically as "3 findings". The second is the recurring defect in this area
wearing a success costume — an empty or indiscriminate scan reporting as a pass.
Attributing each finding to an example, and stating what that example was
expected to do, makes the asymmetry the thing a reader sees: findings on the
examples that should fail, none on the examples that should pass.

That also makes the demo checkable rather than merely viewable. The CLI can
assert the asymmetry instead of printing whatever arrives, so a demo that
silently stops finding anything fails rather than looking clean.
### D6 — There are no findings. Withdrawn, and the reason is worth keeping.

**Withdrawn.** The demonstration serves the rule alone; the retrieval response
is the delivery response and carries nothing beside it.

Three rounds went into this. The service offered its harness's `Finding[]`, we
worked out a wrapper attributing each finding to the fixture example that
produced it, and the service then withdrew the whole idea with the argument that
settles it: **the verification gate already refuses a rule whose asymmetry does
not hold.** A rule only reaches delivery having flagged the failing example and
ignored the passing one. So findings alongside the rule are a constant implied
by delivery — a field whose content a reader could infer from having received a
rule at all.

Our wrapper was the right fix to the wrong problem. It was aimed at the case
where a flat list renders identically for a rule that catches the bad examples
and one that fires on everything; that concern is real, and it is already
answered upstream of us rather than needing to be answered in the payload.

**What survives is the concern, relocated.** The demonstration's value is a rule
a reader can run, not a result they can read. That makes the local run the whole
demonstration — and `taskless test` currently reports a runtime rule as passing
when its fixtures never ran, which turns the demonstration back into a claim.
Fixing that is what D6 was reaching for, in the place it actually lives.

### D7 — Each thing the demo cannot do is the behaviour we want, not a shortfall

D3 and D6 describe the demo negatively: it cannot reach the gate, and it must
fail rather than render an indiscriminate rule. Read together they invite the
wrong conclusion, that the demo is a reduced version of something better. It is
not. Every limit is a correct behaviour arriving through the ordinary path.
D3 and D6 describe the demo negatively: it cannot reach the gate, and it serves
no result beside the rule. Read together they invite the wrong conclusion, that
the demo is a reduced version of something better. It is not. Every limit is a
correct behaviour arriving through the ordinary path.

**The rule is real and inspectable.** A developer gets `check.ts` and its
captures on disk and can read what a runtime rule actually is. That is the
Expand Down
44 changes: 15 additions & 29 deletions openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,39 +67,25 @@ drift without anything detecting it.
- **THEN** nothing SHALL be written for that rule
- **AND** the failure SHALL name what was wrong with the payload

### Requirement: The demo shows what the rule found, per example
### Requirement: The demo serves the rule and nothing beside it

The retrieval response SHALL carry, for each example in the service's fixture,
that example's name, whether it is expected to fail or pass, and the findings
the generated check produced against it. Findings SHALL use the published
`Finding` shape rather than a demo-only object.
The retrieval response SHALL be the ordinary delivery response. The CLI SHALL
NOT require, read, or render any field that exists only for the demo.

The CLI SHALL render findings grouped by example, and SHALL report the demo as
failed when the expected asymmetry does not hold: findings against an example
expected to pass, or no findings against an example expected to fail.
**Rationale.** The service's verification gate already refuses a rule whose
fixture asymmetry does not hold, so a rule that reaches delivery has necessarily
flagged the failing example and ignored the passing one. Results served
alongside it would be a constant implied by delivery rather than a finding, and
a demo-only field is the bespoke shape the well-known-format requirement exists
to forbid. What the demonstration offers is a rule a reader can run, not a
result they can read.

**Rationale.** A flat list of findings renders identically for a rule that
catches the failing examples and a rule that fires on everything it is shown.
The second is the failure this area keeps producing — an indiscriminate or empty
scan reporting as a pass — so the demonstration is the asymmetry, not the count.
Asserting it makes the demo a check rather than a picture.
#### Scenario: The demo consumes an ordinary delivery

#### Scenario: Findings are attributed to the example that produced them

- **WHEN** the demo renders its result
- **THEN** each finding SHALL be shown under the example it came from
- **AND** each example SHALL state whether it was expected to fail or pass

#### Scenario: A rule that fires on a passing example fails the demo

- **WHEN** the response carries a finding against an example expected to pass
- **THEN** the demo SHALL report failure rather than rendering the findings as
a successful demonstration

#### Scenario: A rule that finds nothing fails the demo

- **WHEN** the response carries no findings against an example expected to fail
- **THEN** the demo SHALL report failure
- **WHEN** the demo retrieves its rule
- **THEN** the response SHALL be handled by the same code that handles a
generated rule's delivery
- **AND** no demo-only field SHALL be required for the demo to succeed

### Requirement: The demo reports a terminal failure rather than inventing a rule

Expand Down
10 changes: 4 additions & 6 deletions openspec/changes/runtime-demo-path/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ what design D2 rules out. The mainline took the noun first; the demo follows it.

- [x] 0.1 Confirm the endpoint shapes with the generator team (**N9**): `POST /cli/api/demo/request` returning `{ requestId, status }`, `GET /cli/api/demo/request/{requestId}` returning `{ requestId, status, rules[] }` with the published file-set variant. Agreed both sides
- [x] 0.2 Confirm the service can serve the demo endpoints unauthenticated, and get their answer on whether reconcile can bless the sample signature for an anonymous caller. **Answered, both halves.** The endpoints take no authentication at all — public, like `rule-hash-vectors` — because nothing in the demo is scoped to a caller: no ticket, no corpus entry, no bill, and one Taskless-owned installation behind every request. And no blessing, for the reason D3 now records: blessing is recording, and a shared fixed rule is never recorded for the caller, so authenticating would not reach execution either
- [ ] 0.3 Agree what the demo rule is for — the scenario the generated rule addresses — so the demo shows something a person recognises rather than an arbitrary rule. The fixture now has a second job: its examples are what the served findings are attributed to, so it needs examples that are expected to fail AND examples that are expected to pass
- [ ] 0.4 Tell the service the findings shape we want (D6): `Finding[]` verbatim, grouped per fixture example with that example's name and whether it is expected to fail or pass. `Finding` is our published runtime-rule type, so this is the well-known format rather than a demo-only one
- [ ] 0.3 Agree what the demo rule is for — the scenario the generated rule addresses — so the demo shows something a person recognises rather than an arbitrary rule
- [x] 0.4 Tell the service the findings shape we want. **Void, not done**: the service withdrew the findings entirely (D6). The verification gate already refuses a rule whose asymmetry does not hold, so findings alongside the rule are a constant implied by delivery. There is no shape to agree
- [ ] 0.5 Ask the service which status `iterate` returns for a request outside the caller's organization: 404 `request_not_found` or 403 `access_denied`. It decides what `rule improve` against the demo rule reports — `iterateRule` maps the 404 to a `CLIError` carrying `RULE_NOT_FOUND` and the 403 to a plain `Error`, which `improveCommand` reports as `NETWORK_ERROR`, telling an agent to retry an id that will never resolve. The demo request exists under a Taskless-owned installation and is not the caller's, which is 403-shaped. D7 and the spec state the behaviour rather than a code until this is answered

**Nothing below starts until 0.0 and 0.1 land.** Building against a guessed shape is
Expand All @@ -37,8 +37,7 @@ what produced the seam this change is closing.
- [ ] 1.3 Write through `writeRuleFile`. If the demo needs its own writer, stop: the shapes have diverged and that is the finding
- [ ] 1.4 Report a terminal status by surfacing the service's reason, which `unsupportedMessage` already prefers over our own text
- [ ] 1.5 Regenerate `src/generated/api.d.ts` once the endpoints are published
- [ ] 1.6 Render the served findings grouped by example, showing each example's name, whether it was expected to fail or pass, and what the check returned against it
- [ ] 1.7 Assert the asymmetry rather than printing whatever arrives: findings on the examples expected to fail, none on the examples expected to pass. A demo whose rule has stopped finding anything, or that fires on everything, SHALL fail rather than render cleanly (D6)
- [ ] 1.6 Point the reader at running the rule, since that is the whole demonstration once nothing is served beside it. `taskless test` reporting a runtime rule as passing when its fixtures never ran turns the demonstration into a claim, so this task is blocked on that being fixed

## 2. The hidden topic

Expand All @@ -61,8 +60,7 @@ what produced the seam this change is closing.
- [ ] 4.1 Test that a delivered demo rule missing `check.ts` or its captures is refused and nothing is written
- [ ] 4.2 Test that a terminal `unsupported` surfaces the service's reason and writes nothing
- [ ] 4.3 Test the whole path against a mock serving both endpoints, asserting the rule lands and `verify` reports it valid
- [ ] 4.4 Test that a payload whose passing examples carry findings is reported as a failed demonstration, not rendered as success — the indiscriminate-rule case D6 names
- [ ] 4.5 Test that a payload whose failing examples carry no findings is likewise reported as failed, since an empty scan is the shape a silent regression takes
- [ ] 4.4 Test that the retrieval response is consumed as an ordinary delivery, with no demo-only field read from it and none required to be present

## 5. Prove it can be undone

Expand Down
2 changes: 2 additions & 0 deletions openspec/changes/signature-coverage-statement/.openspec.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-09-02
47 changes: 47 additions & 0 deletions openspec/changes/signature-coverage-statement/proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
## Why

The signature envelope is self-describing about **how** a signature was
computed and silent about **what** it covers. `cli-rule-reconciliation`'s
`Canonical rule signature envelope` requirement says the algoVersion determines
"the normalization procedure and hash algorithm" and stops there. Which file a
rule's signature is over is agreed between the two teams and written down in
neither specification.

That gap surfaced when the generator team proposed moving `signature` from the
rule onto each file, on the grounds that "nothing in the payload says which file
the signature covers". The observation was right; the remedy was not. The
binding is a property of the signature scheme rather than of any particular
delivery, so it belongs in the envelope's definition, not distributed across the
collection. The proposal was declined and this statement is what we owe instead.

Stating it also answers the question the proposal was really reaching for: what
happens when an engine needs to sign more than one file. A new algoVersion is
the mechanism that already exists for changing what a signature means, so a
multi-file signature is a v2 signature rather than a new payload shape.

## What Changes

**The envelope's version states its coverage, not only its computation.** A v1
signature covers exactly one file, the engine's `ruleFile` from the layout table
— `check.ts` for runtime. An engine that needs to sign more than one file does
so under a later algoVersion.

This is a documentation-level change to a requirement, not a behavioural one.
No signature changes, no payload changes, and nothing recomputes. The CLI
already signs exactly the rule file and already reads the algoVersion before
parsing parameters; this says so where a reader looks for it.

## Capabilities

### Modified Capabilities

- `cli-rule-reconciliation`: The canonical signature envelope requirement gains
the coverage statement and the versioning rule for changing it.

## Impact

- `openspec/specs/cli-rule-reconciliation/spec.md` — one requirement, amended.
- No source changes. `ALGO_VERSION` stays 1 and `signRuleFile` keeps signing the
rule file it already signs.
- Recorded with the generator team in the cross-team document (round thirteen),
where the per-file proposal was declined and this was offered in its place.
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
## MODIFIED Requirements

### Requirement: Canonical rule signature envelope

The CLI SHALL represent a rule file's canonical signature as a single self-describing
string of the form `<algoVersion>;h=<algo>;d=<digest>`. For algoVersion `1` this is
`1;h=sha-256;d=<hex>`, where `<hex>` is the digest as lowercase hexadecimal. The token
before the **first** `;` is the algoVersion and SHALL be read up to that one delimiter to
detect the version (and therefore the normalization procedure and hash algorithm) before
any `key=value` parameters are parsed. Signatures SHALL be compared as whole strings.

The algoVersion SHALL also determine **what the signature covers**. A signature at
algoVersion `1` covers exactly one file: the engine's `ruleFile` from the rule layout
table, which is `check.ts` for the runtime engine. An engine that requires more than one
file to be signed SHALL do so under a later algoVersion.

**Rationale.** Coverage is a property of the signature scheme, not of the delivery that
carries a signature. Stating it on the version keeps a payload from having to say which
of its files is the signed one, and gives a future multi-file scheme a mechanism that
already exists rather than a new payload shape. Leaving it unstated is what let both
teams hold the same binding as a private assumption.

#### Scenario: Envelope is emitted for algoVersion 1

- **WHEN** the CLI computes a signature for a rule file's bytes using algoVersion 1
- **THEN** the signature SHALL be the string `1;h=sha-256;d=<hex>` for that file's normalized bytes

#### Scenario: Version is read before parameters

- **WHEN** the CLI parses a signature string
- **THEN** it SHALL read the algoVersion as the substring before the first `;`
- **AND** SHALL NOT rely on the `key=value` parameter syntax to determine the version

#### Scenario: Signatures compare as whole strings

- **WHEN** the CLI compares two signatures for equality
- **THEN** it SHALL compare the full envelope strings, not the bare digests

#### Scenario: A v1 signature covers the engine's rule file

- **WHEN** a runtime rule carries a signature at algoVersion 1
- **THEN** that signature SHALL be over `check.ts` and over no other file in the rule directory
Comment thread
thecodedrift marked this conversation as resolved.
Loading
Loading