From 15954dd20dc79718526197097345c6f957a9bd9d Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Tue, 1 Sep 2026 11:44:00 -0700 Subject: [PATCH 1/8] docs(openspec): propose a runtime demo path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The runtime tier cannot be demonstrated or smoke-tested end to end. Everything we can exercise alone covers the pieces; nothing covers the two things that exist only between this client and the service, which are generation producing a runtime rule and delivery handing it over in a shape we accept. Every defect the alignment work surfaced lived in that seam and was found by someone noticing. The shape, as asked of the generator team in N9: two endpoints mirroring the mainline, a ticket then a retrieval, returning the published file-set variant. Mirroring is the decision rather than a convenience — a bespoke demo payload would exercise a path no user is on, which is the one thing a demo must not do, and reusing the well-known formats means the demo covers polling and the terminal states because it is the same client code reaching them. Two things the design writes down because they are the tempting ones to get wrong. The demo stops at `verify`, and says so. A runtime rule executes only when an authenticated reconcile returns its signature, and the demo is unauthenticated, so it reaches a complete verified rule on disk and no further. A client-side bypass keyed on the demo rule id would show the real thing and is refused: the signature gate is the only thing between a payload fetched over the network and arbitrary code running on a developer's machine, and the id it would key on is visible to anyone who reads the recipe. The spec carries that as a requirement rather than a comment. Hiding the topic needs no mechanism. `RECIPE_TOPICS` is a hand-maintained literal and lookup is by filename, so a recipe absent from the list is already fetchable by name and invisible to the index. `cli-agent` gains a requirement saying that is intended, so nobody later reads it as an oversight and closes it. Tasks are blocked on the endpoints existing. Building against a guessed shape is what produced the seam this change is closing. --- .../changes/runtime-demo-path/.openspec.yaml | 2 + openspec/changes/runtime-demo-path/design.md | 136 ++++++++++++++++++ .../changes/runtime-demo-path/proposal.md | 86 +++++++++++ .../runtime-demo-path/specs/cli-agent/spec.md | 24 ++++ .../specs/cli-runtime-demo/spec.md | 98 +++++++++++++ openspec/changes/runtime-demo-path/tasks.md | 49 +++++++ 6 files changed, 395 insertions(+) create mode 100644 openspec/changes/runtime-demo-path/.openspec.yaml create mode 100644 openspec/changes/runtime-demo-path/design.md create mode 100644 openspec/changes/runtime-demo-path/proposal.md create mode 100644 openspec/changes/runtime-demo-path/specs/cli-agent/spec.md create mode 100644 openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md create mode 100644 openspec/changes/runtime-demo-path/tasks.md diff --git a/openspec/changes/runtime-demo-path/.openspec.yaml b/openspec/changes/runtime-demo-path/.openspec.yaml new file mode 100644 index 00000000..b4b3ece7 --- /dev/null +++ b/openspec/changes/runtime-demo-path/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-09-01 diff --git a/openspec/changes/runtime-demo-path/design.md b/openspec/changes/runtime-demo-path/design.md new file mode 100644 index 00000000..197d0de4 --- /dev/null +++ b/openspec/changes/runtime-demo-path/design.md @@ -0,0 +1,136 @@ +## Context + +The runtime tier has three parts, and only two of them can be tested from one +repository. Discovery, signing, validation and writing are ours. Generation is +the service's. The handshake between them — a signature the service blessed +over bytes this client then wrote — belongs to neither and is tested by +neither. + +The payload alignment work made that concrete. Every defect it surfaced lived +in the seam and was found by a person noticing: a runtime rule below the +file-set floor admitted rather than withheld, a signature attached to the wrong +rule, a capture dropped for want of a name, a rule written and never executed. +An empty scan reports success, so the symptom was almost always "no findings" +rather than an error. + +`check` is the wrong place to close that gap. It reports on a user's project, +and the whole of the last change was about it not doing anything else. + +## Goals / Non-Goals + +**Goals:** + +- One deliberately-invoked command that walks generation, delivery, writing and + verification, and says what happened at each step. +- Run on the paths a user runs on, so what it proves is what users get. +- Work with no login, so it can be shown to someone who does not have an + account yet. +- Leave the project recoverable: whatever it writes can be removed. + +**Non-Goals:** + +- Not a correctness test of the generated rule. The rule may be poor; that is + acceptable and `rule delete` removes it. +- Not a replacement for reconcile, and not a second way to get rules. +- Not reachable by routing. An agent deciding how to author a rule must never + land here. +- Not a demonstration of execution. See D3. + +## Decisions + +### D1 — The demo uses the well-known ticket and retrieval formats + +`POST /cli/api/demo/rule` returning `{ ruleId, status }`, then +`GET /cli/api/demo/rule/{ruleId}` returning `{ ruleId, status, rules[] }`. The +same two-stage flow as `rule create`, the same status enum, the same file-set +variant. + +The alternative — one endpoint returning a rule directly — is smaller and +wrong. A demo exists to show the real path, so a payload shape that only the +demo uses would demonstrate something no user is on. Mirroring also means the +demo covers polling and the terminal `failed` and `unsupported` states for +free, because it is the same client code reaching them. + +Concretely: `submitRule`, `pollRuleStatus` and `writeRuleFile` are reused, not +reimplemented. If the demo needs a new writer, that is evidence the shapes have +diverged, and the demo has done its job by failing. + +### D2 — The demo tracks the mainline's naming, whatever it is + +The paths above mirror today's `/cli/api/rule` naming, including its +inconsistency (**N10**: `ruleId` names a ticket, by the service's own field +description). If the mainline is renamed, these are renamed with it in the same +change. + +The demo must never be where a second convention lives. Its value is being +indistinguishable from the path users take; a demo that is tidier than +production is a demo that stops proving anything. + +### D3 — It stops at `verify`, and that is stated rather than worked around + +A runtime rule executes only when an authenticated reconcile returns its +signature in `run`. The demo is unauthenticated, so it cannot reach execution: +`check` skips the rule and reports "not authenticated — runtime rules were not +verified and did not run." + +Three options existed. Stopping at `verify` is honest and shows generation, +delivery, the file set, and validation. `--dangerously-run-scripts` reaches +execution and prints the warning it always prints, which is truthful and +conspicuously not the signature story. A client-side bypass keyed on the demo +rule id would show the real thing and is refused. + +That third option is the one worth writing down, because it is the tempting +one. The signature gate is the only thing between a payload downloaded over the +network and arbitrary code running on a developer's machine. A hole in it that +exists "only for the demo" is a hole, and the id it keys on is attacker-visible +in the recipe. No demo is worth that. + +The consequence is that this change demonstrates everything up to the gate. If +showing a blessed rule execute matters, logging in is the next beat, and it is +a better demo for being the payoff rather than the prerequisite. + +### D4 — Hidden by omission, not by a new mechanism + +`RECIPE_TOPICS` in `commands/agent.ts` is a hand-maintained literal; `getRecipe` +looks up by filename. A recipe file that is not in the list is therefore +fetchable by name and absent from the index already. + +Adding a hiding mechanism — a naming convention the loader understands, a +metadata field — would be new code to serve a property the existing design +already has. The `__` prefix in the topic name is for human readers, not for +the loader. + +The cost is that the index is curated by hand and can drift from the files on +disk. That is already true, and `cli-agent` gains a requirement saying it is +intended rather than an oversight. + +### D5 — Failure is reported, never fabricated + +If generation cannot be served, the service answers with a terminal status and +a reason, and the demo prints it. The client already prefers the service's +reason over its own text for `unsupported`, so this needs nothing new. + +A demo that invents a rule when generation fails would be worse than a demo +that fails, because it would look like success. + +## Risks / Trade-offs + +**The service half does not exist yet.** The endpoints are the service team's +to build, raised as N9. This change assumes the shapes in D1; if they land +differently, the client work is the adapter and the tests move with it. Nothing +here is worth building against a guess, so implementation waits on their +answer. + +**An unauthenticated generation endpoint is abuse surface.** Fixed input and a +regeneration window make it cheap to serve repeatedly, but the exposure is real +and it is the service's to bound. Worth naming here because the client asked +for the no-auth property and therefore owns half the reason it exists. + +**A demo that stops before execution may underwhelm.** Accepted, and preferable +to the alternative. The step it stops at is a signature gate, and explaining +why it stops is a better demonstration of the product than bypassing it would +be. + +**A hidden topic is discoverable by anyone who reads the source.** It is not a +secret and does not need to be. It is out of the index so routing cannot reach +it, not so that people cannot find it. diff --git a/openspec/changes/runtime-demo-path/proposal.md b/openspec/changes/runtime-demo-path/proposal.md new file mode 100644 index 00000000..a5ed668d --- /dev/null +++ b/openspec/changes/runtime-demo-path/proposal.md @@ -0,0 +1,86 @@ +## Why + +The runtime tier cannot be demonstrated, and it cannot be smoke-tested end to +end. Everything we can exercise alone covers the pieces — discovery, signing, +the file-set writer, delivery validation, the repair path — and none of it +covers the two things that only exist between the client and the service: +generation producing a runtime rule, and delivery handing it over in a shape +this client accepts. + +That gap is why the payload alignment work took the shape it did. Both sides +held the same assumptions separately, and every defect the exchange surfaced +lived in the seam: two floors that could disagree, a signature attached to the +wrong rule, a capture silently dropped, a rule written and never run. Each was +found by hand, by someone noticing. There is still no single command that walks +the whole path and shows what happened. + +A demo path closes that. Run one command, get a real generated runtime rule +written to disk, and see it verified. It is a live demo for a person, and the +same command is the closest thing we can have to an integration test across the +repository boundary. + +**Delivery shape: single PR.** The client half is one recipe, one command, and +their tests. Splitting it would separate a command from the recipe that names +it, which is the seam this change exists to remove. + +## What Changes + +**A hidden agent topic, `__undocumented-sample-runtime`.** Absent from the +`agent` index on purpose: `RECIPE_TOPICS` in `commands/agent.ts` is a +hand-maintained literal and lookup is by filename, so a topic that is not in +the list is fetchable by name and invisible otherwise. No new mechanism. This +is something we run deliberately, not something routing sends an agent to. + +**A command that walks the path.** Ticket, poll, retrieve, write, verify — +through the existing `submitRule` / `pollRuleStatus` client code and the +existing `writeRuleFile`, against demo endpoints that mirror the mainline +shapes: + +``` +POST /cli/api/demo/rule -> { ruleId, status } +GET /cli/api/demo/rule/{ruleId} -> { ruleId, status, rules[] } +``` + +Mirroring is the point. A bespoke demo payload would exercise a path no user is +on, which is the one thing a demo must not do. Reusing the well-known formats +means the demo also covers polling and the terminal `failed` / `unsupported` +states rather than a happy path built for the occasion. + +**The demo stops where the gate does, and says so.** A runtime rule executes +only when an authenticated reconcile returns its signature in `run`. The demo +is unauthenticated, so it reaches a complete, well-formed, verified rule on +disk and no further. `check` already explains that state: "not authenticated — +runtime rules were not verified and did not run." + +**No client-side bypass.** Not for a known demo id, not behind a flag of its +own. The signature gate is what stands between a downloaded payload and +arbitrary code execution on a developer's machine, and a demo is not worth a +hole in it. Executing the demo rule uses the documented +`--dangerously-run-scripts`, which prints the warning it always prints. + +## Capabilities + +### New Capabilities + +- `cli-runtime-demo`: A deliberately-invoked path that generates, delivers, + writes and verifies one runtime rule against a service-controlled repository, + so the client-service seam can be shown working rather than described. + +### Modified Capabilities + +- `cli-agent`: The topic index is a curated list rather than every embedded + recipe. A topic may exist and be fetchable by name while staying out of the + index, which is already how the code behaves and is not yet stated as a + requirement. + +## Impact + +- `packages/cli/src/agent/__undocumented-sample-runtime.txt` — new recipe. +- `packages/cli/src/commands/` — the demo command, reusing `submitRule`, + `pollRuleStatus` and `writeRuleFile` rather than duplicating them. +- `packages/cli/src/api/` — the demo endpoints, alongside the existing client. +- Depends on the service half (raised as **N9** in the cross-team document). + The endpoints are theirs to build; the shapes above are what we asked for and + what this change assumes. +- No change to the runtime gate, to reconcile, or to `check`'s execution + policy. diff --git a/openspec/changes/runtime-demo-path/specs/cli-agent/spec.md b/openspec/changes/runtime-demo-path/specs/cli-agent/spec.md new file mode 100644 index 00000000..c425324d --- /dev/null +++ b/openspec/changes/runtime-demo-path/specs/cli-agent/spec.md @@ -0,0 +1,24 @@ +## ADDED Requirements + +### Requirement: The topic index is curated, and a topic may be absent from it + +The `agent` index SHALL list a curated set of topics rather than every embedded +recipe. A recipe MAY exist and be retrievable by name while being absent from +that index. + +**Rationale.** Some topics are invoked deliberately rather than discovered — a +demonstration path, for instance, which routing must never send an agent to. +The index is a hand-maintained list and lookup is by name, so absence from the +index is the existing behaviour; this states it as intended rather than leaving +it to be read as an oversight and "fixed". + +#### Scenario: An unlisted topic is still retrievable by name + +- **WHEN** an agent requests a topic that exists as a recipe but is not in the + index +- **THEN** the CLI SHALL return that recipe + +#### Scenario: An unlisted topic does not appear in the index + +- **WHEN** the topic index is rendered +- **THEN** a topic that is not in the curated list SHALL NOT appear in it diff --git a/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md b/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md new file mode 100644 index 00000000..ba55cdce --- /dev/null +++ b/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md @@ -0,0 +1,98 @@ +## ADDED Requirements + +### Requirement: A deliberately-invoked demo path + +The CLI SHALL provide a path that generates one runtime rule against a +service-controlled repository, writes it, and verifies it, so the boundary +between this client and the rule service can be shown working rather than +described. + +It SHALL be invoked deliberately. Routing SHALL NOT reach it, and no recipe +that decides how to author a rule SHALL name it as a destination. + +#### Scenario: The demo produces a complete runtime rule + +- **WHEN** the demo path is invoked in a project +- **THEN** a runtime rule directory SHALL exist containing `check.ts` and at + least one capture under `captures/` +- **AND** `verify` SHALL report that rule as valid + +#### Scenario: Routing never sends an agent to the demo + +- **WHEN** an agent fetches any authoring or routing recipe +- **THEN** the demo topic SHALL NOT appear as a destination in that recipe + +### Requirement: The demo uses the well-known request and retrieval formats + +The demo SHALL obtain its rule through the same two-stage flow as an ordinary +generated rule: a request that returns a ticket and a status, then a retrieval +by that ticket returning the status and the delivered rules. + +The retrieval response SHALL use the published file-set variant. The demo SHALL +NOT introduce a payload shape that only the demo consumes. + +**Rationale.** A demo exists to show the path a user is on. A bespoke shape +would demonstrate something no user receives, and would let the two shapes +drift without anything detecting it. + +#### Scenario: The demo reuses the ordinary delivery writer + +- **WHEN** the demo receives its rule +- **THEN** the rule SHALL be written by the same writer that writes a generated + rule, subject to the same path, completeness and layout validation + +#### Scenario: A malformed demo payload is refused, not written + +- **WHEN** the demo receives a rule that fails delivery validation +- **THEN** nothing SHALL be written for that rule +- **AND** the failure SHALL name what was wrong with the payload + +### Requirement: The demo reports a terminal failure rather than inventing a rule + +The CLI SHALL report a terminal status the service returns for a demo request, +together with the reason the service sent, and SHALL write nothing for that +request. + +**Rationale.** A demo that invents a rule when generation fails is worse than a +demo that fails, because it looks like success. + +#### Scenario: Generation cannot be served + +- **WHEN** the demo request terminates as unsupported or failed +- **THEN** the CLI SHALL surface the service's reason +- **AND** SHALL NOT write a rule + +### Requirement: The demo does not weaken the runtime execution gate + +The demo SHALL NOT cause a runtime rule to execute without the verification +`check` already requires. No demo rule identifier, and no flag introduced for +the demo, SHALL bypass signature verification. + +**Rationale.** Signature verification is what stands between a payload fetched +over the network and arbitrary code executing on a developer's machine. An +exception that exists for a demo is an exception, and the identifier it keys on +is visible to anyone who reads the recipe. + +#### Scenario: A demo rule is not executed by an unauthenticated check + +- **WHEN** `check` runs with no authentication in a project holding a demo rule +- **THEN** the demo rule SHALL be skipped with the reason `check` already gives + for an unverified runtime rule + +#### Scenario: Executing a demo rule uses the documented escape + +- **WHEN** a user runs `check` with `--dangerously-run-scripts` +- **THEN** the demo rule SHALL execute under that flag's existing warning, and + under no other mechanism + +### Requirement: What the demo writes can be removed + +A demo rule that has been written SHALL be removable by rule identifier through +the ordinary deletion path, leaving no residue that `verify` or `check` would +later report. + +#### Scenario: Deleting the demo rule + +- **WHEN** the user deletes the demo rule by its identifier +- **THEN** its rule directory SHALL be removed +- **AND** a subsequent `check` SHALL NOT report it diff --git a/openspec/changes/runtime-demo-path/tasks.md b/openspec/changes/runtime-demo-path/tasks.md new file mode 100644 index 00000000..9d2bf56d --- /dev/null +++ b/openspec/changes/runtime-demo-path/tasks.md @@ -0,0 +1,49 @@ +# Tasks + +One PR. The command and the recipe that names it are the same seam this change +exists to remove, so they land together. + +## 0. Blocked on the service half + +- [ ] 0.1 Confirm the endpoint shapes with the generator team (**N9**): `POST /cli/api/demo/rule` returning `{ ruleId, status }`, `GET /cli/api/demo/rule/{ruleId}` returning `{ ruleId, status, rules[] }` with the published file-set variant +- [ ] 0.2 Confirm the demo is servable unauthenticated, and get their answer on whether reconcile can bless the sample signature for an anonymous caller (design D3). A "no" is a complete answer and changes nothing below +- [ ] 0.3 Agree what the demo rule is for — the scenario the generated rule addresses — so the demo shows something a person recognises rather than an arbitrary rule + +**Nothing below starts until 0.1 lands.** Building against a guessed shape is +what produced the seam this change is closing. + +## 1. The client path + +- [ ] 1.1 Add the demo endpoints to `src/api/`, reusing the existing client rather than a second fetch layer +- [ ] 1.2 Drive them through `submitRule`/`pollRuleStatus`, so polling and the terminal `failed`/`unsupported` states are the same code an ordinary generation uses +- [ ] 1.3 Write through `writeRuleFile`. If the demo needs its own writer, stop: the shapes have diverged and that is the finding +- [ ] 1.4 Report a terminal status by surfacing the service's reason, which `unsupportedMessage` already prefers over our own text +- [ ] 1.5 Regenerate `src/generated/api.d.ts` once the endpoints are published + +## 2. The hidden topic + +- [ ] 2.1 Add `src/agent/__undocumented-sample-runtime.txt`, absent from `RECIPE_TOPICS` +- [ ] 2.2 The recipe names the command, says the rule may be imperfect, and says how to delete it +- [ ] 2.3 The recipe states where the demo stops and why, so a reader meets the signature gate as a design rather than as a failure +- [ ] 2.4 Test that the topic is fetchable by name and absent from the index + +## 3. Prove it does not weaken the gate + +- [ ] 3.1 Test that an unauthenticated `check` skips a demo rule, with the reason it already gives +- [ ] 3.2 Test that the demo rule executes under `--dangerously-run-scripts` and under nothing else +- [ ] 3.3 Test that no demo rule identifier is special-cased anywhere on the execution path + +## 4. Prove it does not write a broken rule + +- [ ] 4.1 Test that a delivered demo rule missing `check.ts` or its captures is refused and nothing is written +- [ ] 4.2 Test that a terminal `unsupported` surfaces the service's reason and writes nothing +- [ ] 4.3 Test the whole path against a mock serving both endpoints, asserting the rule lands and `verify` reports it valid + +## 5. Prove it can be undone + +- [ ] 5.1 Test that deleting the demo rule by id removes its directory and that a later `check` does not report it + +## 6. Close out + +- [ ] 6.1 Tell the generator team the release that ships this +- [ ] 6.2 Archive the change From c02a372e0d0cbbdc5d25c18085dc45f78ee4e362 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Tue, 1 Sep 2026 12:03:11 -0700 Subject: [PATCH 2/8] docs(openspec): sequence the demo behind the request rename The generator team is renaming the request resource to `request`/`requestId` as its own change, ahead of this one. Their `openspec/specs/cli/spec.md` already specified that noun, and their live `meta` block already carries `ticketId` per delivered rule, so the route was the outlier rather than the convention. D2 said the demo must never be where a second convention lives. That was a principle; the rename makes it a sequencing constraint. Naming these endpoints today would make the demo the debut of the new noun, which is the thing D2 rules out. Task 0.0 waits for the rename and the regenerated schema, and nothing starts before it. Also corrects a premise the generator team refuted. This proposal said generation runs against "a public repository the service controls". It does not: CLI-bound generation never clones, the clone is deferred to the git-bound push that only happens for a pull request, and `repositoryUrl` on a CLI request is authorization scoping rather than model input. The fixed input is a prompt and its examples held as data. Recorded as D2a, because the wrong version was specific enough to be believed, and a later reader would have tried to make the repository load-bearing. --- openspec/changes/runtime-demo-path/design.md | 26 ++++++++++++++++++- .../changes/runtime-demo-path/proposal.md | 9 ++++--- .../specs/cli-runtime-demo/spec.md | 7 +++-- openspec/changes/runtime-demo-path/tasks.md | 18 +++++++++++-- 4 files changed, 50 insertions(+), 10 deletions(-) diff --git a/openspec/changes/runtime-demo-path/design.md b/openspec/changes/runtime-demo-path/design.md index 197d0de4..42595684 100644 --- a/openspec/changes/runtime-demo-path/design.md +++ b/openspec/changes/runtime-demo-path/design.md @@ -55,7 +55,7 @@ Concretely: `submitRule`, `pollRuleStatus` and `writeRuleFile` are reused, not reimplemented. If the demo needs a new writer, that is evidence the shapes have diverged, and the demo has done its job by failing. -### D2 — The demo tracks the mainline's naming, whatever it is +### D2 — The demo tracks the mainline's naming, and follows rather than leads The paths above mirror today's `/cli/api/rule` naming, including its inconsistency (**N10**: `ruleId` names a ticket, by the service's own field @@ -66,6 +66,24 @@ The demo must never be where a second convention lives. Its value is being indistinguishable from the path users take; a demo that is tidier than production is a demo that stops proving anything. +**This is now a sequencing constraint rather than a principle.** The generator +team is renaming the request resource to `request`/`requestId` as its own +change, ahead of this one. So the demo endpoints are named after the rename +lands, not before — building them first would make the demo the debut of the +new convention, which is precisely what this decision forbids. + +### D2a — The fixed input is data the service holds, not a repository + +An earlier draft of this proposal said "a public repository the service +controls". That was wrong, and the generator team corrected it: CLI-bound +generation never clones. The clone is deferred to the git-bound push, which +happens only for a pull request, and `repositoryUrl` on a CLI request is +authorization scoping and a ticket field rather than model input. + +So the fixture is a prompt and its examples, held as data alongside the +rule-hash vectors. Naming a repository would have been decoration, and +decoration is what a later reader tries to make load-bearing. + ### D3 — It stops at `verify`, and that is stated rather than worked around A runtime rule executes only when an authenticated reconcile returns its @@ -115,6 +133,12 @@ that fails, because it would look like success. ## Risks / Trade-offs +**A rename lands first, and this waits for it.** The request resource becomes +`request`/`requestId`, which is a change to paths and to a field name this +client reads. Regenerating the types catches most of it at compile time; two +call sites build their URL as a template string and would not fail to compile, +so they are changed deliberately rather than found later. + **The service half does not exist yet.** The endpoints are the service team's to build, raised as N9. This change assumes the shapes in D1; if they land differently, the client work is the adapter and the tests move with it. Nothing diff --git a/openspec/changes/runtime-demo-path/proposal.md b/openspec/changes/runtime-demo-path/proposal.md index a5ed668d..072ba055 100644 --- a/openspec/changes/runtime-demo-path/proposal.md +++ b/openspec/changes/runtime-demo-path/proposal.md @@ -31,7 +31,7 @@ hand-maintained literal and lookup is by filename, so a topic that is not in the list is fetchable by name and invisible otherwise. No new mechanism. This is something we run deliberately, not something routing sends an agent to. -**A command that walks the path.** Ticket, poll, retrieve, write, verify — +**A command that walks the path.** Request, poll, retrieve, write, verify — through the existing `submitRule` / `pollRuleStatus` client code and the existing `writeRuleFile`, against demo endpoints that mirror the mainline shapes: @@ -63,8 +63,8 @@ hole in it. Executing the demo rule uses the documented ### New Capabilities - `cli-runtime-demo`: A deliberately-invoked path that generates, delivers, - writes and verifies one runtime rule against a service-controlled repository, - so the client-service seam can be shown working rather than described. + writes and verifies one runtime rule from a fixed, service-held input, so the + client-service seam can be shown working rather than described. ### Modified Capabilities @@ -82,5 +82,8 @@ hole in it. Executing the demo rule uses the documented - Depends on the service half (raised as **N9** in the cross-team document). The endpoints are theirs to build; the shapes above are what we asked for and what this change assumes. +- Sequenced behind the `request`/`requestId` rename (**N10**), which the + generator team is doing as its own change. The demo takes whichever noun + wins, after the mainline takes it. - No change to the runtime gate, to reconcile, or to `check`'s execution policy. diff --git a/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md b/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md index ba55cdce..a066bb01 100644 --- a/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md +++ b/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md @@ -2,10 +2,9 @@ ### Requirement: A deliberately-invoked demo path -The CLI SHALL provide a path that generates one runtime rule against a -service-controlled repository, writes it, and verifies it, so the boundary -between this client and the rule service can be shown working rather than -described. +The CLI SHALL provide a path that generates one runtime rule from a fixed +input the service holds, writes it, and verifies it, so the boundary between +this client and the rule service can be shown working rather than described. It SHALL be invoked deliberately. Routing SHALL NOT reach it, and no recipe that decides how to author a rule SHALL name it as a destination. diff --git a/openspec/changes/runtime-demo-path/tasks.md b/openspec/changes/runtime-demo-path/tasks.md index 9d2bf56d..c30c20d1 100644 --- a/openspec/changes/runtime-demo-path/tasks.md +++ b/openspec/changes/runtime-demo-path/tasks.md @@ -3,13 +3,27 @@ One PR. The command and the recipe that names it are the same seam this change exists to remove, so they land together. -## 0. Blocked on the service half +## 0. Blocked on the rename, then on the service half + +The generator team is renaming the request resource first, as its own change: +`request`/`requestId` rather than `rule`/`ruleId`. Their own +`openspec/specs/cli/spec.md` already specified it, and their live `meta` block +already carries `ticketId` per delivered rule, so the route was the outlier. + +- [ ] 0.0 Wait for the rename to land and the new schema to publish, then regenerate `src/generated/api.d.ts` against it + +**The demo does not start before that.** Naming these endpoints today would make +the demo the place a new convention debuts, which is the one thing design D2 +rules out. Whichever noun wins, both demo endpoints take it, and they take it +after the mainline does. + +## 0b. Blocked on the service half - [ ] 0.1 Confirm the endpoint shapes with the generator team (**N9**): `POST /cli/api/demo/rule` returning `{ ruleId, status }`, `GET /cli/api/demo/rule/{ruleId}` returning `{ ruleId, status, rules[] }` with the published file-set variant - [ ] 0.2 Confirm the demo is servable unauthenticated, and get their answer on whether reconcile can bless the sample signature for an anonymous caller (design D3). A "no" is a complete answer and changes nothing below - [ ] 0.3 Agree what the demo rule is for — the scenario the generated rule addresses — so the demo shows something a person recognises rather than an arbitrary rule -**Nothing below starts until 0.1 lands.** Building against a guessed shape is +**Nothing below starts until 0.0 and 0.1 land.** Building against a guessed shape is what produced the seam this change is closing. ## 1. The client path From 1f614650bdfd4de8f274739dbd3e70f4edb5f2e7 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Tue, 1 Sep 2026 16:42:30 -0700 Subject: [PATCH 3/8] docs(openspec): the rename landed, so the demo takes its noun Task 0.0 waited for `request`/`requestId` to become canonical. It has: verified from `GET /cli/api/__schema` that `/cli/api/request/*` is canonical and the `rule/*` family is `deprecated: true` while still serving. So the demo endpoints are `POST /cli/api/demo/request` and `GET /cli/api/demo/request/{requestId}`. That was the whole reason for holding this change behind the rename: naming them earlier would have made the demo the place a new convention debuted, which is what D2 rules out. The mainline took the noun first and the demo follows it. Adopting the renamed paths in the ordinary client is its own change and is deliberately not a dependency of this one. The legacy family still serves, so the two can land in either order. --- openspec/changes/runtime-demo-path/proposal.md | 10 +++++----- openspec/changes/runtime-demo-path/tasks.md | 10 +++++----- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/openspec/changes/runtime-demo-path/proposal.md b/openspec/changes/runtime-demo-path/proposal.md index 072ba055..98e63108 100644 --- a/openspec/changes/runtime-demo-path/proposal.md +++ b/openspec/changes/runtime-demo-path/proposal.md @@ -37,8 +37,8 @@ existing `writeRuleFile`, against demo endpoints that mirror the mainline shapes: ``` -POST /cli/api/demo/rule -> { ruleId, status } -GET /cli/api/demo/rule/{ruleId} -> { ruleId, status, rules[] } +POST /cli/api/demo/request -> { requestId, status } +GET /cli/api/demo/request/{requestId} -> { requestId, status, rules[] } ``` Mirroring is the point. A bespoke demo payload would exercise a path no user is @@ -82,8 +82,8 @@ hole in it. Executing the demo rule uses the documented - Depends on the service half (raised as **N9** in the cross-team document). The endpoints are theirs to build; the shapes above are what we asked for and what this change assumes. -- Sequenced behind the `request`/`requestId` rename (**N10**), which the - generator team is doing as its own change. The demo takes whichever noun - wins, after the mainline takes it. +- Sequenced behind the `request`/`requestId` rename (**N10**), which has now + shipped and is verified live. The demo takes that noun. Adopting it in the + ordinary client is a separate change and is not this one's dependency. - No change to the runtime gate, to reconcile, or to `check`'s execution policy. diff --git a/openspec/changes/runtime-demo-path/tasks.md b/openspec/changes/runtime-demo-path/tasks.md index c30c20d1..2735e212 100644 --- a/openspec/changes/runtime-demo-path/tasks.md +++ b/openspec/changes/runtime-demo-path/tasks.md @@ -10,12 +10,12 @@ The generator team is renaming the request resource first, as its own change: `openspec/specs/cli/spec.md` already specified it, and their live `meta` block already carries `ticketId` per delivered rule, so the route was the outlier. -- [ ] 0.0 Wait for the rename to land and the new schema to publish, then regenerate `src/generated/api.d.ts` against it +- [x] 0.0 Wait for the rename to land and the new schema to publish. **Done**: verified from `GET /cli/api/__schema` that `/cli/api/request/*` is canonical and the `rule/*` family is `deprecated: true` while still serving. Adopting it in our client is its own change, tracked separately +- [x] 0.0a The demo therefore takes the settled noun: `POST /cli/api/demo/request` and `GET /cli/api/demo/request/{requestId}` -**The demo does not start before that.** Naming these endpoints today would make -the demo the place a new convention debuts, which is the one thing design D2 -rules out. Whichever noun wins, both demo endpoints take it, and they take it -after the mainline does. +The demo did not start before that, which was the point. Naming these endpoints +earlier would have made the demo the place a new convention debuted, which is +what design D2 rules out. The mainline took the noun first; the demo follows it. ## 0b. Blocked on the service half From 65881fb9ad52de40138eb096914480d6a347f3df Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Tue, 1 Sep 2026 16:42:55 -0700 Subject: [PATCH 4/8] docs(openspec): finish the noun change in task 0.1 and D1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous commit renamed the endpoints in the proposal and the task list header and left two references behind, in task 0.1 and in design D1. Both still said `demo/rule` and `ruleId`. Recording it rather than folding it in silently, because it is the exact failure this change is about: the noun moved in one place and not in the places describing it, and nothing measured the gap. Grepped for the old spelling afterwards this time. 0.1 is also checked off — the shapes are agreed on both sides. --- openspec/changes/runtime-demo-path/design.md | 4 ++-- openspec/changes/runtime-demo-path/tasks.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/openspec/changes/runtime-demo-path/design.md b/openspec/changes/runtime-demo-path/design.md index 42595684..3f6435db 100644 --- a/openspec/changes/runtime-demo-path/design.md +++ b/openspec/changes/runtime-demo-path/design.md @@ -40,8 +40,8 @@ and the whole of the last change was about it not doing anything else. ### D1 — The demo uses the well-known ticket and retrieval formats -`POST /cli/api/demo/rule` returning `{ ruleId, status }`, then -`GET /cli/api/demo/rule/{ruleId}` returning `{ ruleId, status, rules[] }`. The +`POST /cli/api/demo/request` returning `{ requestId, status }`, then +`GET /cli/api/demo/request/{requestId}` returning `{ requestId, status, rules[] }`. The same two-stage flow as `rule create`, the same status enum, the same file-set variant. diff --git a/openspec/changes/runtime-demo-path/tasks.md b/openspec/changes/runtime-demo-path/tasks.md index 2735e212..e0b71aaa 100644 --- a/openspec/changes/runtime-demo-path/tasks.md +++ b/openspec/changes/runtime-demo-path/tasks.md @@ -19,7 +19,7 @@ what design D2 rules out. The mainline took the noun first; the demo follows it. ## 0b. Blocked on the service half -- [ ] 0.1 Confirm the endpoint shapes with the generator team (**N9**): `POST /cli/api/demo/rule` returning `{ ruleId, status }`, `GET /cli/api/demo/rule/{ruleId}` returning `{ ruleId, status, rules[] }` with the published file-set variant +- [x] 0.1 Confirm the endpoint shapes with the generator team (**N9**): `POST /cli/api/demo/request` returning `{ requestId, status }`, `GET /cli/api/demo/request/{requestId}` returning `{ requestId, status, rules[] }` with the published file-set variant. Agreed both sides - [ ] 0.2 Confirm the demo is servable unauthenticated, and get their answer on whether reconcile can bless the sample signature for an anonymous caller (design D3). A "no" is a complete answer and changes nothing below - [ ] 0.3 Agree what the demo rule is for — the scenario the generated rule addresses — so the demo shows something a person recognises rather than an arbitrary rule From a722f8cc76657f99954bbf4413caadd4367f9c85 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Tue, 1 Sep 2026 17:22:56 -0700 Subject: [PATCH 5/8] docs(openspec): settle D2's tense and name where the demo command lives D2 still framed the request rename as pending while D1 and tasks 0.0/0.0a already recorded it as landed, so a reader met a contradiction about which naming is current. D2 now states the principle, then the rename as history, and the Risks entry describes the skew that actually remains: the demo is born on request/requestId while the ordinary client stays on the deprecated family. The cli-agent requirement also only covered RECIPE_TOPICS, but `taskless agent` prints two independently-sourced listings and the other one iterates the top-level command tree, filtering only `agent` itself. Nothing said where the demo command lives, so a top-level verb would have been hidden as a recipe and advertised as a command in the same output. D4a settles it as `rule demo`, nested under the existing rule command, obtaining the property by not registering rather than by teaching a renderer to skip. --- openspec/changes/runtime-demo-path/design.md | 68 ++++++++++++++----- .../changes/runtime-demo-path/proposal.md | 13 ++-- .../runtime-demo-path/specs/cli-agent/spec.md | 21 ++++++ .../specs/cli-runtime-demo/spec.md | 21 ++++++ openspec/changes/runtime-demo-path/tasks.md | 6 +- 5 files changed, 105 insertions(+), 24 deletions(-) diff --git a/openspec/changes/runtime-demo-path/design.md b/openspec/changes/runtime-demo-path/design.md index 3f6435db..4905d537 100644 --- a/openspec/changes/runtime-demo-path/design.md +++ b/openspec/changes/runtime-demo-path/design.md @@ -57,20 +57,24 @@ diverged, and the demo has done its job by failing. ### D2 — The demo tracks the mainline's naming, and follows rather than leads -The paths above mirror today's `/cli/api/rule` naming, including its -inconsistency (**N10**: `ruleId` names a ticket, by the service's own field -description). If the mainline is renamed, these are renamed with it in the same -change. - -The demo must never be where a second convention lives. Its value is being +The paths above mirror the mainline's naming, whatever it currently is. The +demo must never be where a second convention lives. Its value is being indistinguishable from the path users take; a demo that is tidier than -production is a demo that stops proving anything. - -**This is now a sequencing constraint rather than a principle.** The generator -team is renaming the request resource to `request`/`requestId` as its own -change, ahead of this one. So the demo endpoints are named after the rename -lands, not before — building them first would make the demo the debut of the -new convention, which is precisely what this decision forbids. +production is a demo that stops proving anything. If the mainline is renamed +again, these are renamed with it. + +**That principle already cost this change a wait, which is the evidence it is +real.** The generator team renamed the request resource from `rule`/`ruleId` to +`request`/`requestId` as its own change (**N10**, the inconsistency being that +`ruleId` named a ticket, by the service's own field description). That rename +has landed and is verified live, so the demo takes the settled noun above. +Naming the demo endpoints before it landed would have made the demo the debut +of the new convention, which is precisely what this decision forbids. + +Adopting the new noun in the _ordinary_ client is a separate change and is not +this one's dependency. `/cli/api/rule/*` still serves, marked `deprecated`, so +the mainline client is not broken by having not moved yet — but the demo, being +new, has no reason to be born on the deprecated spelling. ### D2a — The fixed input is data the service holds, not a repository @@ -122,6 +126,32 @@ The cost is that the index is curated by hand and can drift from the files on disk. That is already true, and `cli-agent` gains a requirement saying it is intended rather than an oversight. +### D4a — The demo command nests under `rule`, so the second listing never sees it + +`taskless agent` prints **two** lists, and omission from `RECIPE_TOPICS` only +governs one of them. The first, headed `Topics:`, is built by iterating +`Object.entries(subCommands)` — the top-level command tree from `src/index.ts`, +gated by `SUBCOMMAND_NAMES` in `commands/names.ts` — and it filters exactly one +name, `agent` itself. Nothing in it consults `RECIPE_TOPICS`. The second, +headed `Authoring recipes:`, is `RECIPE_TOPICS`, and that is the list D4 is +about. + +So a demo registered as a new top-level verb would be hidden as a recipe and +advertised as a command in the same output, which is the opposite of the +property this change wants. **The demo command is therefore `rule demo`, a +subcommand of the existing `rule` command**, alongside `create`, `improve`, +`meta` and `delete`. It writes a rule, so it reads correctly there, and it adds +no entry to `SUBCOMMAND_NAMES` or `subCommands` — meaning the `Topics:` listing +is untouched and no filtering code has to be written to keep it that way. + +This is the same reasoning as D4 rather than an exception to it: the property is +obtained by not registering the thing, not by teaching a renderer to skip it. +The consequence is that `taskless rule --help` does list `demo`, from citty's +own usage output. That is intended. The requirement is that **routing** never +sends an agent to the demo, not that the demo is a secret — a person reading +`rule --help` has already chosen to look, and D3's risks say the same about the +recipe. + ### D5 — Failure is reported, never fabricated If generation cannot be served, the service answers with a terminal status and @@ -133,11 +163,13 @@ that fails, because it would look like success. ## Risks / Trade-offs -**A rename lands first, and this waits for it.** The request resource becomes -`request`/`requestId`, which is a change to paths and to a field name this -client reads. Regenerating the types catches most of it at compile time; two -call sites build their URL as a template string and would not fail to compile, -so they are changed deliberately rather than found later. +**The demo is born on `request`/`requestId` while the ordinary client is still +on `rule`/`ruleId`.** That skew is deliberate (D2) and bounded: the `rule/*` +family still serves, marked `deprecated`, so nothing is broken by the mainline +not having moved yet. When it does adopt the rename, regenerating the types +catches most of it at compile time; two call sites build their URL as a +template string and would not fail to compile, so they are changed deliberately +rather than found later. **The service half does not exist yet.** The endpoints are the service team's to build, raised as N9. This change assumes the shapes in D1; if they land diff --git a/openspec/changes/runtime-demo-path/proposal.md b/openspec/changes/runtime-demo-path/proposal.md index 98e63108..46999095 100644 --- a/openspec/changes/runtime-demo-path/proposal.md +++ b/openspec/changes/runtime-demo-path/proposal.md @@ -31,7 +31,7 @@ hand-maintained literal and lookup is by filename, so a topic that is not in the list is fetchable by name and invisible otherwise. No new mechanism. This is something we run deliberately, not something routing sends an agent to. -**A command that walks the path.** Request, poll, retrieve, write, verify — +**A command that walks the path, as `rule demo`.** Request, poll, retrieve, write, verify — through the existing `submitRule` / `pollRuleStatus` client code and the existing `writeRuleFile`, against demo endpoints that mirror the mainline shapes: @@ -71,13 +71,18 @@ hole in it. Executing the demo rule uses the documented - `cli-agent`: The topic index is a curated list rather than every embedded recipe. A topic may exist and be fetchable by name while staying out of the index, which is already how the code behaves and is not yet stated as a - requirement. + requirement. The index's command listing is separately sourced from the + top-level command tree, so staying out of the curated recipe list says + nothing about that half. ## Impact - `packages/cli/src/agent/__undocumented-sample-runtime.txt` — new recipe. -- `packages/cli/src/commands/` — the demo command, reusing `submitRule`, - `pollRuleStatus` and `writeRuleFile` rather than duplicating them. +- `packages/cli/src/commands/rules.ts` — the demo as a `rule demo` subcommand, + reusing `submitRule`, `pollRuleStatus` and `writeRuleFile` rather than + duplicating them. Nested rather than top-level so it adds nothing to + `SUBCOMMAND_NAMES` and therefore nothing to the `agent` index's command + listing (design D4a). - `packages/cli/src/api/` — the demo endpoints, alongside the existing client. - Depends on the service half (raised as **N9** in the cross-team document). The endpoints are theirs to build; the shapes above are what we asked for and diff --git a/openspec/changes/runtime-demo-path/specs/cli-agent/spec.md b/openspec/changes/runtime-demo-path/specs/cli-agent/spec.md index c425324d..2224d026 100644 --- a/openspec/changes/runtime-demo-path/specs/cli-agent/spec.md +++ b/openspec/changes/runtime-demo-path/specs/cli-agent/spec.md @@ -22,3 +22,24 @@ it to be read as an oversight and "fixed". - **WHEN** the topic index is rendered - **THEN** a topic that is not in the curated list SHALL NOT appear in it + +### Requirement: The rendered index has two independently-sourced listings + +The `agent` index SHALL be treated as two separate listings: a command listing +built from the registered top-level subcommands, and the curated recipe +listing. Absence from the curated recipe listing SHALL NOT be relied on to keep +an entry out of the command listing, because the two share no source. + +**Rationale.** The command listing iterates the top-level command tree and +filters only `agent` itself; it never consults the curated recipe list. A +deliberately-invoked capability that registers a new top-level command is +therefore advertised there no matter what the recipe list says. Stating the two +listings separately keeps a later reader from assuming one requirement covers +both, which is how a topic ends up hidden in one half of its own output. + +#### Scenario: A deliberately-invoked capability adds no top-level command + +- **WHEN** a capability is meant to be invoked deliberately rather than + discovered from the index +- **THEN** it SHALL NOT be registered as a new top-level subcommand +- **AND** the command listing SHALL be unchanged by its addition diff --git a/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md b/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md index a066bb01..3aac5f01 100644 --- a/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md +++ b/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md @@ -21,6 +21,27 @@ that decides how to author a rule SHALL name it as a destination. - **WHEN** an agent fetches any authoring or routing recipe - **THEN** the demo topic SHALL NOT appear as a destination in that recipe +### Requirement: The demo is invoked as a subcommand of an existing command + +The demo SHALL be dispatched as `rule demo`, a subcommand of the existing +`rule` command. It SHALL NOT be registered as a new top-level subcommand, and +SHALL add no entry to the CLI's top-level command name list. + +**Rationale.** The `agent` index renders a command listing built from the +top-level command tree, separately from its curated recipe listing. A new +top-level verb would be advertised there regardless of the recipe being +unlisted, so the demo would be hidden in one half of the same output and +announced in the other. Nesting under `rule` obtains the property by not +registering the command, which is the same reasoning that keeps the recipe out +of the curated list rather than teaching a renderer to skip it. + +#### Scenario: The demo adds nothing to the top-level command listing + +- **WHEN** the `agent` index is rendered +- **THEN** the command listing SHALL NOT contain a demo entry +- **AND** the listing SHALL be identical to the one rendered before the demo + existed + ### Requirement: The demo uses the well-known request and retrieval formats The demo SHALL obtain its rule through the same two-stage flow as an ordinary diff --git a/openspec/changes/runtime-demo-path/tasks.md b/openspec/changes/runtime-demo-path/tasks.md index e0b71aaa..8ad25c71 100644 --- a/openspec/changes/runtime-demo-path/tasks.md +++ b/openspec/changes/runtime-demo-path/tasks.md @@ -5,7 +5,7 @@ exists to remove, so they land together. ## 0. Blocked on the rename, then on the service half -The generator team is renaming the request resource first, as its own change: +The generator team renamed the request resource first, as its own change: `request`/`requestId` rather than `rule`/`ruleId`. Their own `openspec/specs/cli/spec.md` already specified it, and their live `meta` block already carries `ticketId` per delivered rule, so the route was the outlier. @@ -20,7 +20,7 @@ what design D2 rules out. The mainline took the noun first; the demo follows it. ## 0b. Blocked on the service half - [x] 0.1 Confirm the endpoint shapes with the generator team (**N9**): `POST /cli/api/demo/request` returning `{ requestId, status }`, `GET /cli/api/demo/request/{requestId}` returning `{ requestId, status, rules[] }` with the published file-set variant. Agreed both sides -- [ ] 0.2 Confirm the demo is servable unauthenticated, and get their answer on whether reconcile can bless the sample signature for an anonymous caller (design D3). A "no" is a complete answer and changes nothing below +- [ ] 0.2 Confirm the service can serve the demo endpoints unauthenticated, and get their answer on whether reconcile can bless the sample signature for an anonymous caller. This asks whether the service can do it; design D3 already settles what the demo does either way — it stops at `verify` — so a "no" on the reconcile half is a complete answer and changes nothing below. A "no" on serving unauthenticated at all is the one answer that reopens the design - [ ] 0.3 Agree what the demo rule is for — the scenario the generated rule addresses — so the demo shows something a person recognises rather than an arbitrary rule **Nothing below starts until 0.0 and 0.1 land.** Building against a guessed shape is @@ -36,10 +36,12 @@ what produced the seam this change is closing. ## 2. The hidden topic +- [ ] 2.0 Register the command as `rule demo` under the existing `rule` command in `src/commands/rules.ts`. Not a top-level verb: `SUBCOMMAND_NAMES` and `subCommands` are unchanged, so the `agent` index's command listing gains nothing (design D4a) - [ ] 2.1 Add `src/agent/__undocumented-sample-runtime.txt`, absent from `RECIPE_TOPICS` - [ ] 2.2 The recipe names the command, says the rule may be imperfect, and says how to delete it - [ ] 2.3 The recipe states where the demo stops and why, so a reader meets the signature gate as a design rather than as a failure - [ ] 2.4 Test that the topic is fetchable by name and absent from the index +- [ ] 2.5 Test that the `agent` index's command listing is unchanged by the demo, so the second listing is covered and not just `RECIPE_TOPICS` ## 3. Prove it does not weaken the gate From 58edbae9b23003f4fee30a7b29c1d6b3a8a6a67c Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Tue, 1 Sep 2026 17:34:34 -0700 Subject: [PATCH 6/8] docs(openspec): say what "reuse" means for each of the three MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review feedback on #235. D1 claimed `submitRule`, `pollRuleStatus` and `writeRuleFile` are "reused, not reimplemented", and for two of the three that is not true yet. Both requesters are hardcoded to the rule-family endpoints and take a required `token`, which `createApiClient` turns unconditionally into an `Authorization: Bearer` header. The demo is unauthenticated by requirement, so neither is usable as written. The path half resolves itself when the client adopts the renamed family; the token half does not, and is the durable part. The reason this is worth more than the wording: D1's failure test says "if the demo needs a new writer, the shapes have diverged, and the demo has done its job by failing". That test is a claim about PAYLOAD SHAPES, and it was already tripping — for the requester and the poller, where it means nothing of the kind. A requester needing a second argument says nothing about whether the demo and the mainline agree on what a rule looks like. So D1 now splits them. `writeRuleFile` is byte-for-byte reuse and is what the failure test is really about. The other two are two small changes to functions written when every caller was authenticated, and they are planned as task 1.2a rather than discovered mid-implementation, which is what the previous wording set up. 1.2b is the part nobody raised: an unauthenticated call must send no `Authorization` header rather than one carrying an empty token. A header the service then has to decide how to ignore is a worse contract than its absence. --- openspec/changes/runtime-demo-path/design.md | 29 ++++++++++++++++++-- openspec/changes/runtime-demo-path/tasks.md | 2 ++ 2 files changed, 28 insertions(+), 3 deletions(-) diff --git a/openspec/changes/runtime-demo-path/design.md b/openspec/changes/runtime-demo-path/design.md index 4905d537..4432dc27 100644 --- a/openspec/changes/runtime-demo-path/design.md +++ b/openspec/changes/runtime-demo-path/design.md @@ -51,9 +51,32 @@ demo uses would demonstrate something no user is on. Mirroring also means the demo covers polling and the terminal `failed` and `unsupported` states for free, because it is the same client code reaching them. -Concretely: `submitRule`, `pollRuleStatus` and `writeRuleFile` are reused, not -reimplemented. If the demo needs a new writer, that is evidence the shapes have -diverged, and the demo has done its job by failing. +Concretely, and the three are not equal — saying "reuse" without splitting them +would make the failure test below fire for a reason it was not built to catch. + +**`writeRuleFile` is reused byte for byte, and it is what the test is about.** +It carries the path checks, the completeness rules, the layout validation and +the refusal to write a runtime rule with no captures. **If the demo needs a +different writer, the payload shapes have diverged**, and the demo has done its +job by failing. That is the signal worth having, and it is the whole reason for +mirroring the delivery variant. + +**`submitRule` and `pollRuleStatus` need work first, and that is a task rather +than a divergence signal.** Both are hardcoded to the rule-family endpoints and +both take a required `token`, which `createApiClient` turns unconditionally into +an `Authorization: Bearer` header. The demo is unauthenticated by requirement +(D3), so neither is usable as it stands: the endpoint has to be a parameter, and +the token has to be optional. + +The path half of that resolves itself when the client adopts the renamed family, +which is out of scope here. The token half does not, and is the durable part. +Neither is evidence of anything having gone wrong — they are two small changes +to functions written when every caller was authenticated, and they are planned +in task 1.2a rather than discovered during implementation. + +The distinction matters because D1's failure test is a **claim about payload +shapes**. A requester that needs a second argument says nothing about whether +the demo and the mainline agree on what a rule looks like. ### D2 — The demo tracks the mainline's naming, and follows rather than leads diff --git a/openspec/changes/runtime-demo-path/tasks.md b/openspec/changes/runtime-demo-path/tasks.md index 8ad25c71..6d5620c2 100644 --- a/openspec/changes/runtime-demo-path/tasks.md +++ b/openspec/changes/runtime-demo-path/tasks.md @@ -30,6 +30,8 @@ what produced the seam this change is closing. - [ ] 1.1 Add the demo endpoints to `src/api/`, reusing the existing client rather than a second fetch layer - [ ] 1.2 Drive them through `submitRule`/`pollRuleStatus`, so polling and the terminal `failed`/`unsupported` states are the same code an ordinary generation uses +- [ ] 1.2a Make that reuse possible first: `submitRule` and `pollRuleStatus` are hardcoded to the rule-family endpoints and take a required `token` that `createApiClient` turns into an `Authorization` header unconditionally. Parameterize the endpoint and make the token optional. **Not a divergence signal** — see D1; these were written when every caller was authenticated, and the demo is the first that is not +- [ ] 1.2b Confirm an unauthenticated call sends no `Authorization` header at all, rather than one with an empty token. A header the service must then decide how to ignore is a worse contract than its absence - [ ] 1.3 Write through `writeRuleFile`. If the demo needs its own writer, stop: the shapes have diverged and that is the finding - [ ] 1.4 Report a terminal status by surfacing the service's reason, which `unsupportedMessage` already prefers over our own text - [ ] 1.5 Regenerate `src/generated/api.d.ts` once the endpoints are published From b3f5e50e8b9a6cc878e84c00286c686ab23a390c Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Tue, 1 Sep 2026 20:27:25 -0700 Subject: [PATCH 7/8] docs(openspec): the demo is account-free, and its limits are the design The generator team settled the demonstration as account-free: no ticket, no corpus entry, no billing, no authentication, one Taskless-owned installation behind every request. Three things follow. The endpoints take no auth at all rather than an ignorable empty token, so the header ask is a property instead of a convention. Retrieval also serves what the check found against the fixture's examples, which the verification gate was already computing and discarding. D3 said logging in was "the next beat" to reach a blessed execution. That is wrong: blessing is recording, and a fixed rule shared by every caller is never recorded for any of them, so authenticating reaches nothing. The gate is unreachable by construction from a shared fixture rather than one step away. Exercising the handshake needs a rule recorded for the org that runs it, which is an integration test in an org we own, not a demo. D6 takes the findings as `Finding[]` verbatim, grouped per example with what that example was expected to do. `Finding` is our own runtime-rule type, so a narrower demo-only object would be the bespoke shape D1 forbids. The grouping is what carries the demonstration: a flat list renders identically for a rule that catches the failing examples and one that fires on everything, and the second is this area's recurring defect wearing a success costume. D7 states what the negative framing obscures. The rule is real and inspectable; it is inert because no signature for it is in any org's run set, which is the gate working on the ordinary path; it is removable through the delete flow we own; and improve refuses it, correctly, because nothing pre-generated can match a rule authored against the developer's own repo. Tasks 3.4 and 5.2 pin the two that would fail silently. --- openspec/changes/runtime-demo-path/design.md | 91 ++++++++++++++++++- .../changes/runtime-demo-path/proposal.md | 31 +++++-- .../specs/cli-runtime-demo/spec.md | 54 +++++++++++ openspec/changes/runtime-demo-path/tasks.md | 11 ++- 4 files changed, 176 insertions(+), 11 deletions(-) diff --git a/openspec/changes/runtime-demo-path/design.md b/openspec/changes/runtime-demo-path/design.md index 4432dc27..889945f7 100644 --- a/openspec/changes/runtime-demo-path/design.md +++ b/openspec/changes/runtime-demo-path/design.md @@ -130,9 +130,25 @@ network and arbitrary code running on a developer's machine. A hole in it that exists "only for the demo" is a hole, and the id it keys on is attacker-visible in the recipe. No demo is worth that. -The consequence is that this change demonstrates everything up to the gate. If -showing a blessed rule execute matters, logging in is the next beat, and it is -a better demo for being the payoff rather than the prerequisite. +The consequence is that this change demonstrates everything up to the gate. + +**An earlier draft of this section said logging in was "the next beat". That is +wrong, and the correction matters more than the wording.** Blessing is +recording: a signature exists for a rule because that rule was written into an +organization's corpus. The demo rule is one fixed rule, served to every caller +from a Taskless-owned installation, and it is never recorded for the caller. So +there is no signature for it in any caller's `run` set, and authenticating adds +nothing — the gate is not a step the demo stops just short of, it is unreachable +by construction from a shared fixture. + +That also disposes of the idea that the demo half-serves the blessed-execution +handshake and should be extended to serve it fully. It cannot. Exercising "we +bless, you run" needs a rule recorded for the organization that runs it, which +is what an ordinary authored rule already is. If that seam wants automated +coverage, the honest shape is an integration test against an organization we +own — not a demo — and it should be argued on those terms rather than folded in +here. Until someone does, the seam is covered by unit tests on each side and by +nothing that spans them, and this change records that rather than obscuring it. ### D4 — Hidden by omission, not by a new mechanism @@ -184,6 +200,75 @@ reason over its own text for `unsupported`, so this needs nothing new. A demo that invents a rule when generation fails would be worse than a demo that fails, because it would look like success. +### D6 — The findings ship as `Finding[]`, grouped by the example that produced them + +The service offered either its harness's `Finding[]` or something narrower, and +left the shape to us since we render it. + +`Finding[]` as-is, because it is not the service's internal type: `Finding` is +declared in `types/runtime-rule.ts` as part of the runtime-rule contract, and is +what a check returns on this side. Choosing a narrower demo-only object would be +introducing a payload only the demo consumes, which is the thing D1 exists to +forbid. The same reasoning that reuses `submitRule`, `pollRuleStatus` and +`writeRuleFile` applies to the type a check's results already have. + +What the demo does add is a wrapper, and the wrapper is the part carrying the +demonstration: + +``` +examples: [ { name, expectation: "fails" | "passes", findings: Finding[] } ] +``` + +A flat list of findings cannot distinguish a rule that catches the failing +examples from one that fires on everything it is shown, and those two render +identically as "3 findings". The second is the recurring defect in this area +wearing a success costume — an empty or indiscriminate scan reporting as a pass. +Attributing each finding to an example, and stating what that example was +expected to do, makes the asymmetry the thing a reader sees: findings on the +examples that should fail, none on the examples that should pass. + +That also makes the demo checkable rather than merely viewable. The CLI can +assert the asymmetry instead of printing whatever arrives, so a demo that +silently stops finding anything fails rather than looking clean. + +### D7 — Each thing the demo cannot do is the behaviour we want, not a shortfall + +D3 and D6 describe the demo negatively: it cannot reach the gate, and it must +fail rather than render an indiscriminate rule. Read together they invite the +wrong conclusion, that the demo is a reduced version of something better. It is +not. Every limit is a correct behaviour arriving through the ordinary path. + +**The rule is real and inspectable.** A developer gets `check.ts` and its +captures on disk and can read what a runtime rule actually is. That is the +demonstration. Note what is _not_ on disk: a signature. `run-set.ts` computes +one from the check's bytes at reconcile time; delivery never writes a signature +artifact. So there is nothing to inspect there and nothing to tamper with, and +the demo's value is the rule's content rather than a token. + +**It is inert, and inert is the correct state.** An ordinary `check` skips it +because no signature for it is in any organization's `run` set. A developer who +runs `check` and sees the rule skipped is watching the gate work, on the +ordinary code path, with the reason `check` already prints. Nothing about the +demo is special-cased to produce that. + +**It is removable, through the delete flow we already own.** No demo-specific +teardown, no residue `verify` or `check` reports afterwards. + +**`improve` does not work on it, and should not.** Improvement resolves a +request the caller's organization holds, and the demo is recorded in no +organization, so it terminates as `RULE_NOT_FOUND` — the correct code, since the +service genuinely has no such record. The deeper reason is worth stating: a +pre-generated rule built against a shared fixture is a bad base to iterate on. +Nothing generated in advance can match a rule authored against this developer's +own repository and context, so offering the demo rule as a starting point would +be offering a worse starting point wearing the demo's credibility. + +**This is correct-by-accident until it is pinned.** Each of these follows from +existing behaviour rather than from code written for the demo, which is exactly +the situation the `metadata.taskless` test was written for: behaviour we depend +on, produced by code that never knew about us. Tasks 3.4 and 5.2 pin the two +that would be silent if they broke. + ## Risks / Trade-offs **The demo is born on `request`/`requestId` while the ordinary client is still diff --git a/openspec/changes/runtime-demo-path/proposal.md b/openspec/changes/runtime-demo-path/proposal.md index 46999095..1a6989bd 100644 --- a/openspec/changes/runtime-demo-path/proposal.md +++ b/openspec/changes/runtime-demo-path/proposal.md @@ -38,19 +38,38 @@ shapes: ``` POST /cli/api/demo/request -> { requestId, status } -GET /cli/api/demo/request/{requestId} -> { requestId, status, rules[] } +GET /cli/api/demo/request/{requestId} -> { requestId, status, rules[], examples[] } ``` +**The endpoints take no authentication at all.** We asked that an +unauthenticated call send no `Authorization` header rather than one carrying an +empty token. The service went further: with nothing scoped to a caller there is +nothing for authentication to decide, so the demo endpoints are public, like +`rule-hash-vectors`. There is no header to send and none for the service to +decide how to ignore. Making `submitRule` and `pollRuleStatus` tolerate an +absent token remains ours (task 1.2a); the server half is now a property rather +than a convention. + +**The retrieval also serves what the rule found.** The service's verification +gate already executes the generated `check.ts` against the fixture's failing and +passing examples on its way to deciding whether to accept the rule, and was +discarding the result. `examples[]` carries it: for each fixture example, its +name, whether it is expected to fail or pass, and the `Finding[]` the check +produced against it. + Mirroring is the point. A bespoke demo payload would exercise a path no user is on, which is the one thing a demo must not do. Reusing the well-known formats means the demo also covers polling and the terminal `failed` / `unsupported` states rather than a happy path built for the occasion. -**The demo stops where the gate does, and says so.** A runtime rule executes -only when an authenticated reconcile returns its signature in `run`. The demo -is unauthenticated, so it reaches a complete, well-formed, verified rule on -disk and no further. `check` already explains that state: "not authenticated — -runtime rules were not verified and did not run." +**The demo cannot reach the gate, and that is structural rather than a +choice.** A runtime rule executes only when an authenticated reconcile returns +its signature in `run`, and a signature exists because the rule was recorded +into an organization's corpus — recording is what blessing is. The demo rule is +one fixed rule, generated under a Taskless-owned installation and never recorded +for the caller, so no caller's `run` set can contain it. Authenticating would +not change that. The demo reaches a complete, well-formed, verified rule on disk +and stops, and `check` explains that state in the words it already uses. **No client-side bypass.** Not for a known demo id, not behind a flag of its own. The signature gate is what stands between a downloaded payload and diff --git a/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md b/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md index 3aac5f01..f745bf0f 100644 --- a/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md +++ b/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md @@ -67,6 +67,40 @@ drift without anything detecting it. - **THEN** nothing SHALL be written for that rule - **AND** the failure SHALL name what was wrong with the payload +### Requirement: The demo shows what the rule found, per example + +The retrieval response SHALL carry, for each example in the service's fixture, +that example's name, whether it is expected to fail or pass, and the findings +the generated check produced against it. Findings SHALL use the published +`Finding` shape rather than a demo-only object. + +The CLI SHALL render findings grouped by example, and SHALL report the demo as +failed when the expected asymmetry does not hold: findings against an example +expected to pass, or no findings against an example expected to fail. + +**Rationale.** A flat list of findings renders identically for a rule that +catches the failing examples and a rule that fires on everything it is shown. +The second is the failure this area keeps producing — an indiscriminate or empty +scan reporting as a pass — so the demonstration is the asymmetry, not the count. +Asserting it makes the demo a check rather than a picture. + +#### Scenario: Findings are attributed to the example that produced them + +- **WHEN** the demo renders its result +- **THEN** each finding SHALL be shown under the example it came from +- **AND** each example SHALL state whether it was expected to fail or pass + +#### Scenario: A rule that fires on a passing example fails the demo + +- **WHEN** the response carries a finding against an example expected to pass +- **THEN** the demo SHALL report failure rather than rendering the findings as + a successful demonstration + +#### Scenario: A rule that finds nothing fails the demo + +- **WHEN** the response carries no findings against an example expected to fail +- **THEN** the demo SHALL report failure + ### Requirement: The demo reports a terminal failure rather than inventing a rule The CLI SHALL report a terminal status the service returns for a demo request, @@ -105,6 +139,26 @@ is visible to anyone who reads the recipe. - **THEN** the demo rule SHALL execute under that flag's existing warning, and under no other mechanism +### Requirement: The demo rule cannot be used as a base for improvement + +The CLI SHALL NOT offer the demo rule as an input to rule improvement. An +attempt to improve it SHALL terminate with the service's not-found result and +SHALL write nothing. + +**Rationale.** Improvement resolves a request the caller's organization holds, +and the demo is recorded in no organization, so not-found is the accurate +answer rather than a missing feature. It is also the answer we want on its +merits: a rule pre-generated against a shared fixture is a poor base to iterate +on, and nothing generated in advance can match a rule authored against this +developer's own repository and context. Offering it as a starting point would +lend a worse starting point the demo's credibility. + +#### Scenario: Improving the demo rule is refused + +- **WHEN** rule improvement is invoked against the demo rule +- **THEN** the CLI SHALL report the service's not-found result +- **AND** SHALL write no rule + ### Requirement: What the demo writes can be removed A demo rule that has been written SHALL be removable by rule identifier through diff --git a/openspec/changes/runtime-demo-path/tasks.md b/openspec/changes/runtime-demo-path/tasks.md index 6d5620c2..b9120788 100644 --- a/openspec/changes/runtime-demo-path/tasks.md +++ b/openspec/changes/runtime-demo-path/tasks.md @@ -20,8 +20,9 @@ what design D2 rules out. The mainline took the noun first; the demo follows it. ## 0b. Blocked on the service half - [x] 0.1 Confirm the endpoint shapes with the generator team (**N9**): `POST /cli/api/demo/request` returning `{ requestId, status }`, `GET /cli/api/demo/request/{requestId}` returning `{ requestId, status, rules[] }` with the published file-set variant. Agreed both sides -- [ ] 0.2 Confirm the service can serve the demo endpoints unauthenticated, and get their answer on whether reconcile can bless the sample signature for an anonymous caller. This asks whether the service can do it; design D3 already settles what the demo does either way — it stops at `verify` — so a "no" on the reconcile half is a complete answer and changes nothing below. A "no" on serving unauthenticated at all is the one answer that reopens the design -- [ ] 0.3 Agree what the demo rule is for — the scenario the generated rule addresses — so the demo shows something a person recognises rather than an arbitrary rule +- [x] 0.2 Confirm the service can serve the demo endpoints unauthenticated, and get their answer on whether reconcile can bless the sample signature for an anonymous caller. **Answered, both halves.** The endpoints take no authentication at all — public, like `rule-hash-vectors` — because nothing in the demo is scoped to a caller: no ticket, no corpus entry, no bill, and one Taskless-owned installation behind every request. And no blessing, for the reason D3 now records: blessing is recording, and a shared fixed rule is never recorded for the caller, so authenticating would not reach execution either +- [ ] 0.3 Agree what the demo rule is for — the scenario the generated rule addresses — so the demo shows something a person recognises rather than an arbitrary rule. The fixture now has a second job: its examples are what the served findings are attributed to, so it needs examples that are expected to fail AND examples that are expected to pass +- [ ] 0.4 Tell the service the findings shape we want (D6): `Finding[]` verbatim, grouped per fixture example with that example's name and whether it is expected to fail or pass. `Finding` is our published runtime-rule type, so this is the well-known format rather than a demo-only one **Nothing below starts until 0.0 and 0.1 land.** Building against a guessed shape is what produced the seam this change is closing. @@ -35,6 +36,8 @@ what produced the seam this change is closing. - [ ] 1.3 Write through `writeRuleFile`. If the demo needs its own writer, stop: the shapes have diverged and that is the finding - [ ] 1.4 Report a terminal status by surfacing the service's reason, which `unsupportedMessage` already prefers over our own text - [ ] 1.5 Regenerate `src/generated/api.d.ts` once the endpoints are published +- [ ] 1.6 Render the served findings grouped by example, showing each example's name, whether it was expected to fail or pass, and what the check returned against it +- [ ] 1.7 Assert the asymmetry rather than printing whatever arrives: findings on the examples expected to fail, none on the examples expected to pass. A demo whose rule has stopped finding anything, or that fires on everything, SHALL fail rather than render cleanly (D6) ## 2. The hidden topic @@ -50,16 +53,20 @@ what produced the seam this change is closing. - [ ] 3.1 Test that an unauthenticated `check` skips a demo rule, with the reason it already gives - [ ] 3.2 Test that the demo rule executes under `--dangerously-run-scripts` and under nothing else - [ ] 3.3 Test that no demo rule identifier is special-cased anywhere on the execution path +- [ ] 3.4 Test that `rule improve` against the demo rule terminates as `RULE_NOT_FOUND` and writes nothing. The organization holds no record of it, so this is the correct code rather than a gap; pin it so it cannot regress into a partial improve flow (D7) ## 4. Prove it does not write a broken rule - [ ] 4.1 Test that a delivered demo rule missing `check.ts` or its captures is refused and nothing is written - [ ] 4.2 Test that a terminal `unsupported` surfaces the service's reason and writes nothing - [ ] 4.3 Test the whole path against a mock serving both endpoints, asserting the rule lands and `verify` reports it valid +- [ ] 4.4 Test that a payload whose passing examples carry findings is reported as a failed demonstration, not rendered as success — the indiscriminate-rule case D6 names +- [ ] 4.5 Test that a payload whose failing examples carry no findings is likewise reported as failed, since an empty scan is the shape a silent regression takes ## 5. Prove it can be undone - [ ] 5.1 Test that deleting the demo rule by id removes its directory and that a later `check` does not report it +- [ ] 5.2 Test that an ordinary `check` skips the demo rule through the existing unverified-runtime path, asserting the reason comes from that path rather than from anything the demo added (D7) ## 6. Close out From 76b395272419505ab63fe40fd25665954b4028f2 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Tue, 1 Sep 2026 21:06:03 -0700 Subject: [PATCH 8/8] docs(openspec): retire three claims the code no longer supports Review of the assembled change found three places where the documents assert something that is not true, or not yet agreed. **The skew premise was stale.** D1, D2 and the Risks entry described the `request`/`requestId` rename as landed on the service and pending in our client. It landed in our client too: `submitRule` posts to `/cli/api/request`, `pollRuleStatus` gets `/cli/api/request/{requestId}`, `iterateRule` and `restoreRule` follow, and no hand-written call site targets the `rule/*` family. The reasoning stays and moves to past tense, and the Risks entry now records what the move taught: the types caught none of it, because a deprecated path is still in the OpenAPI document and type-checks as well as the canonical one. A vendored document and `api-deprecated-paths.test.ts` are what close that gap. D1's "the path half resolves itself" is corrected in place - it did not, and the literal being a literal was always the real obstacle. **The proposal overclaimed `examples[]`.** Its retrieval code block showed `{ requestId, status, rules[], examples[] }` as settled while D1 and task 0.1 say `rules[]` and task 0.4 is open. The block is back to the agreed shape, with a sentence saying the findings payload is an open ask. D6 keeps its reasoning and gains a note that its shape is under revision. **`RULE_NOT_FOUND` on `improve` was assumed.** `iterateRule` maps a 404 `request_not_found` to `CLIError(RULE_NOT_FOUND)` but a 403 `access_denied` to a plain `Error`, which `improveCommand` reports as `NETWORK_ERROR`. The demo request exists under a Taskless-owned installation and is not the caller's, which is 403-shaped. D7 and the spec requirement now state the behaviour - improvement refused, nothing written, and a terminal failure rather than one inviting a retry - without asserting a code in either direction. New task 0.5 asks the service which status a foreign-org ticket returns; task 3.4 pins the behaviour now and the code after. --- openspec/changes/runtime-demo-path/design.md | 101 +++++++++++++----- .../changes/runtime-demo-path/proposal.md | 20 ++-- .../specs/cli-runtime-demo/spec.md | 29 +++-- openspec/changes/runtime-demo-path/tasks.md | 7 +- 4 files changed, 113 insertions(+), 44 deletions(-) diff --git a/openspec/changes/runtime-demo-path/design.md b/openspec/changes/runtime-demo-path/design.md index 889945f7..5f7ec9f4 100644 --- a/openspec/changes/runtime-demo-path/design.md +++ b/openspec/changes/runtime-demo-path/design.md @@ -62,17 +62,24 @@ job by failing. That is the signal worth having, and it is the whole reason for mirroring the delivery variant. **`submitRule` and `pollRuleStatus` need work first, and that is a task rather -than a divergence signal.** Both are hardcoded to the rule-family endpoints and -both take a required `token`, which `createApiClient` turns unconditionally into -an `Authorization: Bearer` header. The demo is unauthenticated by requirement -(D3), so neither is usable as it stands: the endpoint has to be a parameter, and -the token has to be optional. - -The path half of that resolves itself when the client adopts the renamed family, -which is out of scope here. The token half does not, and is the durable part. -Neither is evidence of anything having gone wrong — they are two small changes -to functions written when every caller was authenticated, and they are planned -in task 1.2a rather than discovered during implementation. +than a divergence signal.** Both hardcode their endpoint as a literal, and both +take a required `token`, which `createApiClient` turns unconditionally into an +`Authorization: Bearer` header. The demo is unauthenticated by requirement (D3), +so neither is usable as it stands: the endpoint has to be a parameter, and the +token has to be optional. + +**An earlier draft said the path half "resolves itself when the client adopts +the renamed family". It did not, and the correction is worth keeping.** The +client has since adopted it — those two functions now call `/cli/api/request` +and `/cli/api/request/{requestId}` — and they are no less hardcoded for it. The +demo's endpoint is `/cli/api/demo/request` either way, so what stood between the +demo and reuse was never which noun the literal spelled; it was that the literal +is not a parameter. Both halves are ours, and both are planned in task 1.2a +rather than discovered during implementation. + +Neither is evidence of anything having gone wrong. They are two small changes to +functions written when every caller was authenticated and every caller wanted +the same route. The distinction matters because D1's failure test is a **claim about payload shapes**. A requester that needs a second argument says nothing about whether @@ -94,10 +101,12 @@ has landed and is verified live, so the demo takes the settled noun above. Naming the demo endpoints before it landed would have made the demo the debut of the new convention, which is precisely what this decision forbids. -Adopting the new noun in the _ordinary_ client is a separate change and is not -this one's dependency. `/cli/api/rule/*` still serves, marked `deprecated`, so -the mainline client is not broken by having not moved yet — but the demo, being -new, has no reason to be born on the deprecated spelling. +Adopting the new noun in the _ordinary_ client was a separate change and was +never this one's dependency. `/cli/api/rule/*` still serves, marked +`deprecated`, so the mainline client was not broken by having not moved yet, and +the demo, being new, had no reason to be born on the deprecated spelling. That +separate change has since landed as well, so the two agree; the Risks section +records what the move taught. ### D2a — The fixed input is data the service holds, not a repository @@ -202,6 +211,11 @@ that fails, because it would look like success. ### D6 — The findings ship as `Finding[]`, grouped by the example that produced them +**This shape is under revision and is not yet agreed with the service.** Task +0.4 — telling the service what findings shape we want — is open, so what follows +is the ask rather than the contract. The retrieval shape the two sides have +settled is D1's, and it does not include this. + The service offered either its harness's `Finding[]` or something narrower, and left the shape to us since we render it. @@ -255,10 +269,27 @@ demo is special-cased to produce that. teardown, no residue `verify` or `check` reports afterwards. **`improve` does not work on it, and should not.** Improvement resolves a -request the caller's organization holds, and the demo is recorded in no -organization, so it terminates as `RULE_NOT_FOUND` — the correct code, since the -service genuinely has no such record. The deeper reason is worth stating: a -pre-generated rule built against a shared fixture is a bad base to iterate on. +request the caller's organization holds. The demo request is held by a +Taskless-owned installation and by no caller's organization, so improvement is +refused and nothing is written. That behaviour is the part this design rests on. + +**Which status the service returns for it is not settled, and this design no +longer asserts one.** An earlier draft said `RULE_NOT_FOUND`. That is a claim +about the service rather than about us, and the client's own handling makes it +consequential: `iterateRule` turns a 404 `request_not_found` into a `CLIError` +carrying `RULE_NOT_FOUND`, and turns a 403 `access_denied` into a plain `Error`, +which `improveCommand` reports as `NETWORK_ERROR`. A request that genuinely +exists and merely is not yours is 403-shaped rather than 404-shaped. If that is +what the service sends, `improve` on the demo rule reports `NETWORK_ERROR` and +tells an agent to retry an id that will never resolve — the exact +miscategorisation `iterateRule`'s own comment was written to prevent. Asserting +`NETWORK_ERROR` instead would trade one unconfirmed claim for another, so task +0.5 asks the service which status a ticket outside the caller's organization +returns, and the requirement states the behaviour until that answer exists. + +The deeper reason for refusing is worth stating separately, because it holds +whatever the status turns out to be: a pre-generated rule built against a shared +fixture is a bad base to iterate on. Nothing generated in advance can match a rule authored against this developer's own repository and context, so offering the demo rule as a starting point would be offering a worse starting point wearing the demo's credibility. @@ -271,13 +302,31 @@ that would be silent if they broke. ## Risks / Trade-offs -**The demo is born on `request`/`requestId` while the ordinary client is still -on `rule`/`ruleId`.** That skew is deliberate (D2) and bounded: the `rule/*` -family still serves, marked `deprecated`, so nothing is broken by the mainline -not having moved yet. When it does adopt the rename, regenerating the types -catches most of it at compile time; two call sites build their URL as a -template string and would not fail to compile, so they are changed deliberately -rather than found later. +**The demo was born on `request`/`requestId` ahead of the ordinary client, and +that skew has since closed.** It was deliberate (D2) and bounded while it +lasted: the `rule/*` family still serves, marked `deprecated`, so nothing was +broken by the mainline not having moved yet. The mainline has now moved. +`submitRule` posts to `/cli/api/request`, `pollRuleStatus` gets +`/cli/api/request/{requestId}`, `iterateRule` and `restoreRule` follow, and no +hand-written call site targets the `rule/*` family. + +**How that move landed is the part worth keeping, because this section's +expectation was wrong.** It assumed regenerating the types would catch the +typed call sites at compile time, leaving only the two that build their URL as a +template string to change by hand. It caught none of them. A deprecated path is +still IN the OpenAPI document, so `openapi-typescript` emits it as an ordinary +entry and `client.GET("/cli/api/rule/{ruleId}")` type-checks exactly as well as +the canonical spelling; all four typed call sites were reverted and `tsc +--noEmit` passed clean. What closes that gap is a test rather than the type +system: `generate:api` vendors the OpenAPI document next to the types it +generates, and `test/api-deprecated-paths.test.ts` reads that document to fail +if any source file still calls a path it marks deprecated. Deprecation is data +in the schema, not a type error, so catching it takes something that reads the +schema. + +That is the standing lesson for D2. When the mainline is renamed again and the +demo follows it, the rename is not self-enforcing, and the vendored document is +what makes it so. **The service half does not exist yet.** The endpoints are the service team's to build, raised as N9. This change assumes the shapes in D1; if they land diff --git a/openspec/changes/runtime-demo-path/proposal.md b/openspec/changes/runtime-demo-path/proposal.md index 1a6989bd..c5e8423d 100644 --- a/openspec/changes/runtime-demo-path/proposal.md +++ b/openspec/changes/runtime-demo-path/proposal.md @@ -38,9 +38,14 @@ shapes: ``` POST /cli/api/demo/request -> { requestId, status } -GET /cli/api/demo/request/{requestId} -> { requestId, status, rules[], examples[] } +GET /cli/api/demo/request/{requestId} -> { requestId, status, rules[] } ``` +That is the shape both sides have agreed (task 0.1). What the retrieval carries +about what the rule found is a separate question and is still open: the findings +payload is an ask rather than a contract, sketched in design D6 and tracked by +task 0.4, which has not been answered. + **The endpoints take no authentication at all.** We asked that an unauthenticated call send no `Authorization` header rather than one carrying an empty token. The service went further: with nothing scoped to a caller there is @@ -53,9 +58,10 @@ than a convention. **The retrieval also serves what the rule found.** The service's verification gate already executes the generated `check.ts` against the fixture's failing and passing examples on its way to deciding whether to accept the rule, and was -discarding the result. `examples[]` carries it: for each fixture example, its -name, whether it is expected to fail or pass, and the `Finding[]` the check -produced against it. +discarding the result. What we asked for is a per-example carrier: for each +fixture example, its name, whether it is expected to fail or pass, and the +`Finding[]` the check produced against it. That shape is under revision and is +not part of the agreed contract above; task 0.4 tracks settling it. Mirroring is the point. A bespoke demo payload would exercise a path no user is on, which is the one thing a demo must not do. Reusing the well-known formats @@ -106,8 +112,10 @@ hole in it. Executing the demo rule uses the documented - Depends on the service half (raised as **N9** in the cross-team document). The endpoints are theirs to build; the shapes above are what we asked for and what this change assumes. -- Sequenced behind the `request`/`requestId` rename (**N10**), which has now +- Sequenced behind the `request`/`requestId` rename (**N10**), which has shipped and is verified live. The demo takes that noun. Adopting it in the - ordinary client is a separate change and is not this one's dependency. + ordinary client was a separate change, was never this one's dependency, and + has since landed as well: no hand-written call site targets the `rule/*` + family any more. - No change to the runtime gate, to reconcile, or to `check`'s execution policy. diff --git a/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md b/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md index f745bf0f..7e9268b6 100644 --- a/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md +++ b/openspec/changes/runtime-demo-path/specs/cli-runtime-demo/spec.md @@ -142,21 +142,32 @@ is visible to anyone who reads the recipe. ### Requirement: The demo rule cannot be used as a base for improvement The CLI SHALL NOT offer the demo rule as an input to rule improvement. An -attempt to improve it SHALL terminate with the service's not-found result and -SHALL write nothing. +attempt to improve it SHALL terminate with the result the service returns for a +request the caller's organization does not hold, and SHALL write nothing. The +reported failure SHALL be terminal rather than one that invites a retry of the +same identifier. **Rationale.** Improvement resolves a request the caller's organization holds, -and the demo is recorded in no organization, so not-found is the accurate -answer rather than a missing feature. It is also the answer we want on its -merits: a rule pre-generated against a shared fixture is a poor base to iterate -on, and nothing generated in advance can match a rule authored against this -developer's own repository and context. Offering it as a starting point would -lend a worse starting point the demo's credibility. +and the demo is held by a Taskless-owned installation instead, so refusal is the +accurate answer rather than a missing feature. Which status the service sends +for such a request is not yet confirmed (task 0.5), and the two candidates map to +different CLI codes — a 404 to `RULE_NOT_FOUND`, a 403 to `NETWORK_ERROR` — so +this requirement names the behaviour and leaves the code to that answer. The +retry distinction is the part that matters: a transport-shaped failure tells an +agent to retry an identifier that will never resolve. + +Refusal is also the answer we want on its merits: a rule pre-generated against a +shared fixture is a poor base to iterate on, and nothing generated in advance can +match a rule authored against this developer's own repository and context. +Offering it as a starting point would lend a worse starting point the demo's +credibility. #### Scenario: Improving the demo rule is refused - **WHEN** rule improvement is invoked against the demo rule -- **THEN** the CLI SHALL report the service's not-found result +- **THEN** the CLI SHALL report the terminal result the service returns +- **AND** SHALL NOT present it as a failure worth retrying with the same + identifier - **AND** SHALL write no rule ### Requirement: What the demo writes can be removed diff --git a/openspec/changes/runtime-demo-path/tasks.md b/openspec/changes/runtime-demo-path/tasks.md index b9120788..9350aa3b 100644 --- a/openspec/changes/runtime-demo-path/tasks.md +++ b/openspec/changes/runtime-demo-path/tasks.md @@ -10,7 +10,7 @@ The generator team renamed the request resource first, as its own change: `openspec/specs/cli/spec.md` already specified it, and their live `meta` block already carries `ticketId` per delivered rule, so the route was the outlier. -- [x] 0.0 Wait for the rename to land and the new schema to publish. **Done**: verified from `GET /cli/api/__schema` that `/cli/api/request/*` is canonical and the `rule/*` family is `deprecated: true` while still serving. Adopting it in our client is its own change, tracked separately +- [x] 0.0 Wait for the rename to land and the new schema to publish. **Done**: verified from `GET /cli/api/__schema` that `/cli/api/request/*` is canonical and the `rule/*` family is `deprecated: true` while still serving. Adopting it in our client was its own change, tracked separately, and it has since landed - [x] 0.0a The demo therefore takes the settled noun: `POST /cli/api/demo/request` and `GET /cli/api/demo/request/{requestId}` The demo did not start before that, which was the point. Naming these endpoints @@ -23,6 +23,7 @@ what design D2 rules out. The mainline took the noun first; the demo follows it. - [x] 0.2 Confirm the service can serve the demo endpoints unauthenticated, and get their answer on whether reconcile can bless the sample signature for an anonymous caller. **Answered, both halves.** The endpoints take no authentication at all — public, like `rule-hash-vectors` — because nothing in the demo is scoped to a caller: no ticket, no corpus entry, no bill, and one Taskless-owned installation behind every request. And no blessing, for the reason D3 now records: blessing is recording, and a shared fixed rule is never recorded for the caller, so authenticating would not reach execution either - [ ] 0.3 Agree what the demo rule is for — the scenario the generated rule addresses — so the demo shows something a person recognises rather than an arbitrary rule. The fixture now has a second job: its examples are what the served findings are attributed to, so it needs examples that are expected to fail AND examples that are expected to pass - [ ] 0.4 Tell the service the findings shape we want (D6): `Finding[]` verbatim, grouped per fixture example with that example's name and whether it is expected to fail or pass. `Finding` is our published runtime-rule type, so this is the well-known format rather than a demo-only one +- [ ] 0.5 Ask the service which status `iterate` returns for a request outside the caller's organization: 404 `request_not_found` or 403 `access_denied`. It decides what `rule improve` against the demo rule reports — `iterateRule` maps the 404 to a `CLIError` carrying `RULE_NOT_FOUND` and the 403 to a plain `Error`, which `improveCommand` reports as `NETWORK_ERROR`, telling an agent to retry an id that will never resolve. The demo request exists under a Taskless-owned installation and is not the caller's, which is 403-shaped. D7 and the spec state the behaviour rather than a code until this is answered **Nothing below starts until 0.0 and 0.1 land.** Building against a guessed shape is what produced the seam this change is closing. @@ -31,7 +32,7 @@ what produced the seam this change is closing. - [ ] 1.1 Add the demo endpoints to `src/api/`, reusing the existing client rather than a second fetch layer - [ ] 1.2 Drive them through `submitRule`/`pollRuleStatus`, so polling and the terminal `failed`/`unsupported` states are the same code an ordinary generation uses -- [ ] 1.2a Make that reuse possible first: `submitRule` and `pollRuleStatus` are hardcoded to the rule-family endpoints and take a required `token` that `createApiClient` turns into an `Authorization` header unconditionally. Parameterize the endpoint and make the token optional. **Not a divergence signal** — see D1; these were written when every caller was authenticated, and the demo is the first that is not +- [ ] 1.2a Make that reuse possible first: `submitRule` and `pollRuleStatus` hardcode their endpoint as a literal (now `/cli/api/request` and `/cli/api/request/{requestId}`, since the client adopted the rename) and take a required `token` that `createApiClient` turns into an `Authorization` header unconditionally. Parameterize the endpoint and make the token optional. **Not a divergence signal** — see D1; these were written when every caller was authenticated, and the demo is the first that is not - [ ] 1.2b Confirm an unauthenticated call sends no `Authorization` header at all, rather than one with an empty token. A header the service must then decide how to ignore is a worse contract than its absence - [ ] 1.3 Write through `writeRuleFile`. If the demo needs its own writer, stop: the shapes have diverged and that is the finding - [ ] 1.4 Report a terminal status by surfacing the service's reason, which `unsupportedMessage` already prefers over our own text @@ -53,7 +54,7 @@ what produced the seam this change is closing. - [ ] 3.1 Test that an unauthenticated `check` skips a demo rule, with the reason it already gives - [ ] 3.2 Test that the demo rule executes under `--dangerously-run-scripts` and under nothing else - [ ] 3.3 Test that no demo rule identifier is special-cased anywhere on the execution path -- [ ] 3.4 Test that `rule improve` against the demo rule terminates as `RULE_NOT_FOUND` and writes nothing. The organization holds no record of it, so this is the correct code rather than a gap; pin it so it cannot regress into a partial improve flow (D7) +- [ ] 3.4 Test that `rule improve` against the demo rule terminates without writing anything, and reports a terminal failure rather than one that invites retrying the same id. Pin the behaviour so it cannot regress into a partial improve flow (D7). Pin the specific code once 0.5 answers which status the service sends; asserting one before that would encode a guess ## 4. Prove it does not write a broken rule