Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "pathmode",
"displayName": "Pathmode",
"version": "0.1.44",
"version": "0.1.45",
"description": "Run a deterministic preflight on your intent before an agent builds: /preflight scores six calibrated gates and names the exact blockers, no model call, no key. Pathmode also interrogates the intent behind a request (compile-intent), writes it to intent.md in your repo, and checks the code that comes back against the outcomes and constraints you agreed to. Free: uses the models you already have access to in Claude Code. Connect a Pathmode workspace to sync intent and evidence across a team.",
"author": {
"name": "Pathmode",
Expand Down
2 changes: 1 addition & 1 deletion .mcp.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"mcpServers": {
"pathmode": {
"command": "npx",
"args": ["-y", "@pathmode/mcp-server@1.37.0"],
"args": ["-y", "@pathmode/mcp-server@1.38.0"],
"env": {
"PATHMODE_API_KEY": "${user_config.api_key}"
}
Expand Down
11 changes: 6 additions & 5 deletions CALIBRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Three things get confused here, and they are not the same claim:
3. **Whether the spec describes the right product decision.** **Nothing here measures this**, and
no corpus of this shape could. A spec can score 6/6 and be a bad idea, clearly stated.

Part A is a **regression baseline, not independent validation.** These 111 fixtures were written
Part A is a **regression baseline, not independent validation.** These 192 fixtures were written
alongside the rules they score, and the rules were tuned until they agreed. A high score is
therefore evidence that the gate still behaves as it did when it was calibrated. It is not
evidence that the gate generalizes. Independently collected, previously unseen specs would be
Expand Down Expand Up @@ -76,7 +76,7 @@ Each case records **where its defect actually shipped**, and they are not all eq
| Case | Provenance |
|---|---|
| prose sections are content, not absence | shipped in `<= 1.26.3`, fixed in `1.27.0` |
| non-English is unconfirmed, never absent | the known limitation the four-state verdict reports honestly |
| an unread language is unconfirmed, never absent | the known limitation the four-state verdict reports honestly |
| `## Why` is the objective, `## What` is not the outcomes | shipped in `<= 1.36.1`, fixed in `1.37.0` |
| italic text the author wrote is content | never released; introduced and fixed between two commits |
| a subsection heading is not a verification check | never released; same |
Expand Down Expand Up @@ -106,9 +106,10 @@ nothing was written.

## Known limitations, stated plainly

- The gates match a **fixed English vocabulary**. A non-English objective cannot be confirmed, and
reports as `unconfirmed`. Part B has a Finnish case that holds this behaviour honest.
- 111 fixtures is small, and they are **our own authored examples**, not sampled from real repos.
- The gates match a **fixed English and Finnish vocabulary**. An objective in any other language
cannot be confirmed, and reports as `unconfirmed`. Part B has a German case that holds this
behaviour honest.
- 192 fixtures is small, and they are **our own authored examples**, not sampled from real repos.
- The gates judge **phrasing**, not correctness, novelty, feasibility, or whether the decision is
a good one.
- Part A cannot detect a document-parsing regression, as shown above.
Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,9 @@ For workspace API access without a local intent, you can still create an API key

## What's bundled

**MCP server** — `@pathmode/mcp-server@1.37.0`, pinned so the plugin skills and server tool contract update together. Local mode with no key; cloud mode with one.
**MCP server** — `@pathmode/mcp-server@1.38.0`, pinned so the plugin skills and server tool contract update together. Local mode with no key; cloud mode with one.

**Check the gate yourself** — `node scripts/readiness-suite.mjs` runs the pinned server's preflight over 111 labelled field fixtures and a set of whole `intent.md` documents, and prints where it disagrees. Read [CALIBRATION.md](CALIBRATION.md) first: the field score is a regression baseline, not an accuracy claim.
**Check the gate yourself** — `node scripts/readiness-suite.mjs` runs the pinned server's preflight over 192 labelled field fixtures and a set of whole `intent.md` documents, and prints where it disagrees. Read [CALIBRATION.md](CALIBRATION.md) first: the field score is a regression baseline, not an accuracy claim.

**Command** — `/preflight` runs the deterministic six-gate readiness check and always ends in a verdict: on your `intent.md` if one exists, on a spec described in the conversation, or on a provisional draft it builds from context with assumptions marked.

Expand Down Expand Up @@ -109,9 +109,9 @@ completely silent in repos with no intent. If you would rather it did not run, r
against. That tuning set is in this repo, mirrored from the monorepo that owns it, so you can check
it rather than take the number on faith.

[`readiness-corpus.json`](./readiness-corpus.json) holds 111 hand-labeled spec fragments, 58 labeled
`good` and 53 labeled `vague`, spread across the gates: 24 objectives, 37 outcomes, 20 titles,
18 constraints, 12 verification checks. Each item carries the text, the label, and the tags that
[`readiness-corpus.json`](./readiness-corpus.json) holds 192 hand-labeled spec fragments (121 English, 71 Finnish),
101 labeled `good` and 91 labeled `vague`, spread across the gates: 44 objectives, 78 outcomes,
27 titles, 27 constraints, 16 verification checks. Each item carries the text, the label, and the tags that
explain the call:

```json
Expand Down
6 changes: 3 additions & 3 deletions fixtures/documents.json
Original file line number Diff line number Diff line change
Expand Up @@ -68,10 +68,10 @@
}
},
{
"name": "non-English is unconfirmed, never absent",
"name": "an unread language is unconfirmed, never absent",
"file": "non-english-prose.md",
"why": "The gates match a fixed English vocabulary, so they cannot confirm this objective or constraint. That is a limit of the gate, and the verdict has to say so: 'unconfirmed' means it read the text and its word lists could not confirm it, which is different from claiming nothing was written. An audit of real specs found this class of failure is usually ours, not the author's.",
"provenance": "the known limitation the four-state verdict exists to report honestly",
"why": "The gates match a fixed English and Finnish vocabulary, so they cannot confirm this German objective or constraint. That is a limit of the gate, and the verdict has to say so: 'unconfirmed' means it read the text and its word lists could not confirm it, which is different from claiming nothing was written. An audit of real specs found this class of failure is usually ours, not the author's.",
"provenance": "the known limitation the four-state verdict exists to report honestly; the sample was Finnish until 1.38.0 started grading Finnish",
"expect": { "objective": "unconfirmed", "constraints": "unconfirmed" }
},
{
Expand Down
8 changes: 4 additions & 4 deletions fixtures/documents/non-english-prose.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
title: "Parempi hakukokemus"
title: "Bessere Sucherfahrung"
---

# Parempi hakukokemus
# Bessere Sucherfahrung

## Objective
Käyttäjät eivät löydä tuotteita hakutoiminnolla, koska haku ei ymmärrä taivutusmuotoja.
Nutzer finden keine Produkte über die Suche, weil die Suche keine Flexionsformen versteht.

## Constraints
Ei saa hidastaa sivun latausta.
Darf die Seite nicht verlangsamen.
Loading
Loading