Make docs/knowledge/ queryable: requirement-ID lookup CLI, generated ID map, and a repo skill

[Wahbeh-Mohammad]

Problem

docs/knowledge/ holds 518 KB across 39 topic files roughly 130k tokens. Nothing loads it before implementation starts, so the harvested styleguide and spec knowledge is effectively write-only. Reading a single topic costs 20–27 KB (pipeline.md is 27 KB / 83 entries) to reach one relevant rule.

The sharper gap: CLAUDE.md says "before implementing anything, find the requirement IDs it must satisfy," but the corpus has no requirement-ID index. 451 distinct IDs (HTTP-7, XCUT-12, SEAM-29, …) appear inside entry text, and INDEX.md maps topic → file only. Going from an ID to the knowledge that governs it means grepping blind or reading whole files.

The corpus is already well-structured enough to query precisely it just has no query surface:

• Every topic file uses the same six sections: Rules, Constraints, Conclusions, Reference, Conflicts, Superseded.
• Every entry is exactly one - bullet plus one <sub> provenance line carrying role, source path:lines, confidence, and source sha.
• 173 KB (33% of the corpus) is those provenance lines valuable for citation, pure overhead when you just want the rule.

Who this is for

Primary consumer: plan execution. The phase workflow in CLAUDE.md says to read the design spec, the implementation plan, and the coverage checklist before touching code and each numbered task in a plan names the requirement IDs it satisfies. That is exactly the moment the knowledge corpus should be consulted and currently is not, because pulling in the governing knowledge means loading a 20 KB topic file per task.

This tool exists to make that lookup cost ~200 tokens instead of ~6k, so it actually happens on every task rather than getting skipped.

Explicit non-goal: this is not a CI gate. Nothing here blocks a build. It is a lookup tool used while writing code.

Proposed work

1. scripts/knowledge.mjs query CLI

Zero-dependency Node ESM, matching the existing scripts/verify-*.mjs convention. Parses docs/knowledge/*.md into ~1470 entry records ({file, line, section, text, role, source, confidence, sha}) by walking lines and tracking the current ## Section.

flag	effect
--req HTTP-7 (repeatable)	entries citing that requirement ID
--topic pipeline,retry	restrict to files (substring match)
--section rules,constraints	filter to one or more of the six sections
--role spec|design|styleguide	filter by provenance role
--grep <regex> / bare words	case-insensitive text match
--brief	drop <sub> provenance lines (−33% tokens)
--json	machine-readable output
--list-reqs	print the full requirement ID → location map to stdout
--coverage	report canonical IDs with no knowledge entry (see D1)

Filters AND together. Zero matches exits 1 with nearest-ID and topic suggestions rather than failing silently.

Target: --req HTTP-7 returns ~2 entries / ~180 tokens, against a 20 KB whole-file read today. 81% of IDs resolve to exactly one location.

2. Wiring

• package.json: "knowledge": "node scripts/knowledge.mjs".
• CLAUDE.md: ~8 lines under the documentation-hierarchy table with three example invocations, and a line in the Phase workflow section pointing at the skill. CLAUDE.md loads every session; the 518 KB corpus never will.
• No .github/workflows/ci.yml change. Deliberate — see "Who this is for".

3. Local repo skill, .claude/skills/knowledge-lookup/

The skill is what makes this get used, so it carries as much weight as the CLI. It should cover:

• Trigger conditions starting any numbered task in a docs/superpowers/plans/ file, implementing against any requirement ID, or hitting a styleguide citation ("styleguide 6.7", "ch08").
• Two equal entry points, chosen by what the caller already knows:
  • ID-first appendix-c to locate the ID → knowledge --req <ID> for the governing knowledge → docs/sdk-design-nodejs/ for the TypeScript mapping.
  • Topic-first knowledge --topic <area> --section rules --brief. See D3: ID-first alone reaches only a fifth of the corpus.
• The prohibition never cat a whole topic file when a filtered query answers the question. This is the behavior the tool exists to change.
• Citing results back the <sub> line already carries source path, line range, and sha, which is what test-file header comments and deferral notes need.

Testing

scripts/knowledge.test.mjs run via node --test scripts/, kept outside Bun's 80% coverage floor. Cases: entry parsing including multi-line bullets, section attribution, ID tokenization (HTTP-7 must not match HTTP-70), filter AND-ing, and --coverage arithmetic.

Out of scope

• Any CI step. Stated above, repeated here so it does not get added back by reflex.
• A committed BY-REQUIREMENT.md index file. --list-reqs prints the same map on demand and is always correct; a committed generated file would go stale silently with no gate to catch it, and this repo already carries more generated docs than it needs.
• Per-topic "brief" digest files the --brief flag covers the same need without 39 more files to keep in sync.
• Building a knowledge graph over the corpus the flat structure is regular enough that grep-class tooling suffices.
• Any change to the harvest format itself. docs/knowledge/ stays byte-compatible so re-running the harvest keeps working.

Open decisions

Measurements below come from parsing the corpus and docs/product-spec/appendix-c-consolidated-normative-requirement-index.md directly.

D1 What to do about the 194 uncovered requirement IDs

Every ID in the knowledge corpus already exists in appendix-c {zero inventions} so appendix-c is a valid strict oracle. Checking against it today: 194 of appendix-c's 645 canonical IDs have no knowledge entry citing them.

prefix	uncovered	prefix	uncovered
RETRY	41	CTX	20
AUTH	36	IO	5
RECOV	29	SEAM	5
PIPE	27	PAGE / SSE / BODY	2 each
REDIR	25

Six subsystems account for 178 of the 194. Those subsystems are not under-harvested retry-and-resilience.md holds 68 entries and authentication.md 49. The entries are there; they simply do not cite their requirement IDs. This is a citation gap in the harvest output, not missing knowledge.

It matters directly for the primary use case: a plan task naming RETRY-12 gets zero results from --req, and the fallback is the 22 KB whole-file read this tool was built to avoid. Retry, auth, redirect, and pipeline are exactly the subsystems the upcoming phases cover.

Since there is no CI gate, this is not a blocking-vs-advisory question it is a scoping question:

1. Ship --coverage as a report only. The number is visible to anyone who runs it, and the gap gets fixed whenever someone gets to it.
2. Ship the report and file the citation gap as its own issue, so it is tracked rather than discovered.
3. Fix the citations as part of this issue re-annotate ~194 entries across six topic files against appendix-c.

Recommendation: (2). The annotation pass is real work with its own review burden and should not ride along with tooling. But it should be a tracked issue, not a paragraph in this one, and the skill should warn that RETRY/AUTH/RECOV/PIPE/REDIR/CTX lookups may need --topic instead of --req until it lands.

D2 Where does the ID tokenizer get its prefix allowlist?

A naive \b[A-Z]{2,12}-\d+\b over the corpus yields 453 IDs, but two are false positives: UTF-8 (8 occurrences) and a SHA- hit. Left unfixed, --req UTF-8 becomes a valid-looking query and --list-reqs prints junk rows.

Options: hardcode a denylist (UTF, SHA, RFC, ISO), hardcode an allowlist of the 18 real prefixes, or derive the allowlist from appendix-c at runtime. The third is self-maintaining a new requirement family added to the spec starts being recognized with no code change and it costs one extra file read. It also makes D1's --coverage report fall out of the same parse.

Recommendation: derive from appendix-c. Fail loudly if that file cannot be parsed, rather than silently falling back to a regex that admits UTF-8.

Tokenization must also be exact-token: --req HTTP-7 must not match HTTP-70. That belongs in the test suite regardless of which option wins.

D3 What serves the 79% of entries that carry no ID?

Only 310 of 1470 bullets cite a requirement ID. ID-first lookup reaches a fifth of the corpus; the styleguide-derived topics (naming-conventions, function-design, variables-and-declarations, performance, testing) carry almost no IDs at all, because the styleguide numbers its rules by chapter and section, not by requirement.

Two consequences to decide on:

• Is --topic + --section a first-class path, or a fallback? It has to be first-class. "Load the Rules section of naming-conventions" is a legitimate, common query, and --section rules --brief on that file is ~4 KB against 10 KB raw. The skill should present ID-first and topic-first as two equal entry points, not a primary and a fallback.
• Should styleguide entries get a citation handle of their own? The <sub> line already carries styleguide/typescript/06-classes-and-data-modeling.md:NN, and CLAUDE.md already cites rules as "styleguide 6.7". A --cite 6.7 flag resolving chapter.section against those source paths would give styleguide knowledge the same precision --req gives spec knowledge. My read is follow-up, once the base tool proves out but worth deciding explicitly rather than by omission.

D4 Reconcile the two entry counts

INDEX.md's per-topic entries column sums to 1682. Parsing - bullets across the same files yields 1470. The 212-entry gap is unexplained plausibly multi-line bullets counted differently, or entries merged and superseded across harvest runs without the index being updated.

Whichever number is right, the CLI will report one of them, and anyone comparing it against INDEX.md will hit the discrepancy immediately. Decide whether to reconcile as part of this work or open it separately; either way, it should not be discovered by someone debugging the CLI.