Make docs/knowledge/ queryable: lookup CLI and repo skill - #35
Open
Wahbeh-Mohammad wants to merge 2 commits into
Open
Conversation
The corpus is 39 topic files and ~1470 entries. Answering "what do we
already know about RETRY-13" meant grepping blind or reading a whole
20 KB file; there was no requirement-ID index and no query surface.
scripts/knowledge.mjs parses the corpus into entry records and filters
them by requirement ID, topic, section, provenance role, styleguide
chapter, and text. Different filters AND together; values within one
filter OR. A requirement-ID query runs ~120-580 tokens against a topic
file of ~1800-5200.
Two behaviours are load-bearing rather than incidental:
- The ID prefix allowlist is derived from appendix C at runtime, never
hardcoded. A bare \b[A-Z]{2,12}-\d+\b also claims UTF-8, SHA-256,
ISO-8601 and RFC-3986; parsing failure throws rather than falling
back to that regex. Matching tokenizes then compares whole tokens, so
--req HTTP-7 cannot match HTTP-70.
- Appendix B is the conformance checklist: its entries roll several IDs
into one sentence and state none of them. 256 of the 641 cited IDs
resolve only to a roll-up, which exits 0 and reads as answered. Those
results are tagged [appendix-B roll-up], a --req answered entirely by
them warns, and --coverage reports substantive (385) separately from
roll-up-only (256) and uncited (4).
The skill carries the workflow the CLI alone cannot: run --section
conflicts once per phase (6 entries corpus-wide, where design-vs-
styleguide contradictions are recorded resolved or open); pass a task's
whole ID set in one call; check a hit is not a roll-up before trusting
it; reach the 16 topics that carry no requirement ID via --topic or
--chapter. It also records that styleguide <sub> paths are absolute to
a sibling repo and need their machine prefix stripped before being used
as a citation.
Tests run under node --test, not bun test: bunfig.toml scopes discovery
to packages so the 80% coverage floor stays a statement about
packages/core rather than about repo tooling.
No CI step. --coverage is a report run by hand, and --list-reqs prints
to stdout rather than generating another doc.
194 of appendix C's 645 IDs had no entry naming them, concentrated in six subsystems: RETRY 41, AUTH 36, RECOV 29, PIPE 27, REDIR 25, CTX 20. That was a citation gap, not a knowledge gap -- retry-and-resilience.md already held 68 entries and authentication.md 49; the entries simply did not name the IDs they govern, so `--req RETRY-12` found nothing on the subsystems most in need of lookup. Each entry's <sub> line records the spec file and line range it was harvested from, and those lines carry the bolded requirement IDs, so candidates were derived from the source rather than guessed, then matched one by one against appendix C's per-ID text. RETRY, AUTH, PIPE, REDIR and CTX now reach full coverage. Four IDs were left alone as genuine knowledge gaps rather than annotated to make the number look better: RECOV-32 (idempotency-key step), RECOV-33 (client- identity step), RECOV-34 (retry-config validation) and SEAM-28 (the projection's operation identifier) have no entry stating their content. RECOV-17..34 are defined only in appendix C -- no numbered chapter body carries them -- so they restate the recovery stack's contract that chapter 09 expresses as RETRY-*. Those are annotated as pairs, e.g. "(RETRY-27 / RECOV-20)". Structure is unchanged and stays byte-compatible with what /knowledge-harvest emits: 1470 bullets, 1470 <sub> lines, 234 sections. Every one of the 234 changed lines is a "- " bullet; no provenance line, source path, line range or sha was touched. Note that this is a hand-authored pass, not generated output -- nothing regenerates it, and a future re-harvest of an annotated topic file would overwrite these citations. CLAUDE.md's knowledge-query section is included here because its substantive-vs-roll-up figures (385/256/4) only read true once this pass has landed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #33.
Summary
docs/knowledge/holds 39 topic files and about 1470 entries. There was no wayto query them. To find one requirement, you had to grep blind or read a full file.
This pull request adds a query tool, a skill that drives it, and requirement-ID
citations in the corpus.
What is new
scripts/knowledge.mjsfilters entries by requirement ID, topic, section, role,styleguide chapter, and text. Run it with
bun run knowledge..claude/skills/knowledge-lookup/tells you when and how to query.A requirement-ID query costs 120 to 580 tokens. The topic file that holds the
entry costs 1800 to 5200.
Two important behaviors
The tool reads the ID prefix list from appendix C at run time. It does not
hardcode the list, because a simple pattern also matches UTF-8 and SHA-256.
If appendix C does not parse, the tool stops. It does not fall back. The tool
compares full tokens, so --req HTTP-7 does not match HTTP-70.
The tool marks appendix-B roll-ups. Appendix B is the conformance checklist. Each
of its entries names three to five IDs but states none of them. 256 of the 641
cited IDs resolve only to a roll-up, and such a result looks correct. The tool now
tags these entries and prints a warning. --coverage reports 385 substantive IDs,
256 roll-up-only IDs, and 4 uncited IDs.
Requirement-ID citations
194 IDs had no entry that cited them. RETRY, AUTH, PIPE, REDIR, and CTX now have
full coverage. Four IDs stay uncited because no entry states their content:
RECOV-32, RECOV-33, RECOV-34, and SEAM-28.
Only bullet lines changed. No provenance line, source path, line range, or SHA
changed. The corpus keeps 1470 bullets, 1470 lines, and 234 sections.
This is a hand-authored pass. Nothing regenerates it. A future re-harvest of an
annotated topic file would overwrite the citations.
Tests and gates
node --test 'scripts/*.test.mjs' runs 25 tests. bunfig.toml limits bun test
to packages, so the 80% coverage floor stays a statement about packages/core.
Lint, typecheck, and both test suites pass. No workflow runs the tool. --coverage
is a report you run by hand.
Two notes on the title: it drops the "generated ID map" from the issue title, because that was settled as stdout-only (
--list-reqs) with no file written. The branch name still carries the original wording.