Document every command, and check that we did - #18
Conversation
`convert` shipped in 0.6.0 with a section in COMMANDS.md that the table of contents never listed. The edit that was supposed to add it matched on "- [`split`](#split)" with backticks; the file writes "- [split](#split)" without them, so neither the replacement nor its fallback fired and nothing said so. The app help string -- the first line of `adata --help` -- named neither `convert` nor `create`. `create` has been missing from it since it was added, which is the clearest sign this needed a test rather than more care. The existing docs suite only walks one direction: everything the docs claim must exist. That catches a documented flag that was renamed, and is blind to a command nobody documented. Three checks now walk the other way, over the Click tree rather than a hardcoded list, so a command added tomorrow is covered without anyone remembering: each visible command needs a section in COMMANDS.md, an entry in its table of contents, and a mention in the app help. Against the previous state they fail three times -- convert in the contents, convert and create in the help -- and nowhere else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2f713aed81
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| import typer | ||
|
|
||
| help_text = typer.main.get_command(app).help or "" | ||
| assert name in help_text, ( |
There was a problem hiding this comment.
When a future visible command has a name that is already a substring elsewhere in the prose, this test passes even if the command is omitted from the parenthesized list. For example, adding a store command without updating the help succeeds because store matches stores, contradicting the stated guarantee that commands added tomorrow are covered. Parse the listed commands or use delimiter-aware whole-name matching instead.
Useful? React with 👍 / 👎.
You were right —
convertwas under-documented, and the gap was wider than it looked.What was missing
docs/COMMANDS.mdtable of contentsconvertadata --help, first line)convertandcreateThe
## convertsection itself was present, as were the README overview and the index "At a glance" — so it read as documented right up until you went looking for it in the contents.The ToC edit that should have added it matched on
- [`split`](#split)with backticks; the file writes- [split](#split)without them. Neither the replacement nor its fallback fired, and nothing failed.createhas been absent from the app help since it was added — which is the clearest sign this wanted a test rather than more care from me.Why the existing suite missed it
test_docs_are_accurate.pywalks one direction only: everything the docs claim must exist. That catches a documented flag that got renamed. It is structurally blind to a command nobody documented.Three checks now walk the other way, derived from the Click tree rather than a hardcoded list, so a command added tomorrow is covered without anyone remembering to update a list:
## \name`` section in COMMANDS.mdAgainst the previous state they fail exactly three times —
convertin the contents,convertandcreatein the help — and nowhere else.Verification
995 tests pass. Same-direction coverage as before is unchanged; this only adds the inverse.
🤖 Generated with Claude Code