Skip to content

docs(contributing): the make vermin trap is the opposite of what was recorded - #659

Merged
JarryShaw merged 1 commit into
mainfrom
docs/contributing-vermin-and-precision-fixes
Sep 22, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
docs/contributing-vermin-and-precision-fixes

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of this pull request?

  • docs — documentation only

Follow-up to #656, which merged while the cross-review of it was still running. Four corrections to
the section #656 rewrote: one is a claim that is actively wrong and inverts what a contributor
should do
, three are precision fixes the cross-review raised.

1. make vermin fails on violations — the opposite of what #656 said

#656 shipped this, and it is false:

The status you get back is that viewer's rather than vermin's, so a run that found violations still
looks like a success.

That sentence was not invented for #656 — it is what Makefile:152-157 and lint.yml:150-152 both
assert, and it was carried across in good faith. The reasoning behind it looks at the shell semantics
of command -v code >/dev/null && code temp/vermin.txt || cat temp/vermin.txt and correctly concludes
that line always exits 0. What it misses is that the redirect is a separate recipe line, and make
abandons a target at the first line that exits non-zero, so the viewer line is never reached at all.

Measured on this tree, not reasoned about:

$ PATH=.venv/bin:$PATH make vermin RUN=
mkdir -p temp
vermin pcapkit --backport argparse ... pcapkit > temp/vermin.txt
make: *** [vermin] Error 1
MAKE_EXIT=2
$ vermin pcapkit --backport argparse ... pcapkit      # the same invocation, un-redirected
Minimum required versions: 3.11
Incompatible versions:     2.x
Target versions not met:   3.6
VERMIN_EXIT=1

temp/vermin.txt is written (365 lines) but never displayed, and REACHED_VIEWER_LINE in a minimal
reproduction of the same three-line recipe is never printed. A control where the tool succeeds does
reach the viewer line and exits 0 — which is what isolates the redirect, rather than the A && B || C
chain, as the deciding factor. GNU Make 3.82, no .ONESHELL: anywhere in the Makefile.

vermin exits 1 today because vermin.ini sets targets = 3.6 while the code's real floor is 3.11.

Why this matters more than the wording: a contributor who believed the old text would see a red
make vermin, remember being told it cannot fail on violations, and dismiss a genuine failure as a
known false pass. The trap is real but inverted — the failure propagates and the findings are what
get hidden, in a file the target never opens.

2. Three precision fixes from the cross-review of #656

  • The file contradicted itself four lines apart. docs(contributing): correct four statements the recent merges falsified #656's CI paragraph claimed the job runs "the
    same Makefile targets listed above", but lint.yml:166 runs make vermin-ci, which is not in
    that block — and the paragraph further down admitted it. It now says which target each tool takes,
    and rests the no-drift claim on $(VERMIN_FLAGS) and friends being shared between vermin and
    vermin-ci, which is the thing that actually holds.
  • "every step writes its count there" is wrong for vermin: lint.yml:173 emits
    Minimum required versions: / Incompatible versions: / Target versions not met:, which are
    version verdicts, not counts. Now "each step writes its own verdict there".
  • Changelog drift has a third trigger. The job carries no gate-only guard, unlike its sibling
    jobs, so it also runs on every workflow_call into unit-tests.ymlcron-vendor.yml:27,
    cron-conda.yml:29, deploy-pages.yml:37 and create-release.yml:40 all gate on it.

Notes

  • Makefile:152-157 and lint.yml:150-152 still carry the wrong claim. Deliberately not fixed
    here: .github/workflows/** should not be touched ahead of the 1.5.0b5 release, and the two
    comments want their own change. Worth doing, because they are what this documentation was derived
    from in the first place — the error will propagate again otherwise.
  • No changelog entry, deliberately — not user-visible, and a bullet would collide with the
    changelog consolidation in flight.
  • No file outside CONTRIBUTING.md is modified. The test suite was not run; nothing here executes
    library code.
  • One commit, authored Jarry Shaw <jarryshaw@icloud.com>, on top of 7b0df4a9c.

…s recorded

#656 said `make vermin`'s exit status is the viewer's, so a run that found
violations still looks like a success. That is what `Makefile:152-157` and
`lint.yml:150-152` both assert, and it is wrong: the redirect into `temp/` is its
own recipe line, so make gives up there and never reaches the line that would
open the report. Measured on this tree -- `make vermin RUN=` exits 2 with
`*** [vermin] Error 1`, vermin itself exits 1 on `Target versions not met: 3.6`
against a real floor of 3.11, and the 365-line report is left unread in
`temp/vermin.txt`. A control run with a succeeding command does reach the viewer
line, which is what makes the redirect the deciding factor rather than the shell
semantics of `A && B || C`.

So the trap is real but inverted: the failure propagates and the *findings* are
what get hidden. A contributor told the old version would ignore a red `make
vermin` as a known false pass.

Three precision fixes in the same section, from a cross-review of #656:

* The CI paragraph claimed the job runs "the same `Makefile` targets listed
  above". It runs `vermin-ci` for vermin, which is not one of them, and the
  paragraph four lines later said so -- the file contradicted itself. It now
  names which target each tool takes, and rests the no-drift claim on the shared
  flag variables, which is what actually holds.
* "every step writes its count" was wrong for vermin, whose summary line is a
  version verdict rather than a count.
* `Changelog drift` also runs on every `workflow_call` into `unit-tests.yml` --
  the vendor and conda updates, the pages deploy and the release all gate on it
  -- because that job carries no `gate-only` guard.

The two stale comments in `Makefile` and `lint.yml` are deliberately left alone:
a workflow edit ahead of the 1.5.0b5 release is not worth the risk, and they
want their own change.

No changelog entry, deliberately: none of this is user-visible, and a bullet
here would collide with the changelog consolidation currently in flight.
@JarryShaw

Copy link
Copy Markdown
Owner Author

The cross-review verdict that produced this PR is on #656: #656 (comment)

It opened NEEDS CHANGES against #656 and raised items 1-3 of the section above. Item 1 here — the make vermin inversion — is not from the review: the review marked that claim CONFIRMED, agreeing with the false version. It was found separately by testing the target instead of reading it.

Unpublished-equivalent status: this PR is open and awaiting your review. Nothing here has been merged, tagged or released.

@JarryShaw JarryShaw added the docs Pull requests that change documentation only (docs: subject prefix) label Sep 22, 2026
@JarryShaw
JarryShaw merged commit 84d05a6 into main Sep 22, 2026
12 of 25 checks passed
@JarryShaw
JarryShaw deleted the docs/contributing-vermin-and-precision-fixes branch September 22, 2026 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Pull requests that change documentation only (docs: subject prefix)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant