fix(asvs): survey progress counted a read-and-parked cell as never read - #150
Merged
Conversation
`ASVS-CURRENT.md` reported "123 of 345 requirements have been read ... 222 have not" directly above
its own table saying **221** unverified. Two numbers for one quantity, on the page that IS the record
under ADR 0156.
The renderer computed survey progress from `DECIDED_VERDICTS`, which deliberately excludes
`needs-review` -- correctly, for verdict COUNTS, since that cell was examined and then left open on
purpose. But the survey line does not report what was decided, it reports what was **read**, and a
`needs-review` cell has been read. The constant's own docstring says so: *"the second was examined
and left open on purpose."* The code used the right set for the wrong question.
Adds `EXAMINED_VERDICTS = DECIDED_VERDICTS | {"needs-review"}` and uses it for progress only. Verdict
counts, and the `inherited` warning about a decided verdict lacking a `last_verified` date, keep
`DECIDED_VERDICTS` -- a parked cell is not an inherited verdict.
**Why this survived every gate since the renderer was written.** With zero `needs-review` cells the
two sets are identical, so no test, no CI run and no reading of the output could tell them apart. It
became observable only when the scorecard acquired its first one (11.4.4, in the V7/V11 baseline
sweep) -- a latent defect that needed new *data*, not new code, to show itself. That is the shape
this repo keeps finding: a control that passes because the case it gets wrong has never occurred.
The regression test asserts the two printed figures agree, because a reader comparing them is the
only thing that ever noticed. Proved red against `HEAD` before being trusted green -- the renderer
was reverted with `git show HEAD:` and the file restored afterwards, since `git stash` is shared with
the primary checkout and the worktree gate refuses it.
ruff check + format, mypy strict, and all 35 scorecard tests pass.
wshallwshall
enabled auto-merge (squash)
August 2, 2026 15:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ASVS-CURRENT.md— the CI-gated page that is the ASVS record under ADR 0156 — printed two different numbers for one quantity:directly above its own table saying 221 unverified. 345 − 221 = 124.
The defect
Survey progress was computed from
DECIDED_VERDICTS, which deliberately excludesneeds-review. That exclusion is right for verdict counts — the cell was examined and then left open on purpose. But the survey line does not report what was decided, it reports what was read, and aneeds-reviewcell has been read. The constant's own docstring says so: "the second was examined and left open on purpose."The code used the right set for the wrong question.
The fix
EXAMINED_VERDICTS = DECIDED_VERDICTS | {"needs-review"}, used for progress only. Verdict counts keepDECIDED_VERDICTS, and so does theinheritedwarning — a parked cell is not an inherited verdict.Why it survived every gate since the renderer was written
With zero
needs-reviewcells the two sets are identical, so no test, no CI run and no reading of the output could distinguish them. It became observable only when the scorecard acquired its first one (11.4.4, in the V7/V11 baseline sweep) — a latent defect that needed new data, not new code, to surface. That is the shape this repo keeps finding: a control that passes because the case it gets wrong has never occurred.Verification
HEADbefore being trusted green.git stashis shared with the primary checkout and the worktree gate refuses it, so the renderer was reverted viagit show HEAD:and restored afterwards.ruff check+ruff format --check,mypystrict, 35/35 scorecard tests.Data-only, no behaviour change outside the rendered progress line. The vault's mirrored copy of
scripts/asvs/scorecard.pywill be synced after merge — its gate now reds on drift between the two.