Repository navigation
test: the release-gate tables cannot tell == 'false' from != 'true' on an evidence output #967
Description
tests/project/test_release_gates.py, as it stands after #966, cannot tell
== 'false' from != 'true' on an evidence output. Measured against the module's
own helpers on #966's head e403c9166:
tag: evaluator=0 text=0 (mutant: == 'false' -> != 'true')
pypi: evaluator=0 text=0
conda: evaluator=0 text=0 (unmutated condition also 0/0, as control)
Two reasons it is invisible. text_gate_violations only checks that the evidence
name appears in the condition, never which comparison is made against it. And
every row in gate_rows that holds version_check at success also holds the
evidence outputs at a literal 'true'/'false', so the two spellings agree on
every modelled row.
The spellings diverge only when an output is neither — empty or absent. For a
publishing job that is the dangerous direction: != 'true' runs where
== 'false' skips, so unknown evidence would become a publish attempt.
The two curl-based checks cannot produce that: both default complete=false and
only then set true, under set -euo pipefail. But PCAPKIT_TAG_EXISTS and
PCAPKIT_CONDA_TAG_EXISTS come from mukunku/tag-exists-action@v1.7.0
(create-release.yml:189,201), a third-party output this repo does not control,
and tag's gate reads one of them.
Fix is cheap: add a version_check=success row to gate_rows holding one
evidence output at NULL, and assert the gate skips. Found by the cross-review on
#966; not a gate on it.
Activity
JarryShaw commentedon Oct 1, 2026
Blocked on #966, and the blocker is checkable rather than remembered:
The fix here is one extra row in
gate_rowsinsidetests/project/test_release_gates.py— the same file #966 rewrites (+1200/−8),and #966 is open at
e403c9166withreview: good-to-go. A worker starting nowwould be editing a function that does not exist on
mainyet, and would conflictwith #966 rather than build on it.
Unblocks the moment #966 merges. Not waiting on anything else.
JarryShaw commentedon Oct 1, 2026
Unblocked — #966 merged as
d8f695903(14:58:48Z) and is onmain:So
gate_rowsand the evaluator this issue extends now exist onmain(
tests/project/test_release_gates.py).blockedremoved, work starting.test(project): tell == 'false' from != 'true' on an evidence output (#…