[pjhartlieb]

release-check and stable-guard read the release version only from the package manifest (pyproject.toml, or through it Cargo.toml). A Go or Docker-only repository has neither, so it can't release through these workflows. bbot-io-developer-portal is the first. Its version now lives only in its publish job's image tags.

manifest.py now falls back to a VERSION file at the repository root when there is no manifest version:

• A repository with only VERSION: that is the release version, under the same MAJOR.MINOR.PATCH, chart and git-dependency checks.
• Both present: they must agree, or the release stops, as a chart version that disagrees with the manifest already does.
• Neither: the error now names the missing source.

Nothing changes for existing callers, which all have a manifest version, so this is a minor version: 1.2.0.

Tests: five new cases (VERSION only, a prerelease in VERSION, a mismatch, a match, neither). 46 pass. Against the old manifest.py, four of the five new tests fail; the fifth guards that a matching VERSION file changes nothing. ruff clean; zizmor 1.30.1 (online audits) and actionlint 1.7.12 report nothing.

[en0f]

Reviewed at 60ad464.

🔴 Validation states results but shows no output

The body says 46 tests pass, four of the five new tests fail against the old manifest.py, ruff is clean, and zizmor 1.30.1 and actionlint 1.7.12 report nothing, but shows none of the commands or their output. git.md (Pull requests) and the PR template's Validation section ask for the commands run and their real output, not a statement that they passed. The tests themselves are the right ones, so this only asks you to paste what you ran. Under Validation, paste each command and its output, or drop the claim: uv run --group dev pytest, the run against the old manifest.py, ruff, zizmor and actionlint.

🟢 A VERSION that disagrees with a declared manifest version stops the release

When pyproject.toml or Cargo.toml declares a version, version() refuses a VERSION file that differs and names both values, the same way a chart behind the manifest already stops a release. Neither source silently wins, so a stale VERSION can't ship the wrong tag in those repositories.

🟢 The new source goes through the existing checks, tested at the CLI the workflows run

Only version() changed and cmd_release consumes it as before, so a VERSION-only repository gets the same MAJOR.MINOR.PATCH rule and Helm chart agreement as a manifest repository. The new tests run manifest.py --root ... release as a subprocess, which is exactly what stable-guard.yml and release-check.yml run.

🟢 1.2.0 is the right bump

No workflow file changed, and none of the org repositories pushed since v1.0.0 existed has a root VERSION file, so the agreement check stops no known caller's release. Under the README's rule that is a minor version, as the body says.

Reproduce this review

# save the script below as verify-16.sh, then:
bash verify-16.sh blacklanternsecurity/release-workflows 60ad464d45aecfdbb62b28707ca9a648af8c11ce
# expected: SUMMARY 13 PASS, 2 FAIL, 0 SKIP, 0 BROKEN   (V12 and V13 are edge cases noted in my record, not blockers)
# control:  bash verify-16.sh blacklanternsecurity/release-workflows df2b81c5476c5a5af42b4575762a08c9ab9d54a2   -> V01 V02 V03 V05 V06 V07 V11 = FAIL, V12 V13 = PASS

#!/usr/bin/env bash
# verify-16.sh: reproducible checks for blacklanternsecurity/release-workflows#16
# Usage: bash verify-16.sh <owner/repo> <sha> [<control-patch>]
# Changes nothing outside a mktemp directory and nothing on GitHub (PR-VF-02).
# Runs scripts/manifest.py exactly as stable-guard.yml and release-check.yml do:
#   uv run --no-project --with packaging --with pyyaml python scripts/manifest.py --root <dir> release
#
# CONTROL: sha df2b81c5476c5a5af42b4575762a08c9ab9d54a2
# CONTROL-EXPECT: V01=FAIL V02=FAIL V03=FAIL V05=FAIL V06=FAIL V07=FAIL V11=FAIL V12=PASS V13=PASS
set -uo pipefail

REPO="${1:?owner/repo}"; SHA="${2:?sha}"; PATCH="${3:-}"
PASS=0; FAIL=0; SKIP=0; BROKEN=0
pass()   { echo "PASS $1 $2"; PASS=$((PASS+1)); }
fail()   { echo "FAIL $1 $2 -- $3"; FAIL=$((FAIL+1)); }
skip()   { echo "SKIP $1 $2 -- $3"; SKIP=$((SKIP+1)); }
broken() { echo "BROKEN $1 $2 -- $3"; BROKEN=$((BROKEN+1)); }
summary() { echo "SUMMARY $PASS PASS, $FAIL FAIL, $SKIP SKIP, $BROKEN BROKEN"; [ "$FAIL" -eq 0 ] && [ "$BROKEN" -eq 0 ]; }

echo "script_sha256 $(shasum -a 256 "$0" | cut -d' ' -f1)"
echo "git $(git --version | cut -d' ' -f3)"
echo "python3 $(python3 -c 'import sys; print(sys.version.split()[0])')"
echo "uv 0.11.20 (PyPI wheel, sha256-verified)"

WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 GIT_TERMINAL_PROMPT=0
export GIT_CONFIG_COUNT=2 GIT_CONFIG_KEY_0=credential.helper GIT_CONFIG_VALUE_0= GIT_CONFIG_KEY_1=credential.helper GIT_CONFIG_VALUE_1='!gh auth git-credential'
GIT=(git)
"${GIT[@]}" clone --quiet --filter=blob:none --no-checkout "https://github.com/$REPO.git" "$WORK/src" || { broken V00 "clone" "clone failed"; summary; exit 1; }
cd "$WORK/src"
"${GIT[@]}" cat-file -e "$SHA^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "+refs/pull/*/head:refs/remotes/pr/*" || true
"${GIT[@]}" checkout --quiet --detach "$SHA" || { broken V00 "checkout $SHA" "checkout failed"; summary; exit 1; }
[ "$(git rev-parse HEAD)" = "$SHA" ] || { broken V00 "HEAD is $SHA" "HEAD mismatch"; summary; exit 1; }
if [ -n "$PATCH" ]; then git apply "$PATCH" || { broken V00 "apply control patch" "git apply failed"; summary; exit 1; }; fi
SRC="$WORK/src"

# uv 0.11.20, pinned and checksum-verified against PyPI's published sha256 (PR-VF-03)
case "$(uname -s)-$(uname -m)" in
  Darwin-arm64)  WHL=uv-0.11.20-py3-none-macosx_11_0_arm64.whl; H=c9f2062096e146b351fd5da3cd43e15a5c43bfa37166c509d884dab3dbb72f03
                 URL=https://files.pythonhosted.org/packages/39/3e/e3d39361b95c262b43ccfe260f41184da71c536a08f39d4ad59ab962e459/$WHL;;
  Darwin-x86_64) WHL=uv-0.11.20-py3-none-macosx_10_12_x86_64.whl; H=f3bacc52778775cef671867ddab744b50c4183bc3cd6419a5fb4eb01a02f9526
                 URL=https://files.pythonhosted.org/packages/02/7d/bbaad5f0c616f7824149a4ac0271db14107b859b36bd75d16db1486c495f/$WHL;;
  Linux-x86_64)  WHL=uv-0.11.20-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl; H=61588768f04a24b0b4d87b43f03b4521ba66d5eec3ab1aca37cb59b1d52337db
                 URL=https://files.pythonhosted.org/packages/7e/38/f844d125db277d8ce0c921f0219078d292d0ee72d314d0c12f4cf510aa40/$WHL;;
  *) WHL=;;
esac
UVB=
if [ -n "$WHL" ] && curl -fsSLo "$WORK/$WHL" "$URL"; then
  if echo "$H  $WORK/$WHL" | shasum -a 256 -c - >/dev/null 2>&1; then pass V20 "uv 0.11.20 wheel checksum matches PyPI"
    (cd "$WORK" && python3 -c "import zipfile,sys; z=zipfile.ZipFile(sys.argv[1]); [open('uv','wb').write(z.read(n)) for n in z.namelist() if n.endswith('/scripts/uv')]" "$WHL") && chmod +x "$WORK/uv" && UVB="$WORK/uv"
  else broken V20 "uv 0.11.20 wheel checksum matches PyPI" "mismatch"; fi
  if echo "0000000000000000000000000000000000000000000000000000000000000000  $WORK/$WHL" | shasum -a 256 -c - >/dev/null 2>&1; then broken V21 "wrong-hash control fails" "accepted a wrong hash"; else pass V21 "wrong-hash control fails"; fi
else skip V20 "uv 0.11.20 wheel checksum matches PyPI" "no wheel for this platform or download failed"; skip V21 "wrong-hash control fails" "no wheel"; fi
# uv config, cache and interpreters stay in $WORK (PR-VF-04)
uvx_() { UV_NO_CONFIG=1 UV_CACHE_DIR="$WORK/uvcache" UV_PYTHON_INSTALL_DIR="$WORK/uvpy" UV_PYTHON_DOWNLOADS=never "$UVB" "$@"; }

# Fixture repositories: files written, then git init + add (manifest.py lists charts with git ls-files).
fixture() { local d="$WORK/fx/$1"; shift; mkdir -p "$d"; while [ $# -gt 0 ]; do mkdir -p "$(dirname "$d/$1")"; printf '%b' "$2" > "$d/$1"; shift 2; done
            (cd "$d" && git init -q && git add -A); echo "$d"; }
# release <root>: the workflows' exact command; sets RC, OUT, ERR
release() { OUT=$(cd "$SRC" && uvx_ run --no-project --with packaging --with pyyaml python scripts/manifest.py --root "$1" release 2>"$WORK/err"); RC=$?; ERR=$(cat "$WORK/err"); }
first() { echo "$1" | grep -v '^ *$' | grep -vi -E '^(installed|resolved|prepared|downloading|audited)' | tail -1; }

if [ -z "$UVB" ]; then
  for v in V01 V02 V03 V04 V05 V06 V07 V08 V11; do skip $v "manifest.py release" "uv unavailable"; done
  summary; exit $?
fi

# V01 a repository with only VERSION (Go layout) releases at that version
D=$(fixture v01 VERSION '1.7.2\n' go.mod 'module example.com/x\n'); release "$D"
if [ $RC -eq 0 ] && echo "$OUT" | grep -qx 'version=1.7.2' && echo "$OUT" | grep -qx 'tag=v1.7.2'; then pass V01 "VERSION-only repo: release emits version=1.7.2 tag=v1.7.2"
else fail V01 "VERSION-only repo: release emits version=1.7.2 tag=v1.7.2" "exit $RC: $(first "$ERR")"; fi

# V02 a VERSION file that disagrees with pyproject.toml stops the release, naming both
D=$(fixture v02 pyproject.toml '[project]\nname = "x"\nversion = "1.2.3"\n' VERSION '1.2.4\n'); release "$D"
if [ $RC -ne 0 ] && echo "$ERR" | grep -q '1.2.4' && echo "$ERR" | grep -q '1.2.3'; then pass V02 "VERSION 1.2.4 vs pyproject 1.2.3: release refused, both named"
else fail V02 "VERSION 1.2.4 vs pyproject 1.2.3: release refused, both named" "exit $RC, stdout: $(echo "$OUT" | grep '^version=')"; fi

# V03 with neither source, the error names the missing source and is not a traceback
D=$(fixture v03 go.mod 'module example.com/x\n'); release "$D"
if [ $RC -ne 0 ] && ! echo "$ERR" | grep -q Traceback && echo "$ERR" | grep -q VERSION; then pass V03 "no version source: plain error naming VERSION, no traceback"
else fail V03 "no version source: plain error naming VERSION, no traceback" "exit $RC: $(first "$ERR")"; fi

# V04 an existing manifest-only caller is unchanged (regression guard; passes at base too)
D=$(fixture v04 pyproject.toml '[project]\nname = "x"\nversion = "1.2.3"\n'); release "$D"
if [ $RC -eq 0 ] && echo "$OUT" | grep -qx 'version=1.2.3' && echo "$OUT" | grep -qx 'pypi_name=x'; then pass V04 "pyproject-only repo: version=1.2.3 pypi_name=x, as before"
else fail V04 "pyproject-only repo: version=1.2.3 pypi_name=x, as before" "exit $RC: $(first "$ERR")"; fi

# V05 in a VERSION-only repo, a Helm chart must match VERSION (chart rule applies to the new source)
D=$(fixture v05 VERSION '1.7.2\n' charts/x/Chart.yaml 'name: x\nversion: 1.7.1\nappVersion: 1.7.2\n'); release "$D"
if [ $RC -ne 0 ] && echo "$ERR" | grep -q 'Chart.yaml version 1.7.1'; then pass V05 "VERSION-only repo: chart version 1.7.1 != 1.7.2 refused"
else fail V05 "VERSION-only repo: chart version 1.7.1 != 1.7.2 refused" "exit $RC: $(first "$ERR")"; fi

# V06 a prerelease or v-prefixed VERSION is refused as not MAJOR.MINOR.PATCH
ok=1
for val in '1.7.2-rc1' 'v1.7.2'; do
  D=$(fixture "v06-$val" VERSION "$val\n"); release "$D"
  { [ $RC -ne 0 ] && echo "$ERR" | grep -q 'MAJOR.MINOR.PATCH'; } || ok=0
done
if [ $ok -eq 1 ]; then pass V06 "VERSION 1.7.2-rc1 and v1.7.2 refused as not MAJOR.MINOR.PATCH"
else fail V06 "VERSION 1.7.2-rc1 and v1.7.2 refused as not MAJOR.MINOR.PATCH" "last exit $RC: $(first "$ERR")"; fi

# V07 surrounding whitespace (CRLF, trailing newline) is ignored
D=$(fixture v07 VERSION '1.7.2\r\n'); release "$D"
if [ $RC -eq 0 ] && echo "$OUT" | grep -qx 'version=1.7.2'; then pass V07 "VERSION with CRLF line ending reads as 1.7.2"
else fail V07 "VERSION with CRLF line ending reads as 1.7.2" "exit $RC: $(first "$ERR")"; fi

# V11 VERSION is read from --root (the caller's working-directory), not the repository root
D=$(fixture v11 svc/VERSION '3.0.1\n' VERSION '9.9.9\n'); release "$D/svc"
if [ $RC -eq 0 ] && echo "$OUT" | grep -qx 'version=3.0.1'; then pass V11 "--root svc reads svc/VERSION (3.0.1), not the repo-root VERSION"
else fail V11 "--root svc reads svc/VERSION (3.0.1), not the repo-root VERSION" "exit $RC: $(first "$ERR")"; fi

# V12 (challenge C1) a pyproject with a dynamic version is not silently released at the VERSION file's value
D=$(fixture v12 pyproject.toml '[build-system]\nbuild-backend = "hatchling.build"\n[project]\nname = "x"\ndynamic = ["version"]\n[tool.hatch.version]\nsource = "vcs"\n' VERSION '1.2.4\n'); release "$D"
if [ $RC -eq 0 ] && echo "$OUT" | grep -qx 'version=1.2.4' && echo "$OUT" | grep -qx 'pypi_name=x'; then fail V12 "hatch-vcs pyproject + VERSION: not released at VERSION unchecked" "exit 0, version=1.2.4, pypi_name=x"
else pass V12 "hatch-vcs pyproject + VERSION: not released at VERSION unchecked"; fi

# V13 (challenge C11) a package.json version is compared with VERSION before VERSION is released
D=$(fixture v13 package.json '{"name": "x", "version": "2.0.0"}\n' VERSION '1.0.0\n'); release "$D"
if [ $RC -eq 0 ] && echo "$OUT" | grep -qx 'version=1.0.0'; then fail V13 "package.json 2.0.0 vs VERSION 1.0.0: compared before release" "exit 0, version=1.0.0"
else pass V13 "package.json 2.0.0 vs VERSION 1.0.0: compared before release"; fi

# V08 the repository's full test suite passes, with pytest at its uv.lock version.
# (The dev group also holds actionlint-py, an sdist that pytest does not need; it is left out.)
# The matrix tests shell out to `uv python list`, so the pinned uv goes first on PATH.
PYTEST=$(python3 -c 'import tomllib,sys; print(next(p["version"] for p in tomllib.load(open(sys.argv[1],"rb"))["package"] if p["name"]=="pytest"))' "$SRC/uv.lock")
out=$(cd "$SRC" && PATH="$WORK:$PATH" uvx_ run --no-project --with "pytest==$PYTEST" --with packaging --with pyyaml pytest -q -p no:cacheprovider 2>&1); rc=$?
line=$(echo "$out" | grep -E '[0-9]+ (passed|failed)' | tail -1); [ -n "$line" ] || line=$(echo "$out" | tail -1)
if [ $rc -eq 0 ]; then pass V08 "full suite passes: $line"; else fail V08 "full suite passes" "exit $rc: $line"; fi

# V09 pyproject.toml and uv.lock carry the same project version
pv=$(python3 -c 'import tomllib,sys; print(tomllib.load(open(sys.argv[1],"rb"))["project"]["version"])' "$SRC/pyproject.toml")
lv=$(python3 -c 'import tomllib,sys; print(next(p["version"] for p in tomllib.load(open(sys.argv[1],"rb"))["package"] if p["name"]=="release-workflows"))' "$SRC/uv.lock")
if [ -n "$pv" ] && [ "$pv" = "$lv" ]; then pass V09 "pyproject.toml and uv.lock both at $pv"; else fail V09 "pyproject.toml and uv.lock agree" "pyproject $pv, uv.lock $lv"; fi

# V10 no workflow file changed against the base (the callers' contract is unchanged)
BASE=df2b81c5476c5a5af42b4575762a08c9ab9d54a2
"${GIT[@]}" cat-file -e "$BASE^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "$BASE" 2>/dev/null || true
changed=$(git diff --name-only "$BASE" -- .github/ 2>/dev/null); drc=$?
if [ $drc -ne 0 ]; then broken V10 "no .github/ file differs from base" "git diff failed"
elif [ -z "$changed" ]; then pass V10 "no .github/ file differs from base $BASE"
else fail V10 "no .github/ file differs from base" "$changed"; fi

summary