[pjhartlieb]

Promotes dev to stable as v1.2.0 (version in pyproject.toml).

• Read the release version from a VERSION file (1.2.0) #16: release-check and stable-guard read the release version from a root VERSION file in a repository without a package manifest (Go or Docker-only, such as bbot-io-developer-portal). Where both exist they must agree.

stable has nothing dev lacks (only #12's merge commit), so no merge-back afterwards.

Merge with a merge commit. On merge, release.yml waits twice at the release environment. Each approval needs a bbot-leads member other than whoever merged.

[en0f]

Reviewed at 89260e0.

🔴 The body shows no validation for the code this ships to stable

git.md (Pull requests) and the org PR template ask the body to say how the change was validated, with the commands run and their real output. This body has no Validation section, and #16's states its results without output. CI at 89260e0 already has the output, so this is a small fix. Add a Validation section linking runs 42704651477 (pytest on 3.11 to 3.15, ruff, actionlint) and 42704650030 (actionlint and zizmor). If you still have the run of the five new tests against stable's manifest.py that #16 describes, paste it there too, but it's optional.

🟢 A VERSION-only repository can now release, and a VERSION that disagrees with pyproject.toml or Cargo.toml stops it

Run as stable-guard.yml and release-check.yml run it, manifest.py release emits v1.7.2 for a Go repository with only VERSION. It refuses a VERSION that disagrees with a pyproject.toml or Cargo.toml version and names both values. With neither source, it says to add a VERSION file. A VERSION-only repository gets the same MAJOR.MINOR.PATCH and Helm chart checks as everyone else, and manifest-only repositories release exactly as they do on stable.

🟢 1.2.0 is the right bump, and no current caller is affected

No file under .github/ changed, so no workflow input, output, secret or permission changed, which under the README's rule makes this a minor version. The four org repositories that call stable-guard.yml or release-check.yml do so without a working-directory, and none of them, nor the three others I checked including the developer portal, has a root VERSION on stable or dev, so the new refusal stops none of their releases.

Reproduce this review

# save the script below as verify-17.sh, then:
bash verify-17.sh blacklanternsecurity/release-workflows 89260e0f92ed3c1f4f7a21efefe8a6fc1ddf9b4c
# expected: SUMMARY 13 PASS, 4 FAIL, 0 SKIP, 0 BROKEN   (V10 to V13 are edge cases noted in my record, not blockers)
# control:  bash verify-17.sh blacklanternsecurity/release-workflows 12139195c6f35d8b4478bb8ec3df94aa275da01e   -> V01 V03 V04 V06 V07 V14 V15 = FAIL, V10 V11 = PASS

#!/usr/bin/env bash
# verify-17.sh: reproducible checks for blacklanternsecurity/release-workflows#17 (Release 1.2.0)
# Usage: bash verify-17.sh <owner/repo> <sha> [<control-patch>]
# Changes nothing outside a mktemp directory and nothing on GitHub (PR-VF-02).
# Runs scripts/manifest.py exactly as stable-guard.yml and release-check.yml do:
#   uv run --no-project --with packaging --with pyyaml python scripts/manifest.py --root <dir> release
#
# CONTROL: sha 12139195c6f35d8b4478bb8ec3df94aa275da01e
# CONTROL-EXPECT: V01=FAIL V03=FAIL V04=FAIL V06=FAIL V07=FAIL V10=PASS V11=PASS V14=FAIL V15=FAIL
set -uo pipefail

REPO="${1:?owner/repo}"; SHA="${2:?sha}"; PATCH="${3:-}"
BASE_SHA=12139195c6f35d8b4478bb8ec3df94aa275da01e
PASS=0; FAIL=0; SKIP=0; BROKEN=0
pass()   { echo "PASS $1 $2"; PASS=$((PASS+1)); }
fail()   { echo "FAIL $1 $2 -- $3"; FAIL=$((FAIL+1)); }
skip()   { echo "SKIP $1 $2 -- $3"; SKIP=$((SKIP+1)); }
broken() { echo "BROKEN $1 $2 -- $3"; BROKEN=$((BROKEN+1)); }
summary() { echo "SUMMARY $PASS PASS, $FAIL FAIL, $SKIP SKIP, $BROKEN BROKEN"; [ "$FAIL" -eq 0 ] && [ "$BROKEN" -eq 0 ]; }

echo "script_sha256 $(shasum -a 256 "$0" | cut -d' ' -f1)"
echo "git $(git --version | cut -d' ' -f3)"
echo "python3 $(python3 -c 'import sys; print(sys.version.split()[0])')"
echo "uv 0.11.20 (PyPI wheel, sha256-verified)"

WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 GIT_TERMINAL_PROMPT=0
export GIT_CONFIG_COUNT=2 GIT_CONFIG_KEY_0=credential.helper GIT_CONFIG_VALUE_0= GIT_CONFIG_KEY_1=credential.helper GIT_CONFIG_VALUE_1='!gh auth git-credential'
GIT=(git)
"${GIT[@]}" clone --quiet --filter=blob:none --no-checkout "https://github.com/$REPO.git" "$WORK/src" || { broken V00 "clone" "clone failed"; summary; exit 1; }
cd "$WORK/src"
"${GIT[@]}" cat-file -e "$SHA^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "+refs/pull/*/head:refs/remotes/pr/*" || true
"${GIT[@]}" checkout --quiet --detach "$SHA" || { broken V00 "checkout $SHA" "checkout failed"; summary; exit 1; }
[ "$(git rev-parse HEAD)" = "$SHA" ] || { broken V00 "HEAD is $SHA" "HEAD mismatch"; summary; exit 1; }
if [ -n "$PATCH" ]; then git apply "$PATCH" || { broken V00 "apply control patch" "git apply failed"; summary; exit 1; }; fi
SRC="$WORK/src"

# ---- checks that need only git and python3 ----------------------------------
# V01 the release version is MAJOR.MINOR.PATCH 1.2.0 in pyproject.toml and uv.lock
pv=$(python3 -c 'import tomllib,sys; print(tomllib.load(open(sys.argv[1],"rb"))["project"]["version"])' "$SRC/pyproject.toml")
lv=$(python3 -c 'import tomllib,sys; print(next(p["version"] for p in tomllib.load(open(sys.argv[1],"rb"))["package"] if p["name"]=="release-workflows"))' "$SRC/uv.lock")
if [ "$pv" = "1.2.0" ] && [ "$lv" = "1.2.0" ]; then pass V01 "pyproject.toml and uv.lock both at 1.2.0"
else fail V01 "pyproject.toml and uv.lock both at 1.2.0" "pyproject $pv, uv.lock $lv"; fi

# V02 stable (the base) adds no content over its merge-base with the head (only #12's merge commit),
#     so nothing on stable is lost and no merge-back is needed
"${GIT[@]}" cat-file -e "$BASE_SHA^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "$BASE_SHA" 2>/dev/null || true
MB=$(git merge-base HEAD "$BASE_SHA" 2>/dev/null)
if [ -z "$MB" ]; then broken V02 "stable adds no content over its merge-base with the head" "no merge-base"
elif git diff --quiet "$MB" "$BASE_SHA"; then pass V02 "stable adds no content over merge-base ${MB:0:8} (commits only: $(git rev-list --count "$MB..$BASE_SHA"))"
else fail V02 "stable adds no content over its merge-base with the head" "$(git diff --name-only "$MB" "$BASE_SHA" | tr '\n' ' ')"; fi

# V09 no workflow file differs from stable: no input, output, secret or permission a caller sees changed
changed=$(git diff --name-only "$BASE_SHA" -- .github/ 2>/dev/null); drc=$?
if [ $drc -ne 0 ]; then broken V09 "no .github/ file differs from stable" "git diff failed"
elif [ -z "$changed" ]; then pass V09 "no .github/ file differs from stable $BASE_SHA"
else fail V09 "no .github/ file differs from stable" "$(echo $changed)"; fi

# ---- uv 0.11.20, pinned and checksum-verified against PyPI's published sha256 (PR-VF-03)
case "$(uname -s)-$(uname -m)" in
  Darwin-arm64)  WHL=uv-0.11.20-py3-none-macosx_11_0_arm64.whl; H=c9f2062096e146b351fd5da3cd43e15a5c43bfa37166c509d884dab3dbb72f03
                 URL=https://files.pythonhosted.org/packages/39/3e/e3d39361b95c262b43ccfe260f41184da71c536a08f39d4ad59ab962e459/$WHL;;
  Darwin-x86_64) WHL=uv-0.11.20-py3-none-macosx_10_12_x86_64.whl; H=f3bacc52778775cef671867ddab744b50c4183bc3cd6419a5fb4eb01a02f9526
                 URL=https://files.pythonhosted.org/packages/02/7d/bbaad5f0c616f7824149a4ac0271db14107b859b36bd75d16db1486c495f/$WHL;;
  Linux-x86_64)  WHL=uv-0.11.20-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl; H=61588768f04a24b0b4d87b43f03b4521ba66d5eec3ab1aca37cb59b1d52337db
                 URL=https://files.pythonhosted.org/packages/7e/38/f844d125db277d8ce0c921f0219078d292d0ee72d314d0c12f4cf510aa40/$WHL;;
  *) WHL=;;
esac
UVB=
if [ -n "$WHL" ] && curl -fsSLo "$WORK/$WHL" "$URL"; then
  if echo "$H  $WORK/$WHL" | shasum -a 256 -c - >/dev/null 2>&1; then pass V20 "uv 0.11.20 wheel checksum matches PyPI"
    (cd "$WORK" && python3 -c "import zipfile,sys; z=zipfile.ZipFile(sys.argv[1]); [open('uv','wb').write(z.read(n)) for n in z.namelist() if n.endswith('/scripts/uv')]" "$WHL") && chmod +x "$WORK/uv" && UVB="$WORK/uv"
  else broken V20 "uv 0.11.20 wheel checksum matches PyPI" "mismatch"; fi
  if echo "0000000000000000000000000000000000000000000000000000000000000000  $WORK/$WHL" | shasum -a 256 -c - >/dev/null 2>&1; then broken V21 "wrong-hash control fails" "accepted a wrong hash"; else pass V21 "wrong-hash control fails"; fi
else skip V20 "uv 0.11.20 wheel checksum matches PyPI" "no wheel for this platform or download failed"; skip V21 "wrong-hash control fails" "no wheel"; fi
# uv config, cache and interpreters stay in $WORK (PR-VF-04)
uvx_() { UV_NO_CONFIG=1 UV_CACHE_DIR="$WORK/uvcache" UV_PYTHON_INSTALL_DIR="$WORK/uvpy" UV_PYTHON_DOWNLOADS=never "$UVB" "$@"; }

if [ -z "$UVB" ]; then
  for v in V03 V04 V05 V06 V07 V08 V10 V11; do skip $v "manifest.py release" "uv unavailable"; done
  summary; exit $?
fi

# Fixture repositories: files written, then git init + add (manifest.py lists charts with git ls-files).
fixture() { local d="$WORK/fx/$1"; shift; mkdir -p "$d"; while [ $# -gt 0 ]; do mkdir -p "$(dirname "$d/$1")"; printf '%b' "$2" > "$d/$1"; shift 2; done
            (cd "$d" && git init -q && git add -A); echo "$d"; }
# release <root>: the workflows' exact command; sets RC, OUT, ERR
release() { OUT=$(cd "$SRC" && uvx_ run --no-project --with packaging --with pyyaml python scripts/manifest.py --root "$1" release 2>"$WORK/err"); RC=$?; ERR=$(cat "$WORK/err"); }
last() { echo "$1" | grep -v '^ *$' | grep -vi -E '^(installed|resolved|prepared|downloading|audited)' | tail -1; }

# V03 a Go repository with only VERSION releases at that version
D=$(fixture v03 VERSION '1.7.2\n' go.mod 'module example.com/x\n'); release "$D"
if [ $RC -eq 0 ] && echo "$OUT" | grep -qx 'version=1.7.2' && echo "$OUT" | grep -qx 'tag=v1.7.2'; then pass V03 "VERSION-only repo: release emits version=1.7.2 tag=v1.7.2"
else fail V03 "VERSION-only repo: release emits version=1.7.2 tag=v1.7.2" "exit $RC: $(last "$ERR")"; fi

# V04 a VERSION that disagrees with pyproject.toml stops the release, naming both
D=$(fixture v04 pyproject.toml '[project]\nname = "x"\nversion = "1.2.3"\n' VERSION '1.2.4\n'); release "$D"
if [ $RC -ne 0 ] && echo "$ERR" | grep -q '1.2.4' && echo "$ERR" | grep -q '1.2.3'; then pass V04 "VERSION 1.2.4 vs pyproject 1.2.3: release refused, both named"
else fail V04 "VERSION 1.2.4 vs pyproject 1.2.3: release refused, both named" "exit $RC, stdout: $(echo "$OUT" | grep '^version=')"; fi

# V05 a VERSION that matches pyproject.toml changes nothing
D=$(fixture v05 pyproject.toml '[project]\nname = "x"\nversion = "1.2.3"\n' VERSION '1.2.3\n'); release "$D"
if [ $RC -eq 0 ] && echo "$OUT" | grep -qx 'version=1.2.3'; then pass V05 "matching VERSION and pyproject: version=1.2.3"
else fail V05 "matching VERSION and pyproject: version=1.2.3" "exit $RC: $(last "$ERR")"; fi

# V06 with no version source, the error names VERSION and is not a traceback
D=$(fixture v06 go.mod 'module example.com/x\n'); release "$D"
if [ $RC -ne 0 ] && ! echo "$ERR" | grep -q Traceback && echo "$ERR" | grep -q VERSION; then pass V06 "no version source: plain error naming VERSION"
else fail V06 "no version source: plain error naming VERSION" "exit $RC: $(last "$ERR")"; fi

# V07 a prerelease or v-prefixed VERSION is refused as not MAJOR.MINOR.PATCH
ok=1
for val in '1.7.2-rc1' 'v1.7.2'; do
  D=$(fixture "v07-$val" VERSION "$val\n"); release "$D"
  { [ $RC -ne 0 ] && echo "$ERR" | grep -q 'MAJOR.MINOR.PATCH'; } || ok=0
done
if [ $ok -eq 1 ]; then pass V07 "VERSION 1.7.2-rc1 and v1.7.2 refused as not MAJOR.MINOR.PATCH"
else fail V07 "VERSION 1.7.2-rc1 and v1.7.2 refused as not MAJOR.MINOR.PATCH" "last exit $RC: $(last "$ERR")"; fi

# V08 a manifest-only caller is unchanged (regression guard; passes at stable too)
D=$(fixture v08 pyproject.toml '[project]\nname = "x"\nversion = "1.2.3"\n' helm/Chart.yaml 'name: x\nversion: 1.2.3\nappVersion: "1.2.3"\n'); release "$D"
if [ $RC -eq 0 ] && echo "$OUT" | grep -qx 'version=1.2.3' && echo "$OUT" | grep -qx 'pypi_name=x'; then pass V08 "pyproject-only repo with a chart: version=1.2.3 pypi_name=x, as on stable"
else fail V08 "pyproject-only repo with a chart: version=1.2.3 pypi_name=x, as on stable" "exit $RC: $(last "$ERR")"; fi

# V10 a pyproject with a dynamic (hatch-vcs) version is not released at the VERSION file's value unchecked
D=$(fixture v10 pyproject.toml '[build-system]\nbuild-backend = "hatchling.build"\n[project]\nname = "x"\ndynamic = ["version"]\n[tool.hatch.version]\nsource = "vcs"\n' VERSION '1.2.4\n'); release "$D"
if [ $RC -eq 0 ] && echo "$OUT" | grep -qx 'version=1.2.4'; then fail V10 "hatch-vcs pyproject + VERSION: not released at VERSION unchecked" "exit 0, version=1.2.4"
else pass V10 "hatch-vcs pyproject + VERSION: not released at VERSION unchecked"; fi

# V11 a package.json version is compared with VERSION before VERSION is released
D=$(fixture v11 package.json '{"name": "x", "version": "2.0.0"}\n' VERSION '1.0.0\n'); release "$D"
if [ $RC -eq 0 ] && echo "$OUT" | grep -qx 'version=1.0.0'; then fail V11 "package.json 2.0.0 vs VERSION 1.0.0: compared before release" "exit 0, version=1.0.0"
else pass V11 "package.json 2.0.0 vs VERSION 1.0.0: compared before release"; fi

# V12 (challenge C12) an empty VERSION file is reported as a missing version source, not a blank version
D=$(fixture v12 VERSION '\n' go.mod 'module example.com/x\n'); release "$D"
if [ $RC -ne 0 ] && echo "$ERR" | grep -q 'add a VERSION file'; then pass V12 "empty VERSION: error names the missing version source"
else fail V12 "empty VERSION: error names the missing version source" "exit $RC: $(last "$ERR")"; fi

# V13 (challenge C11) the README does not say VERSION is read at the repository root (it is read at --root, the working-directory)
if grep -q 'VERSION` file at the repository root' "$SRC/README.md"; then fail V13 "README places VERSION where the code reads it (working-directory)" "README.md:$(grep -n 'VERSION` file at the repository root' "$SRC/README.md" | cut -d: -f1) says 'at the repository root'"
else pass V13 "README places VERSION where the code reads it (working-directory)"; fi
# V14 (round-2 challenge C1) a VERSION that disagrees with a Cargo.toml version stops the release, naming both
D=$(fixture v14 Cargo.toml '[package]\nname = "x"\nversion = "2.0.0"\n' VERSION '2.0.1\n'); release "$D"
if [ $RC -ne 0 ] && echo "$ERR" | grep -q '2.0.1' && echo "$ERR" | grep -q '2.0.0'; then pass V14 "VERSION 2.0.1 vs Cargo.toml 2.0.0: release refused, both named"
else fail V14 "VERSION 2.0.1 vs Cargo.toml 2.0.0: release refused, both named" "exit $RC, stdout: $(echo "$OUT" | grep '^version=')"; fi

# V15 (round-2 challenge C1) in a VERSION-only repo, a Helm chart must match VERSION
D=$(fixture v15 VERSION '1.7.2\n' charts/x/Chart.yaml 'name: x\nversion: 1.7.1\nappVersion: 1.7.2\n'); release "$D"
if [ $RC -ne 0 ] && echo "$ERR" | grep -q 'Chart.yaml version 1.7.1'; then pass V15 "VERSION-only repo: chart version 1.7.1 != 1.7.2 refused"
else fail V15 "VERSION-only repo: chart version 1.7.1 != 1.7.2 refused" "exit $RC: $(last "$ERR")"; fi
# ------------------------------------------------------------------------------
summary