ci(release): gate every publishing job on a deployment environment (#641)

[JarryShaw]

Fixes #641.

⚠️ This gate is inert until you add required reviewers, and it will look like it works

Merging this PR changes nothing on its own. A job naming an environment: that does not exist gets that environment created implicitly, with no protection rules, and runs unapproved — so on merge the release path behaves exactly as it does today, while every reader of the file believes it is gated. That is strictly worse than no gate, because nobody looks twice at a protection they think they have.

I cannot configure this; it is repository settings, not repository content. For each of the four names below:

Settings → Environments → New environment → name it → Required reviewers → add yourself → Save protection rules

The four names: github-release, conda-tag, pypi, anaconda.

Leave "Deployment branches and tags" as "No restriction" on all four. This workflow's push trigger is tags: v*, and a branch policy that does not list that tag pattern does not pause the job for approval — it fails it outright with Branch is not allowed to deploy to <environment>. That would break the release path rather than gate it.

Half-configured is the failure mode that matters, since the ungated ones stay silently automatic. This repository already ships the worked example: GET /repos/JarryShaw/PyPCAPKit/environments returns github-pages, created 2021-01-10, with "protection_rules": [] — an environment that has existed for five years and gates nothing.

Environments that exist today (read-only check, nothing created or modified)

$ gh api repos/JarryShaw/PyPCAPKit/environments
{"total_count": 2, "environments": [
  {"name": "copilot",      "created_at": "2026-08-13T04:11:43Z", "protection_rules": [], "deployment_branch_policy": null},
  {"name": "github-pages", "created_at": "2021-01-10T13:48:29Z", "protection_rules": [], "deployment_branch_policy": null}]}

So none of the four exist, and neither existing one has any protection rule. copilot and github-pages are untouched by this PR.

Every job in the file with an outward-facing effect, and what gates it now

I read the whole workflow rather than only the job carrying the commented line. Six jobs; four reach outside the run, and all four are now gated.

Job	Outward-facing effect	Credential	Environment
github	Creates the public GitHub Release, and the v* tag with it — action-gh-release creates the tag its tag_name names, and every job below checks that tag out	run's GITHUB_TOKEN	github-release
tag	Pushes a commit to main (git push origin HEAD:main) and the conda-* tag	PYPCAPKIT PAT + PYPCAPKIT_ACTIONS_DEPLOY_KEY	conda-tag
pypi	Uploads to PyPI and TestPyPI; attaches wheels to the Release	OIDC trusted publishing	pypi
conda	Uploads to anaconda.org/jarryshaw; attaches packages to the Release	ANACONDA_TOKEN	anaconda
unit-tests	none — test gate	—	none, and none possible: a job that calls a reusable workflow may not declare an environment:
version_check	none — contents: read	—	none needed

tag is the one worth calling out: its permissions: {} makes it read as inert, but the write goes through the deploy key and PAT rather than the run's token, so it pushes to the default branch with no gate at all today.

No docs deploy in this file. There is one in deploy-pages.yml (JamesIves/github-pages-deploy-action@v4) and it is also ungated, but that file is outside this PR's scope and outside #641 — flagging it rather than touching it, and happy to file it separately if you want it.

Why four environments rather than one release

I agree with your inclination, and went one step further on the same reasoning: one environment per distinct credential and blast radius.

The load-bearing fact is that approval is granted to an environment, not to a job. A single release approved once releases PyPI and Anaconda together, which renames the problem rather than fixing it. PyPI in particular has to be answerable alone: a version number it has accepted cannot be reused, and skip-existing: true means the re-upload then succeeds having published nothing — so an unintended publish permanently consumes the number the intended release wanted.

github and tag are split from each other on the same principle: github spends the run's own ephemeral GITHUB_TOKEN, whereas tag spends a long-lived PAT and a deploy key to write to main. Different credential, different blast radius, so a separate answer. It also means you can approve a Release without approving a push to the default branch.

The cost is two approval waves, not four. github and tag need only version_check, so they arrive together; pypi and conda then arrive together too. Each wave is one visit to "Review pending deployments" with two checkboxes.

One thing I could not verify and am not claiming: whether GitHub requests approval once per environment per run or once per matrix leg. pypi is a 7-leg matrix and conda a 10-leg one. The pending-deployments API is keyed by environment, which is why I expect one approval each, but that is a property of the runner and only a real run shows it. The b5 run will settle it.

Credentials untouched

No secret invented, renamed, or moved; no change to how any credential is obtained. Trusted publishing still uses OIDC (# password: ${{ secrets.PYPI_API_TOKEN }} stays commented exactly as it was), conda still uses secrets.ANACONDA_TOKEN, tag still uses its PAT and deploy key. The only permissions: change is cosmetic: the IMPORTANT: this permission is mandatory for trusted publishing note moved from the dead commented block onto the live id-token: write it actually describes.

I did delete the stale commented block (## Specifying a GitHub environment is optional... / #environment: release / the commented permissions:). Leaving #environment: release sitting beside a live environment: pypi would be worse than either. That block is the subject of #641, and its duplicate commented permissions: above a live one is precisely what made the omission read as deliberate.

The b5 validation run

Deliberately untouched, since they are the thing under test: the Select release body step, its absent-file branch, its version-mismatch branch that exits 0 with no path, its match branch, and the body_path / generate_release_notes: true wiring. Verified byte-for-byte, not by eye — both regions from HEAD are exact substrings of the new file:

Select release body 121-147            -> present byte-for-byte in new file = True
body_path/generate_release_notes 158-164 -> present byte-for-byte in new file = True

and the diff's hunks on the old side are lines 33, 34, 88, 176, 233–237 and 239, 319 — none inside 121–147 or 158–164. The single deletion is 233–237, the commented block.

Expect v1.5.0b5 to pause for approval. That is the feature working, not a regression: once you have configured the environments, the run will stop at github/tag and wait for you, then again at pypi/conda. The release-body selection under test runs inside github, so it runs after you approve wave 1 — the test still happens, it just happens on your click. If you would rather validate b5 unimpeded, merge this after b5 rather than before; the ordering is yours and nothing here depends on it.

YAML verified to parse

actionlint is not installed on this host (actionlint not found), so this is a YAML parse plus a structural read-back:

$ python -c "import yaml; d=yaml.safe_load(open('.github/workflows/create-release.yml')); ..."
PARSE OK
jobs: ['unit-tests', 'version_check', 'github', 'tag', 'pypi', 'conda']
  unit-tests     environment=None
  version_check  environment=None
  github         environment='github-release'
  tag            environment='conda-tag'
  pypi           environment='pypi'
  conda          environment='anaconda'
exit=0

Test: tests/project/test_release_gates.py

The rule asserted is not a copy of the current file: any job in this workflow that runs a publishing action or a git push must declare an environment:. So it also catches a new publishing job added later without a gate, and a job renamed or reordered.

How far that generalises, stated precisely, because it is easy to oversell — and the cross-review caught me overselling it in an earlier draft of this description. The detector is a finite, hand-maintained allowlist of four substrings, so the rule generalises across job identity and not across publishing mechanism. A job uploading by a route none of the four names — twine upload, gh release upload, a fork of a pinned action under another name, a curl straight at an index — matches no marker and would pass this test while genuinely ungated. The marker-liveness test only keeps the existing four from going blind; it cannot invent a fifth. Adding a publishing route to this workflow means adding its marker, and that is a maintenance obligation rather than something the test enforces. Both the module docstring and the PUBLISHING_MARKERS comment now say so.

Unit tier, and yaml is not in the test extra, so the scanner is hand-rolled and dependency-free; a separate test cross-checks it against yaml.safe_load when PyYAML happens to be installed, which is what stops the substitute from drifting. Same split as test_the_rewritten_date_is_a_string_and_not_a_yaml_date in test_bump_version.py.

Also pinned, against fixtures rather than the repo so they outlive this file: that #environment: release reads as ungated (the exact #641 shape), that a comment naming a publishing action is not evidence a job publishes, and that the environment:/name: mapping form is read.

Passing, with the fix — exit code read from the file, and the subtest tally is explicit rather than inferred from a PASSED line:

$ python -m pytest tests/project/test_release_gates.py -q
13 passed, 2 subtests passed in 0.06s
EXITCODE=0

Failing, without it. I restored HEAD's workflow by file copy (not by touching git state), re-ran, then restored the fix and confirmed the file was byte-identical again:

$ python -m pytest tests/project/test_release_gates.py -q
FAILED tests/project/test_release_gates.py::TestPublishingJobsAreGated::test_every_publishing_job_declares_an_environment
FAILED tests/project/test_release_gates.py::TestPublishingJobsAreGated::test_the_gated_jobs_are_the_publishing_ones
FAILED tests/project/test_release_gates.py::TestEnvironmentsAreNotShared::test_pypi_and_anaconda_are_separate
3 failed, 10 passed, 2 subtests passed in 0.09s
EXITCODE=1

with the message naming all four, and why each needed a gate:

these jobs publish with no environment:, so nothing holds them for approval: conda (uploads a package to anaconda.org, creates a public GitHub Release, and the tag it names); github (creates a public GitHub Release, and the tag it names); pypi (uploads a distribution to PyPI or TestPyPI, creates a public GitHub Release, and the tag it names); tag (pushes a commit or a tag to the repository)

Scoped suite, unaffected:

$ python -m pytest tests/project/ -q
109 passed, 1 warning, 471 subtests passed in 1.64s
EXITCODE=0

No CI verification claimed — the queue is deep and this may not report for a while.

Labels

ci only — the change is workflow configuration, matching the ci: subject prefix, as #658 (ci(release): ...) was labelled.

Not breaking, and the label's own description is what decides it: "Alters public API or wire output". This alters neither — no pcapkit API, CLI, or wire behaviour moves, and an installing user sees no difference. Two further reasons, in case the intent is broader than the description:

• On merge it changes no behaviour at all. Until the environments carry required reviewers, GitHub creates them implicitly unprotected and every job proceeds exactly as today. The behaviour change arrives when you configure the reviewers, deliberately — so breaking on this commit would point at the wrong commit.
• The release pipeline has exactly one operator, who is the person asking for the gate. There is no third party automating releases of this repository whose automation could break.

I'd still call out the real behavioural consequence plainly, since it is the point of the change rather than a side effect: once configured, a tag push no longer publishes on its own. The bump → tag → PyPI in a median three minutes chain becomes bump → tag → wait for a human. If you would rather that were flagged as breaking anyway, say so and I'll add it — it is additive.

Not test despite adding a test module: the type label tracks the subject prefix, and this is ci:.

Not in scope

No changelog entry on this branch — CHANGELOG.md and docs/source/changelog/ are untouched here, and the bullet goes to #657 (docs/changelog-1.5.0) as its own commit. .github/release.yml untouched. Nothing under pcapkit/ touched; this is workflow-only plus its test, as expected.

[JarryShaw]

GOOD TO GO

Cross-review by an independent agent on a different model (Sonnet) from the one that authored the change (Opus), briefed to falsify rather than bless, running read-only. Verdict verbatim: GOOD TO GO — no load-bearing claim failed verification. It reproduced every check itself rather than taking this description as a starting point, and it found one real overstatement in my prose, which I have now fixed (see the last section).

Independently confirmed

No publishing path left ungated — the finding that mattered most, since a partial gate would make this PR actively harmful. It read the whole file and enumerated the outward-facing sites itself: 3 × softprops/action-gh-release (in github, pypi, conda), 2 × pypa/gh-action-pypi-publish (both in pypi), 1 × anaconda/actions/upload-package (in conda), 2 × git push (both in tag). Every one sits inside a job carrying environment:. unit-tests and version_check have zero outward effect.

environment: is at job level on all four — confirmed by its own yaml.safe_load dump showing environment as a sibling key of permissions/needs/runs-on, not nested in any of them.

The reusable-workflow claim is true, and this is the one I was least certain of. It fetched GitHub's own SchemaStore workflow schema: the reusableWorkflowCallJob definition permits only name, needs, permissions, cache-mode, if, uses, with, secrets, strategy, concurrency, with additionalProperties: false, and environment is not among them. So unit-tests could not legally carry a gate, which is why it has none — and none of the four gated jobs is a reusable-workflow call. It also confirmed actionlint is genuinely absent on this host, matching the disclosure above.

The auto-approve premise holds — the other claim I could not verify locally, and the one the whole breaking-label argument rests on. From GitHub's own documentation: "Running a workflow that references an environment that does not exist will create an environment with the referenced name," and that environment "will not have any protection rules or secrets configured." Authoritative, and it means the ⚠️ warning at the top of this PR is not a caveat but the operative fact.

The b5 protection holds — its own diff -u against origin/main found 5 hunks at old-file lines 31, 86, 174, 230, 317, none overlapping the Select release body step or the body_path/generate_release_notes wiring, with the only deletion being the 5-line commented block.

Credentials unchanged — it extracted every secrets.* reference from both file versions and found the sorted multisets byte-identical; only line numbers moved.

The test genuinely fails without the fix — reproduced independently by sha256-tracked file copy rather than by touching git state: 3 failed / 10 passed / exit 1 without, 13 passed / exit 0 with, tests/project/ 109 passed / 471 subtests / exit 0. It verified by sha256 and git status --porcelain that it left the worktree exactly as found.

The environments read-only check — it re-ran gh api repos/JarryShaw/PyPCAPKit/environments itself: 2 environments, copilot and github-pages, both "protection_rules": [], none of the four new names present. Nothing created or modified. It also judged the per-environment-vs-per-matrix-leg hedge honestly stated, and could not verify it either without a real run.

What it disputed, and what I changed

One finding, and it was right. An earlier draft of this description called the test's rule "the general form" and claimed it catches "a new publishing job added later without a gate" without qualification. The reviewer's objection: PUBLISHING_MARKERS is a finite hand-maintained allowlist of four substrings, so the rule generalises across job identity but not across publishing mechanism — a twine upload, a gh release upload, a renamed fork of a pinned action, or a curl at an index would match nothing and pass while genuinely ungated. Sustained, and not cosmetic, because it is the difference between a guard and a guard people over-trust.

Fixed in e9fc31ab8 rather than argued away: the module docstring and the PUBLISHING_MARKERS comment now state the limitation and name it a maintenance obligation, and the section above says the same. The reviewer classed this as non-blocking; I would rather the file not carry the overstatement at all.

It also checked for vacuous assertions and found none — noting that the one theoretically vacuous-risk assertion (shared == {}, which would trivially hold if the environment reader always returned None) is covered by a sibling test that fails loudly in exactly that scenario.

Agreed as out of scope, with a caveat

deploy-pages.yml's docs deploy is confirmed genuinely ungated. The reviewer agreed with leaving it out of #641 — different blast radius, and a docs deploy is reversible where a consumed PyPI version number is not — but called it a real, lower-severity gap worth its own issue. I'll file it if you want it; I have not, since it was not asked for.

Provenance

No model substitution was needed: Sonnet was the intended reviewing model and the one that ran. One thing this verdict cannot do is record itself anywhere GitHub tracks, unlike a check run — hence posting it here next to the change rather than leaving it in a session transcript.