Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/workflows/cron-conda.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,10 @@ jobs:
conda-update:
name: Update requirements.txt
runs-on: ubuntu-latest
# #1968: the deploy key is a `conda-tag` environment secret, and only
# a job naming the environment can read it. Keep that environment free of
# required reviewers, or every scheduled bump waits for an approval.
environment: conda-tag
needs: [ unit-tests, verdict ]
# `needs` cannot itself be conditional, so a skipped gate (`workflow_run`)
# has to be accepted explicitly -- by default this job would inherit the
Expand Down Expand Up @@ -342,6 +346,10 @@ jobs:
# there, which runs on the same commits.
name: Conda Tag (update)
runs-on: ubuntu-latest
# #1968: the deploy key and the `PYPCAPKIT` PAT are `conda-tag` environment
# secrets, as for create-release.yml's `tag`. Keep that environment free of
# required reviewers, or every scheduled bump waits for an approval.
environment: conda-tag
needs: [ conda-update ]
# `!cancelled()` is what makes this job reachable at all on the
# `workflow_run` path (#1632). `unit-tests` above skips on that event, and
Expand Down Expand Up @@ -434,6 +442,10 @@ jobs:
conda-dist:
name: Conda deployment (update) on ${{ matrix.os }} with Python ${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
# #1968: `ANACONDA_TOKEN` is an `anaconda` environment secret, as for
# create-release.yml's `conda`. Keep that environment free of required
# reviewers, or every leg of a scheduled upload waits for an approval.
environment: anaconda
needs: [ conda-tag, conda-update ]
# #1632, exactly as for `conda-tag` above -- this job was skipped by the
# same implicit `success()` on the same runs.
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/cron-vendor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,10 @@ jobs:

vendor-update:
runs-on: macos-latest
# #1968: the deploy key is a `conda-tag` environment secret, and only
# a job naming the environment can read it. Keep that environment free of
# required reviewers, or every scheduled bump waits for an approval.
environment: conda-tag
needs: [ unit-tests, verdict ]
# `needs` cannot itself be conditional, so a skipped gate (`workflow_run`)
# has to be accepted explicitly -- by default this job would inherit the
Expand Down
25 changes: 21 additions & 4 deletions docs/source/contributing/workflows.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1193,13 +1193,30 @@ Deployment ``environment:``
These jobs declare a deployment ``environment:`` and can therefore pause for an
approval, in ``create-release.yml`` and ``cron-python-versions.yml``:

* ``github`` -- ``environment: github-release`` (``create-release.yml:618``)
* ``tag`` -- ``environment: conda-tag`` (``create-release.yml:806``)
* ``pypi`` -- ``environment: pypi`` (``create-release.yml:969``)
* ``conda`` -- ``environment: anaconda`` (``create-release.yml:1207``)
* ``github`` -- ``environment: github-release``
* ``tag`` -- ``environment: conda-tag``
* ``pypi`` -- ``environment: pypi``
* ``conda`` -- ``environment: anaconda``
* ``pr`` -- ``environment: github-release``, for the release App's key (see
`Python Versions`_)

``ANACONDA_TOKEN`` is a secret of the ``anaconda`` environment, and the
``PYPCAPKIT`` token and ``PYPCAPKIT_ACTIONS_DEPLOY_KEY`` are secrets of
``conda-tag``. GitHub gives an environment's secrets only to a job that names
it, so the scheduled jobs reading them name it too
(`#1968 <https://github.com/JarryShaw/PyPCAPKit/issues/1968>`__):

* Conda Update's ``conda-update`` and ``conda-tag`` -- ``environment: conda-tag``
* Conda Update's ``conda-dist`` -- ``environment: anaconda``
* Vendor Update's ``vendor-update`` -- ``environment: conda-tag``

A required reviewer on ``conda-tag`` or ``anaconda`` would therefore hold every
scheduled bump for an approval, besides the release-graph reason
:doc:`releasing` gives for keeping them without one. Each run of these jobs
also records a deployment to its environment.
:file:`tests/project/test_secret_environments_1968.py` checks that every job
reading one of these secrets names the environment holding it.

No other workflow in scope here declares one -- including
``deploy-pages.yml``, whose ``publish-pages`` job pushes straight to the
``gh-pages`` branch via ``JamesIves/github-pages-deploy-action`` rather than
Expand Down
6 changes: 3 additions & 3 deletions tests/project/test_release_gates.py
Original file line number Diff line number Diff line change
Expand Up @@ -244,9 +244,9 @@
#: :file:`cron-conda.yml` at the end of the file). It is read through
#: :data:`CONDA_WORKFLOW` below, for whether a bump reaches those uploads at
#: all; the ``environment:`` rule in this module is asserted over *this* file
#: only, and extending it to ``conda-dist`` -- which carries no
#: ``environment:`` -- is a #641 question nobody has asked yet rather than
#: something #1632 settles.
#: only. ``conda-dist`` names ``anaconda`` since #1968, but for its
#: ``ANACONDA_TOKEN`` rather than as an approval gate, which
#: :mod:`tests.project.test_secret_environments_1968` checks.
#:
#: :file:`cron-vendor.yml` does build and commit without uploading, and
#: :file:`deploy-pages.yml` publishes documentation rather than a package,
Expand Down
90 changes: 90 additions & 0 deletions tests/project/test_secret_environments_1968.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# -*- coding: utf-8 -*-
"""#1968: every job that reads an environment-scoped secret names its environment.

``ANACONDA_TOKEN`` lives in the ``anaconda`` environment, and the ``PYPCAPKIT``
PAT and ``PYPCAPKIT_ACTIONS_DEPLOY_KEY`` in ``conda-tag``, rather than at
repository level. GitHub hands an environment's secrets only to a job that
names it, so a job reading one without the matching ``environment:`` gets an
empty string -- and the scheduled jobs in Conda Update and Vendor Update did.

The scan covers every workflow, so a new job reading one of these secrets is
held to the same rule. YAML drops comments, so a secret named only in a comment
does not count.

"""

from __future__ import annotations

import re
import unittest
from typing import TYPE_CHECKING

from tests._workflows import WORKFLOWS, load, loads

if TYPE_CHECKING:
from typing import Any, Optional

#: Each environment-scoped secret, mapped to the environment that holds it.
SECRET_ENVIRONMENTS = {
'ANACONDA_TOKEN': 'anaconda',
'PYPCAPKIT': 'conda-tag',
'PYPCAPKIT_ACTIONS_DEPLOY_KEY': 'conda-tag',
}
#: A ``secrets.<name>`` reference in an expression.
SECRET = re.compile(r'\bsecrets\.(\w+)')


def environment(job: 'dict[str, Any]') -> 'Optional[str]':
"""The environment ``job`` names, in either spelling, or :data:`None`."""
value = job.get('environment')
if isinstance(value, dict):
return value.get('name')
return value


def readers() -> 'dict[tuple[str, str], set[str]]':
"""Each ``(workflow, job)`` reading an environment-scoped secret, with the environments those need."""
found = {} # type: dict[tuple[str, str], set[str]]
for path in sorted(WORKFLOWS.glob('*.yml')):
for job_id, job in (load(path.name).get('jobs') or {}).items():
needed = {SECRET_ENVIRONMENTS[name] for name in SECRET.findall(repr(job))
if name in SECRET_ENVIRONMENTS}
if needed:
found[(path.name, job_id)] = needed
return found


class TestSecretsReachTheirJobs(unittest.TestCase):
"""A job reading ``ANACONDA_TOKEN``, ``PYPCAPKIT`` or the deploy key names its environment."""

def test_every_reader_names_the_environment_holding_its_secrets(self) -> None:
for (name, job_id), needed in sorted(readers().items()):
with self.subTest(workflow=name, job=job_id):
# A job can name one environment only, so it cannot read both.
self.assertEqual(len(needed), 1, f'`{job_id}` reads secrets of {sorted(needed)}')
self.assertEqual(environment(load(name)['jobs'][job_id]), next(iter(needed)))

def test_the_scan_finds_the_known_readers(self) -> None:
"""Not vacuous: the scheduled jobs #1968 is about, and the release jobs."""
self.assertLessEqual({
('cron-conda.yml', 'conda-update'): {'conda-tag'},
('cron-conda.yml', 'conda-tag'): {'conda-tag'},
('cron-conda.yml', 'conda-dist'): {'anaconda'},
('cron-vendor.yml', 'vendor-update'): {'conda-tag'},
('create-release.yml', 'tag'): {'conda-tag'},
('create-release.yml', 'conda'): {'anaconda'},
}.items(), readers().items())

def test_a_commented_reference_does_not_count(self) -> None:
job = loads('jobs:\n a:\n # ${{ secrets.PYPCAPKIT }}\n runs-on: x\n'
' steps:\n - run: echo ${{ secrets.ANACONDA_TOKEN }}\n')['jobs']['a']
self.assertEqual(SECRET.findall(repr(job)), ['ANACONDA_TOKEN'])

def test_both_spellings_of_environment(self) -> None:
self.assertIsNone(environment({}))
self.assertEqual(environment({'environment': 'anaconda'}), 'anaconda')
self.assertEqual(environment({'environment': {'name': 'anaconda', 'url': 'x'}}), 'anaconda')


if __name__ == '__main__':
unittest.main()
4 changes: 0 additions & 4 deletions tests/project/test_unit_tests_queue.py
Original file line number Diff line number Diff line change
Expand Up @@ -655,13 +655,9 @@ def test_the_scan_is_not_blind(self) -> None:
('version_check', 'if: >-', "needs.vendor_update.outputs.updated != 'false' }}"),
('unit-tests', 'needs: [ version_check ]', None),
('unit-tests', 'uses: ./.github/workflows/unit-tests.yml', None),
('github', 'environment: github-release', None),
('github', 'needs: [ unit-tests, version_check ]', None),
('tag', 'environment: conda-tag', None),
('tag', 'needs: [ github, unit-tests, version_check ]', None),
('pypi', 'environment: pypi', None),
('pypi', 'needs: [ github, unit-tests, version_check ]', None),
('conda', 'environment: anaconda', None),
('conda', 'needs: [ tag, github, unit-tests, version_check ]', None),
],
}
Expand Down
Loading