Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 40 additions & 8 deletions .github/workflows/create-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,9 @@ concurrency:
# exists, ``pypi`` PyPI's file count, and ``conda`` Anaconda's aggregate count
# (a cost saver) plus, per leg, its own distribution -- the correctness fix,
# since ``anaconda/actions/upload-package`` has no ``skip-existing``, only
# ``force:``, never set here. The ``macos-26-intel`` leg uploads with the
# anaconda-client CLI and ``--skip-existing`` instead (#2044).
# ``force:``, never set here. The ``macos-26-intel`` and ``windows-latest``
# legs upload with the anaconda-client CLI and ``--skip-existing`` instead
# (#2044, #2060).
#
# GitHub skips a job whose ``needs:`` skipped before evaluating its ``if:``,
# unless the ``if:`` calls a status-check function, and ``github`` skips on
Expand Down Expand Up @@ -292,6 +293,8 @@ jobs:
env:
PCAPKIT_VERSION: ${{ steps.get_version.outputs.PCAPKIT_VERSION }}
PCAPKIT_CONDA_BUILDS: ${{ steps.released.outputs.conda-builds }}
# #2060: once tagged, the platforms its tag's commit published.
PCAPKIT_TAG_EXISTS: ${{ steps.check_tag.outputs.exists }}
run: python3 util/release_published.py anaconda /tmp/anaconda_release.json

# #1748: resolved as #1634's check in ``github``, ``pypi`` and ``conda``
Expand Down Expand Up @@ -749,8 +752,9 @@ jobs:
- macos-latest
- macos-26-intel
- ubuntu-latest
# No Windows leg since 2023, and no Windows steps (#1967): adding the leg
# back means writing its steps, as test_conda_no_windows_steps_1967 pins.
# #2060: win-64, once Conda Update's leg had built and smoke-tested it
# green. Its `run:` steps name `bash`, as Windows runs `pwsh`.
- windows-latest
# #1633, #1660: the supported tier only. No KTLO 3.10 leg: the list
# util/conda_vendor.py derives omits typing-extensions, which
# pcapkit/utilities/compat.py imports below 3.11, and pins each
Expand All @@ -775,9 +779,11 @@ jobs:
# Anaconda check, so this reads the live state. #1967: util/release_conda.py
# holds this step's shell, and the build's and the Intel upload's, which
# cron-conda.yml's ``conda-dist`` runs too; before setup-python, so here
# on the image's ``python3``.
# on the image's ``python3``. On Windows, Git Bash hands it ``/tmp`` as
# the user's temporary directory.
- name: Check whether this leg is already on Anaconda
id: check_leg
shell: bash
env:
PCAPKIT_VERSION: ${{ needs.version_check.outputs.PCAPKIT_VERSION }}
PCAPKIT_CONDA_OS: ${{ matrix.os }}
Expand All @@ -795,12 +801,14 @@ jobs:
# installed here: on ``macos-26-intel`` they resolve a cryptography with
# no Intel wheel, which pip has to compile or fail on.
- name: Install and Setup
shell: bash
run: |
set -x

python -m pip install -U pip setuptools wheel

- name: Setup Conda Distribution
shell: bash
run: |
set -x

Expand Down Expand Up @@ -836,7 +844,10 @@ jobs:
conda-remove-defaults: false
show-channel-urls: true

# A login shell, as for the build below: on Windows only that one has the
# `conda` setup-miniconda's `conda init` defines.
- name: Install conda build tools
shell: bash -el {0}
run: |
conda install -n base --override-channels -c conda-forge -y conda-build
conda build --version
Expand All @@ -862,15 +873,21 @@ jobs:
# package is checked as a Conda Update bump's is, before anything below
# uploads it. Not gated on ``check_leg``, since the GitHub upload is not.
- name: Smoke-test the vendored packages
shell: bash
env:
PCAPKIT_CONDA_PACKAGE: ${{ steps.conda_build.outputs.host_package }}
run: |
"${pythonLocation}/bin/python" util/conda_vendor.py --smoke "$PCAPKIT_CONDA_PACKAGE"
python="${pythonLocation}/bin/python"
if [ "$RUNNER_OS" = Windows ]; then
python="$(cygpath -u "$pythonLocation")/python.exe"
fi
"$python" util/conda_vendor.py --smoke "$PCAPKIT_CONDA_PACKAGE"

# #2044: not on macos-26-intel, whose leg "Upload conda packages to
# Anaconda (Intel)" uploads instead.
# Anaconda (Intel)" uploads instead, nor, #2060, on windows-latest, whose
# "Upload conda packages to Anaconda (Windows)" does.
- name: Upload conda packages to Anaconda
if: ${{ matrix.os != 'macos-26-intel' && steps.check_leg.outputs.PCAPKIT_LEG_EXISTS != 'true' }}
if: ${{ matrix.os != 'macos-26-intel' && matrix.os != 'windows-latest' && steps.check_leg.outputs.PCAPKIT_LEG_EXISTS != 'true' }}
uses: anaconda/actions/upload-package@a6f488910b4a1e4f3fe01736241ca1b92b4b355a # v0.3.1
with:
token: ${{ secrets.ANACONDA_TOKEN }}
Expand Down Expand Up @@ -898,6 +915,21 @@ jobs:
PCAPKIT_CONDA_LABELS: ${{ needs.version_check.outputs.PCAPKIT_CONDA_LABEL }}
run: python3 util/release_conda.py upload

# #2060: the Intel leg's call, on win-64, under a login shell as "Build
# conda packages" is: only there is base active, so the script can start
# `conda` as base's `Scripts\conda.exe`, and `python` is base's. The
# script runs base's `Scripts\anaconda.exe`.
- name: Upload conda packages to Anaconda (Windows)
if: ${{ matrix.os == 'windows-latest' && steps.check_leg.outputs.PCAPKIT_LEG_EXISTS != 'true' }}
shell: bash -el {0}
env:
ANACONDA_API_TOKEN: ${{ secrets.ANACONDA_TOKEN }}
ANACONDA_CLI_FORCE_NEW: "1"
PCAPKIT_CONDA_CHANNEL: jarryshaw
PCAPKIT_CONDA_PACKAGES: ${{ steps.conda_build.outputs.packages }}
PCAPKIT_CONDA_LABELS: ${{ needs.version_check.outputs.PCAPKIT_CONDA_LABEL }}
run: python util/release_conda.py upload

# ``pypi``'s upload script, for the same reasons (#1634, #1640). Not
# gated on ``check_leg`` as the Anaconda upload is: a leg whose Anaconda
# upload landed and whose GitHub upload then failed would get no GitHub
Expand Down
58 changes: 19 additions & 39 deletions .github/workflows/cron-conda.yml
Original file line number Diff line number Diff line change
Expand Up @@ -445,12 +445,6 @@ jobs:
))
permissions:
contents: write
# #2060: the Windows leg builds and smoke-tests only, and never blocks. Each
# step that can fail on it has an `id`, and a `continue-on-error` true on
# `windows-latest` alone, so a failure leaves the leg at success, and
# `Windows summary` reports it as a warning, as the KTLO legs do (#1672).
# No job-level `continue-on-error`, which would still turn the leg's check
# red (#1672), so a lost runner fails the job.
strategy:
fail-fast: false
matrix:
Expand All @@ -460,20 +454,17 @@ jobs:
- macos-latest
- macos-26-intel
- ubuntu-latest
# #2060: builds and smoke-tests win-64, and publishes nothing until
# its runs pass. Every upload step below is gated off it, so the
# `ANACONDA_TOKEN` this job's `environment:` holds reaches no step on it.
# #2060: win-64, uploaded with the anaconda-client CLI as the Intel
# leg is, now that its runs have built and smoke-tested it green.
- windows-latest
# #1633: the supported tier, from .github/python-versions.toml through
# `conda-update` (#1660); create-release.yml's ``conda`` job says why
# there is no KTLO 3.10 leg.
python-version: ${{ fromJSON(needs.conda-update.outputs.PYTHON_CONDA) }}
steps:
# No persisted token: nothing here runs `git` with credentials, and the
# Windows leg builds a package nobody has published yet (#2060).
# No persisted token: nothing here runs `git` with credentials (#2060).
- id: checkout
if: github.event_name != 'workflow_dispatch'
continue-on-error: ${{ matrix.os == 'windows-latest' }}
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: conda-${{ needs.conda-update.outputs.PCAPKIT_VERSION }}+${{ needs.conda-update.outputs.PCAPKIT_BUILD }}
Expand All @@ -484,7 +475,6 @@ jobs:
# changes `conda-update` made to it and did not push.
- id: checkout-dispatched
if: github.event_name == 'workflow_dispatch'
continue-on-error: ${{ matrix.os == 'windows-latest' }}
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
Expand All @@ -493,7 +483,6 @@ jobs:
- name: Download conda-update's changes
id: download-changes
if: github.event_name == 'workflow_dispatch'
continue-on-error: ${{ matrix.os == 'windows-latest' }}
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: conda-changes
Expand All @@ -502,7 +491,6 @@ jobs:
- name: Apply conda-update's changes
id: apply-changes
if: github.event_name == 'workflow_dispatch'
continue-on-error: ${{ matrix.os == 'windows-latest' }}
shell: bash
env:
PATCH_FILE: ${{ runner.temp }}/conda-changes/conda-changes.patch
Expand All @@ -512,7 +500,6 @@ jobs:
fi

- id: setup-python
continue-on-error: ${{ matrix.os == 'windows-latest' }}
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
Expand All @@ -521,7 +508,6 @@ jobs:
# the same name. `shell: bash` here and below, as Windows runs `pwsh`.
- name: Install and Setup
id: install
continue-on-error: ${{ matrix.os == 'windows-latest' }}
shell: bash
run: |
set -x
Expand All @@ -534,7 +520,6 @@ jobs:
# this downloaded.
- name: Setup Conda Distribution
id: download
continue-on-error: ${{ matrix.os == 'windows-latest' }}
shell: bash
run: |
set -x
Expand All @@ -546,7 +531,6 @@ jobs:
# exceptions (#2123).
- name: Conda environment creation and activation
id: miniconda
continue-on-error: ${{ matrix.os == 'windows-latest' }}
uses: conda-incubator/setup-miniconda@be893c923ea9cf1cf7cd510fbdde27c7e18cbdcb # v4.1.0
env:
PCAPKIT_VERSION: ${{ needs.conda-update.outputs.PCAPKIT_VERSION }}
Expand All @@ -567,7 +551,6 @@ jobs:
# `conda` setup-miniconda's `conda init` defines.
- name: Install conda build tools
id: conda-build-tools
continue-on-error: ${{ matrix.os == 'windows-latest' }}
shell: bash -el {0}
run: |
conda install -n base --override-channels -c conda-forge -y conda-build
Expand All @@ -578,7 +561,6 @@ jobs:
# it fails a package built for another platform than the leg's.
- name: Build conda packages
id: conda_build
continue-on-error: ${{ matrix.os == 'windows-latest' }}
shell: bash -el {0}
env:
PCAPKIT_VERSION: ${{ needs.conda-update.outputs.PCAPKIT_VERSION }}
Expand All @@ -599,8 +581,6 @@ jobs:
# is `python.exe` at the top of `pythonLocation`, not under `bin/` (#2060).
- name: Smoke-test the vendored packages
id: smoke
if: steps.conda_build.outcome == 'success'
continue-on-error: ${{ matrix.os == 'windows-latest' }}
shell: bash
env:
PCAPKIT_CONDA_PACKAGE: ${{ steps.conda_build.outputs.host_package }}
Expand All @@ -612,10 +592,9 @@ jobs:
"$python" util/conda_vendor.py --smoke "$PCAPKIT_CONDA_PACKAGE"

# #2044: not on macos-26-intel, whose leg "Upload conda packages to
# Anaconda (Intel)" uploads instead. #2060: nor on windows-latest, which
# publishes nothing until a run has shown its package works.
# Neither upload, nor the GitHub Release steps after them, runs on a
# manual run.
# Anaconda (Intel)" uploads instead, nor, #2060, on windows-latest, whose
# "Upload conda packages to Anaconda (Windows)" does. No upload, nor the
# GitHub Release steps after them, runs on a manual run.
- name: Upload conda packages to Anaconda
if: ${{ github.event_name != 'workflow_dispatch' && matrix.os != 'macos-26-intel' && matrix.os != 'windows-latest' }}
uses: anaconda/actions/upload-package@a6f488910b4a1e4f3fe01736241ca1b92b4b355a # v0.3.1
Expand All @@ -640,6 +619,19 @@ jobs:
PCAPKIT_CONDA_LABELS: ${{ needs.conda-update.outputs.PCAPKIT_CONDA_LABEL }}
run: python3 util/release_conda.py upload

# #2060: see create-release.yml's step of the same name, which says why
# it runs under a login shell on conda's base `python`.
- name: Upload conda packages to Anaconda (Windows)
if: ${{ github.event_name != 'workflow_dispatch' && matrix.os == 'windows-latest' }}
shell: bash -el {0}
env:
ANACONDA_API_TOKEN: ${{ secrets.ANACONDA_TOKEN }}
ANACONDA_CLI_FORCE_NEW: "1"
PCAPKIT_CONDA_CHANNEL: jarryshaw
PCAPKIT_CONDA_PACKAGES: ${{ steps.conda_build.outputs.packages }}
PCAPKIT_CONDA_LABELS: ${{ needs.conda-update.outputs.PCAPKIT_CONDA_LABEL }}
run: python util/release_conda.py upload

# #1634: the upload below never makes the release. ncipollo/release-action
# v1.21.0 creates a missing one, and its tag, at ``commit``
# (src/Action.ts:72-92), so a bump made while Create Release waits on
Expand Down Expand Up @@ -702,18 +694,6 @@ jobs:
tag: "v${{ needs.conda-update.outputs.PCAPKIT_VERSION }}"
token: "${{ secrets.GITHUB_TOKEN }}"

# #2060: the Windows leg's last step, as `KTLO summary` is a KTLO leg's
# (#1672): one warning per failed step, from each `outcome`. setup-python's
# `python`, or the image's where that failed.
- name: Windows summary
if: ${{ !cancelled() && matrix.os == 'windows-latest' }}
continue-on-error: true
shell: bash
env:
LEG: conda-dist on ${{ matrix.os }} with Python ${{ matrix.python-version }} (unpublished, non-blocking)
STEPS: ${{ toJSON(steps) }}
run: python util/leg_summary.py conda-windows

# #2030: posts `conda-update`'s report with the tag and upload results, and a
# short one of its own where `head` failed or was cancelled, as cron-vendor.yml's
# `discussion` job does, whose comment gives the rules. It waits for
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/experimental.yml
Original file line number Diff line number Diff line change
Expand Up @@ -705,8 +705,7 @@ jobs:
retention-days: 14

# `experimental`'s, on setup-python's `python`, or the image's where that
# failed, since Windows has no /usr/bin/python3, as cron-conda.yml's
# `Windows summary` runs it (#2060).
# failed, since Windows has no /usr/bin/python3 (#2060).
- name: Experimental summary
if: ${{ !cancelled() }}
continue-on-error: true
Expand Down
17 changes: 11 additions & 6 deletions docs/source/contributing/releasing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -343,9 +343,9 @@ checks *is* the one it is about to create:
already idempotent (``skip-existing: true`` on every upload step), so the
check exists only to save the cost of the build matrix, not for correctness.
* ``conda`` compares Anaconda's aggregate count against the expected one (the
``linux-64``, ``osx-64`` and ``osx-arm64`` files for each Python version of
its matrix, one platform per ``os`` leg,
`#1922 <https://github.com/JarryShaw/PyPCAPKit/issues/1922>`__) as the same
``linux-64``, ``osx-64``, ``osx-arm64`` and ``win-64`` files for each Python
version of its matrix, one platform per ``os`` leg,
`#1922 <https://github.com/JarryShaw/PyPCAPKit/issues/1922>`__, :issue:`2060`) as the same
kind of job-level cost saver, and *also*
checks, per matrix leg, whether that leg's own platform/Python distribution
is already there -- this second check is the actual correctness fix, because
Expand All @@ -355,9 +355,10 @@ checks *is* the one it is about to create:
action's CLI download returns 404 there, so its "Upload conda packages to
Anaconda (Intel)" step runs ``anaconda upload --skip-existing`` instead
(`#2044 <https://github.com/JarryShaw/PyPCAPKit/issues/2044>`__), which skips
such a file. Conda Update's Intel leg runs the same step, so it keeps a file
anaconda.org already has, where the action's ``force: true`` on its other legs
replaces it.
such a file. The ``windows-latest`` leg's "Upload conda packages to Anaconda
(Windows)" makes the same call (:issue:`2060`). Conda Update's Intel and
Windows legs run the same steps, so they keep a file anaconda.org already has,
where the action's ``force: true`` on its other legs replaces it.

These evidence checks all run inside ``version_check``, curling PyPI's and
Anaconda's public, unauthenticated JSON APIs; the per-leg check runs inside
Expand All @@ -374,6 +375,10 @@ gate and ``pypi``, and fail `One Commit per Release`_'s tag check on every run
until the next release. A tag older than the file, as ``v1.5.0b12`` is, counts
the tiers the file was added with, which the change for `#1660 <https://github.com/JarryShaw/PyPCAPKit/issues/1660>`__ copied unchanged from the
workflows of the time.
The Anaconda count's platforms follow the same rule, from the ``SUBDIRS`` line
of :file:`util/release_published.py` at the tag's commit or, for a tag older
than that file, the line it was added with, so a version released before
``win-64`` was published is not read as lacking it (:issue:`2060`).

On the tag-push path ``startsWith(github.ref_name, 'v')`` bypasses the
evidence check unconditionally; on the ``workflow_run`` path it falls through
Expand Down
18 changes: 9 additions & 9 deletions docs/source/contributing/workflows.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1376,7 +1376,8 @@ to a script. Each script's docstring says what it does; each has tests under
conda packages to Anaconda (Intel)`` in that job and in ``cron-conda.yml``'s
``conda-dist``, as ``release_conda.py build`` and ``upload`` (`#1822
<https://github.com/JarryShaw/PyPCAPKit/issues/1822>`__, `#2044
<https://github.com/JarryShaw/PyPCAPKit/issues/2044>`__).
<https://github.com/JarryShaw/PyPCAPKit/issues/2044>`__), and ``Upload conda
packages to Anaconda (Windows)`` in both as ``upload`` (:issue:`2060`).
- No.
* - :file:`util/release_reset_build.py`
- ``Reset Build Number`` in ``create-release.yml``'s ``tag``, on the image's
Expand Down Expand Up @@ -1789,14 +1790,13 @@ it, so the scheduled jobs reading them name it too
* Conda Update's ``conda-dist`` -- ``environment: anaconda``
* Vendor Update's ``vendor-update`` -- ``environment: conda-tag``

``conda-dist`` also has a ``windows-latest`` leg, which builds and smoke-tests a
``win-64`` package and publishes nothing
(`#2060 <https://github.com/JarryShaw/PyPCAPKit/issues/2060>`__). Every upload
step is gated off it, and those are the only steps that read ``ANACONDA_TOKEN``,
so no step on that leg receives the secret.
:file:`tests/project/test_conda_windows_leg_2060.py` checks both. Each step that
can fail there continues on error, and ``Windows summary`` reports each failure
as a warning, as ``KTLO summary`` does for a KTLO leg, so the leg stays green.
Both ``conda-dist`` and the release ``conda`` job have a ``windows-latest`` leg,
which builds, smoke-tests and publishes the ``win-64`` package (:issue:`2060`).
It blocks as the other legs do: no step continues on error. Its one step that
reads ``ANACONDA_TOKEN`` is ``Upload conda packages to Anaconda (Windows)``, the
Intel leg's anaconda-client call (:issue:`2044`) under ``bash -el``, which skips a
manual run as every upload does.
:file:`tests/project/test_conda_windows_leg_2060.py` pins the leg.

A required reviewer on ``conda-tag`` or ``anaconda`` would therefore hold every
scheduled bump for an approval, besides the release-graph reason
Expand Down
Loading
Loading