ci(unit-tests): run the whole suite on latest CPython (#1958)

.github/workflows/unit-tests.yml

@@ -142,10 +142,12 @@ jobs:
       # crypto (cryptography) and NGAP (pycrate) join it for the same reason,
       # covering #738's remaining HAS_CRYPTO gate (test_esp_unit.py) and its
       # HAS_PYCRATE gate (test_ngap_unit.py), both plain
-      # unit-tier. `cli` (emoji) is deliberately NOT added here: its only
+      # unit-tier. `cli` (emoji) is deliberately not on this line: its only
       # HAS_EMOJI-gated tests are in tests/integration/test_cli_subprocess.py,
       # which this job ignores wholesale, so installing it here would unlock
-      # nothing. cryptography is the one compiled wheel of the two; measured
+      # nothing. The `latest` leg gets it from the whole-suite step below
+      # (#1958), whose set every `latest` job shares, and there too it unlocks
+      # nothing in this job. cryptography is the one compiled wheel of the two; measured
       # clean installs on 3.10-3.14 (manylinux wheels, no build step). pycrate
       # is pure Python but its distribution is large (~238MB, see the NGAP
       # extra's comment in pyproject.toml) -- still cheap in wall time, just not
@@ -187,6 +189,23 @@ jobs:
             exit 1
           fi
 
+      # #1958, the owner's ruling E: latest CPython runs the whole suite, so its
+      # leg installs everything the suite gates on that pip can install there --
+      # the engines (Scapy, PyShark), html5lib through `vendor`, Sphinx and the
+      # pinned sphinx-autodoc-typehints through `docs`, emoji through `cli`. The
+      # tests those gate skip on every other leg, which runs the minimal `ALL`
+      # tier. tests/_dependency_gates.py reads this line as the `latest` leg's
+      # extras; tests/project/test_whole_suite_on_latest_1958.py pins it to the
+      # same set as `integration`, `project-tests` and `gate`. Not here: PCAP_CT,
+      # which needs a system libpcap that `engine-tests` installs for its own
+      # cell, and PyPCAP and PyPCAPFile, whose markers resolve to nothing on
+      # `latest`. Never a KTLO leg, so `test-ktlo` always skips it.
+      - name: Install the whole suite's dependencies on latest
+        id: install-latest
+        if: ${{ steps.install.outcome == 'success' && matrix.python-version == needs.python-versions.outputs.latest }}
+        continue-on-error: ${{ matrix.ktlo == true }}
+        run: python -m pip install -e '.[test,Scapy,DPKT,crypto,cli,NGAP,vendor,docs,PyShark]'
+
       # 7.10 is the first release with `[run] patch = ["subprocess"]`, which
       # .github/coverage.toml relies on to measure the xdist workers.
       - name: Install coverage
@@ -500,10 +519,12 @@ jobs:
       # selection (tests/_tiers.py's fixture_tier_paths()) always includes the
       # whole of tests/integration/ -- which is where every HAS_EMOJI-gated
       # method lives (test_cli_subprocess.py). `crypto` and `NGAP`
-      # are deliberately NOT added here: test_esp_unit.py and test_ngap_unit.py
+      # are deliberately not on this line: test_esp_unit.py and test_ngap_unit.py
       # are plain unit-tier and never enter this job's selection, so installing
       # cryptography/pycrate here would unlock nothing -- they are already
-      # covered, on every Python version, by the `test` job above.
+      # covered, on every Python version, by the `test` job above. The `latest`
+      # leg gets both from the whole-suite step below (#1958), whose set every
+      # `latest` job shares, and there too they unlock nothing in this job.
       # PyPCAPFile is NOT added here either, even though this job's selection
       # also reaches test_new_engine_parity_runtime.py's HAS_PYPCAPFILE
       # methods -- see the `test` job's comment above for the bug that used to
@@ -528,6 +549,14 @@ jobs:
             exit 1
           fi
 
+      # #1958: the `test` job's step of the same name. On `latest` this job runs
+      # the whole fixture tier, so it installs the whole suite's set too.
+      - name: Install the whole suite's dependencies on latest
+        id: install-latest
+        if: ${{ steps.install.outcome == 'success' && matrix.python-version == needs.python-versions.outputs.latest }}
+        continue-on-error: ${{ matrix.ktlo == true }}
+        run: python -m pip install -e '.[test,Scapy,DPKT,crypto,cli,NGAP,vendor,docs,PyShark]'
+
       # `shell: bash` rather than the default, because it runs with -o pipefail:
       # without it the `tee` below would swallow the generator's exit status.
       - name: Regenerate sample captures
@@ -1785,12 +1814,12 @@ jobs:
           python-version: ${{ steps.python-versions.outputs.latest }}
           cache: pip
 
-      # `test`'s install line, so a gated pin skips or runs here exactly as it
-      # did on `latest`.
+      # The whole suite's set, the line `test` adds on `latest` (#1958), so a
+      # gated pin runs here exactly as the rest of its tier does there.
       - name: Install package and test dependencies
         run: |
           python -m pip install -U pip setuptools wheel
-          python -m pip install -e '.[test,DPKT,crypto,NGAP]'
+          python -m pip install -e '.[test,Scapy,DPKT,crypto,cli,NGAP,vendor,docs,PyShark]'
 
       # See the `test` job above for why this step exists.
       - name: Report available parallelism
@@ -1869,10 +1898,15 @@ jobs:
       # would be a no-op that misleadingly suggests the extra is exercised by
       # `gate`; #751's `engine-tests` and `pypcap-parity` jobs cover it for
       # real, on the 3.10-3.15 (and 3.10-3.11) legs, 3.10 being KTLO in both.
+      #
+      # #1958: `vendor`, `docs` and `PyShark` join them, so this is the set the
+      # `latest` legs install (see `test`'s "Install the whole suite's
+      # dependencies on latest" step), and the html5lib, Sphinx and PyShark
+      # gates run here as they do there.
       - name: Install package, test and generator dependencies
         run: |
           python -m pip install -U pip setuptools wheel
-          python -m pip install -e '.[test,Scapy,DPKT,crypto,cli,NGAP]'
+          python -m pip install -e '.[test,Scapy,DPKT,crypto,cli,NGAP,vendor,docs,PyShark]'
 
       # See the integration job above for why this step is shaped the way it is.
       - name: Regenerate sample captures

docs/source/contributing/releasing.rst

@@ -52,7 +52,7 @@ Editing ``__version__`` by hand without also moving :file:`CITATION.cff`'s
 (``:494-514``) asserts the two agree, and ``create-release.yml``'s
 ``unit-tests`` job (``:581-597``) calls ``unit-tests.yml`` with
 ``gate-only: true``, which runs the **full** suite rather than the tiered
-subset an ordinary push runs (its ``gate`` job, ``unit-tests.yml:1842-1912``).
+subset an ordinary push runs (its ``gate`` job, ``unit-tests.yml:1871-1946``).
 That gate runs whenever a release will -- a moved ``__version__`` is exactly
 what makes ``version_check``'s evidence read "not yet published" -- and every
 publishing job requires it to have succeeded (``:621,808,974,1210``), so a
@@ -62,7 +62,7 @@ Either run the script, or move both fields by hand in the same commit.
 This is the **only** edit a person makes to get a release started -- the tag,
 the Release, and every upload are the workflow's job from here. Other
 checks run unconditionally on the same path, though none is a step in *this*
-process: the ``changelog`` job (``unit-tests.yml:1646-1672``) fails outright if
+process: the ``changelog`` job (``unit-tests.yml:1675-1701``) fails outright if
 ``CHANGELOG.md`` has drifted from its source entry under
 :file:`docs/source/changelog/`, and the release-body step warns, without
 failing, if that entry's heading still reads "unreleased"

docs/source/contributing/workflows.rst

@@ -329,7 +329,7 @@ the same commit several times over. It also skips ``changes``,
 Checks`_ below), which are gated out by their own ``if:`` rather than by having
 already run. ``changelog`` carries no
 ``if:`` at all and runs on every path regardless -- deliberately, per its own
-comment (``unit-tests.yml:1634-1640``): ``create-release.yml`` feeds
+comment (``unit-tests.yml:1663-1669``): ``create-release.yml`` feeds
 ``CHANGELOG.md`` to the GitHub Release body, so the release path is exactly
 where a drifted file must not go unchecked. Confirmed on run `36210743295
 <https://github.com/JarryShaw/PyPCAPKit/actions/runs/36210743295>`__ (a
@@ -639,7 +639,7 @@ grepping every workflow file for its name:
      - Where
    * - ``Required checks passed``
      - job ``required-checks``
-     - ``unit-tests.yml:2003``
+     - ``unit-tests.yml:2037``
    * - ``Compat Python 3.11``
      - job ``compatibility``, matrix leg ``3.11``
      - ``python-compatibility.yml:60``, ``.github/python-versions.toml:20``
@@ -661,22 +661,22 @@ which added ``Compat Python 3.15`` and removed ``Compat Python 3.10``. Moving a
 between the tiers below means editing the ruleset in the same change.
 
 ``Required checks passed`` is defined by exactly one job
-(``unit-tests.yml:2003``). ``Compat Python`` is defined by one, the
+(``unit-tests.yml:2037``). ``Compat Python`` is defined by one, the
 ``compatibility`` job in ``python-compatibility.yml`` (``:60``), whose matrix
 (``:74``) comes from :file:`.github/python-versions.toml`: the required legs
 are its supported tier (``python-versions.toml:20-24``), and its KTLO legs
 (``python-versions.toml:16-19``) have names ending in ``(KTLO)``, so no required
 context matches one. A comment at
-``unit-tests.yml:1915`` mentions the string without defining it.
+``unit-tests.yml:1949`` mentions the string without defining it.
 
 ``Required checks passed`` is an aggregate, not a single check run. The ruleset
 requires it and the ``Compat Python 3.11``-``3.15`` legs, nothing else, as
-``unit-tests.yml``'s own comment (``unit-tests.yml:1914-1924``) records. It
+``unit-tests.yml``'s own comment (``unit-tests.yml:1948-1958``) records. It
 ``needs:`` the gating jobs of ``unit-tests.yml`` -- ``test``, ``integration``,
 ``engine-tests`` and ``pypcap-parity``, plus ``project-tests``, the one run of
 the project pins, and ``changes`` for the docs-only path -- and checks each
 result explicitly
-(``unit-tests.yml:2002-2058``). Rulesets match check names literally, with no
+(``unit-tests.yml:2036-2092``). Rulesets match check names literally, with no
 wildcard, so requiring the matrix cells one by one would mean editing the
 ruleset whenever a matrix changes; the aggregate keeps the required list fixed.
 It accepts the gating jobs as ``skipped`` only on a docs-only pull request, where
@@ -685,7 +685,7 @@ It accepts the gating jobs as ``skipped`` only on a docs-only pull request, wher
 ``python-compatibility.yml``, and a job cannot ``needs:`` a job in another
 workflow file. This job runs on Unit Tests' own ``push``/``pull_request``
 triggers; the ``gate-only: true`` reusable calls documented above skip it via
-``if: ${{ always() && inputs.gate-only != true }}`` (``unit-tests.yml:2005``),
+``if: ${{ always() && inputs.gate-only != true }}`` (``unit-tests.yml:2039``),
 so it is never produced -- and never expected -- on those paths.
 
 .. _ktlo-tier:
@@ -709,7 +709,7 @@ and keeps its legs' check names, ``Python 3.10 (KTLO)`` and so on. In
 with supported legs. Every job holding KTLO legs works the same way:
 
 * Every step that can fail on a KTLO leg has an ``id`` and
-  ``continue-on-error: ${{ matrix.ktlo == true }}`` (``unit-tests.yml:424-441``),
+  ``continue-on-error: ${{ matrix.ktlo == true }}`` (``unit-tests.yml:443-460``),
   unless it is masked on every leg, as the timings uploads are. Its ``outcome`` is
   then ``failure`` but its ``conclusion`` ``success``, so the job, its check run
   and ``required-checks`` read ``success``. A step that needs an earlier one's work
@@ -976,6 +976,23 @@ The project pins stay in a job of their own rather than joining the ``latest`` l
 rather than lengthening it under coverage, and it is also the job that runs the docs
 readers on a docs-only diff.
 
+Latest CPython runs the whole suite, as the sanity check and the gate (the owner's
+ruling on #1958), so it also installs what the whole suite gates on. In ``test`` and
+``integration`` the step ``Install the whole suite's dependencies on latest`` runs on
+the ``latest`` leg alone, after the install every leg runs, and adds the engines pip
+can install there (``Scapy``, ``PyShark``), html5lib through ``vendor``, Sphinx and
+the pinned sphinx-autodoc-typehints through ``docs``, and emoji through ``cli``.
+``project-tests`` and ``gate``, which run on ``latest`` only, install the same set in
+their ``Install package and test dependencies`` and ``Install package, test and
+generator dependencies`` steps. The other legs keep their install lines and run the
+``ALL`` tier with them, so a test gated on one of these extras skips there and runs on
+``latest``. The other engines stay out: ``PCAP_CT`` needs a system libpcap, which
+``engine-tests`` installs for its own cell, and the ``PyPCAP`` and ``PyPCAPFile``
+markers resolve to nothing on ``latest``. :file:`tests/_dependency_gates.py` reads the
+``latest`` step as that leg's extras, and
+:file:`tests/project/test_whole_suite_on_latest_1958.py` pins the set and the gates it
+closes.
+
 Project Pins
 ------------