From 9529cd1291494a4cd41059cb70ef9e0dd91ad1c8 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Fri, 2 Oct 2026 10:58:19 -0400 Subject: [PATCH 1/2] ci(tests): run tests/vendor in the unittest-ordering job (#981, #985) - tests/vendor was excluded from the unittest-ordering matrix because it held a stale VendorRuntimeWarning generation (#985); that defect is fixed. - Add `vendor` to the `leg` matrix and drop its bullet from the exclusion comment; the count of excluded directories goes from three to two. - tests/corekit and tests/integration stay excluded, bullets unchanged. - util/run_unittest_leg.py does not name the exclusion; no change there. --- .github/workflows/unit-tests.yml | 26 +++++++++----------------- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/.github/workflows/unit-tests.yml b/.github/workflows/unit-tests.yml index 7d89f24bf..599aa81cc 100644 --- a/.github/workflows/unit-tests.yml +++ b/.github/workflows/unit-tests.yml @@ -790,7 +790,14 @@ jobs: # 70s, protocols 847s, toolkit 97s, utilities 42s, one leg per matrix cell -- # see `timeout-minutes` below for why the job-level budget is no longer 20. # - # Three directories are deliberately absent from ``leg``, for reasons that + # tests/vendor was first left out of ``leg`` because running it this way + # surfaced an instance of the very defect this job exists to catch (issue + # #985): a stale VendorRuntimeWarning generation made assertWarnsRegex fail + # in tests/vendor/test_vendor_snapshot_restore_unit.py. That was fixed by + # re-resolving the class fresh per generation, so the directory is now a + # leg (vendor 83s, 118 directory tests, measured on this venv). + # + # Two directories are deliberately absent from ``leg``, for reasons that # are not this job's to fix: # # * tests/corekit -- already known and already accepted, not a new @@ -804,22 +811,6 @@ jobs: # against tests/corekit would be permanently red for a characteristic # the suite's own documentation already treats as expected, not a # regression this job would be reporting. - # * tests/vendor -- while sizing this job, running it this way surfaced a - # previously-unknown instance of the *same* class of defect #981 is - # about, in tests/vendor/test_vendor_snapshot_restore_unit.py: several - # earlier-sorting files in that directory purge pcapkit the same - # asymmetric way, and - # test_a_failure_leaves_the_previous_file_byte_for_byte_intact's own - # assertWarnsRegex(VendorRuntimeWarning, ...) ends up checking a stale - # generation of that class against a warning raised under a fresher - # one. Genuine, but a different file than this change touches and - # outside what #981 asks this particular job to fix -- left out so - # this new job lands green against a known and fixed cause, with the - # finding tracked as #985 rather than silently hidden or fixed in - # passing here. Until #985 lands, excluding the directory also means - # the other 13 tests/vendor modules get no unittest-ordering coverage - # at all, not just the one file #985 is about -- that cost is accepted - # for now, not unnoticed. # * tests/integration -- needs generated captures this script does not # build, and is already run whole, under pytest, by the `integration` # job above. The exclusion lives here, in this matrix, and nowhere else: @@ -862,6 +853,7 @@ jobs: - protocols - toolkit - utilities + - vendor steps: - uses: actions/checkout@v7 From b31f16b38368be6ea8f20c63ac978e8563e9c977 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Fri, 2 Oct 2026 11:17:34 -0400 Subject: [PATCH 2/2] docs(ci): scope the vendor figures to the leg in unittest-ordering's comment (#981) - The `vendor 83s, 118 directory tests` parenthetical measured tests/vendor alone, but the same comment block's wall-time table uses the identical ` Ns` shape for whole-leg times, so a reader sizing `timeout-minutes` read 83s as vendor's leg cost and was 25% low -- in the one comment that exists to size that cap. The leg is 104s / 297 tests; 118 of those are tests/vendor's own, the other 179 the five root-level modules every leg carries. - The wall-time table enumerated nine legs while saying "one leg per matrix cell", and the matrix carries ten: `vendor 104s` was never added. It is now, in the list's existing alphabetical order. - "re-resolving the class fresh per generation" is now "per test", which is what yields the current generation. Comments only: `yaml.safe_load` of the file parses to a document identical to the previous revision's (sha256 5989269..., eight jobs each structurally unchanged, `leg` still ten entries, `timeout-minutes` still 45). --- .github/workflows/unit-tests.yml | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/.github/workflows/unit-tests.yml b/.github/workflows/unit-tests.yml index 599aa81cc..54aeec5c9 100644 --- a/.github/workflows/unit-tests.yml +++ b/.github/workflows/unit-tests.yml @@ -787,15 +787,16 @@ jobs: # largest of the three some 90x short of 29 GB; no figure was taken for # tests/protocols, the largest leg. Measured wall times (this venv, serial): # cli 18s, const 268s, dumpkit 39s, foundation 272s, interface 38s, project - # 70s, protocols 847s, toolkit 97s, utilities 42s, one leg per matrix cell -- - # see `timeout-minutes` below for why the job-level budget is no longer 20. + # 70s, protocols 847s, toolkit 97s, utilities 42s, vendor 104s, one leg per + # matrix cell -- see `timeout-minutes` below for why the job-level budget is + # no longer 20. # # tests/vendor was first left out of ``leg`` because running it this way # surfaced an instance of the very defect this job exists to catch (issue # #985): a stale VendorRuntimeWarning generation made assertWarnsRegex fail # in tests/vendor/test_vendor_snapshot_restore_unit.py. That was fixed by - # re-resolving the class fresh per generation, so the directory is now a - # leg (vendor 83s, 118 directory tests, measured on this venv). + # re-resolving the class fresh per test, so the directory is now a leg (297 + # tests in the leg, 118 of them tests/vendor's own, measured on this venv). # # Two directories are deliberately absent from ``leg``, for reasons that # are not this job's to fix: