Skip to content

fix(tests): keep a faked version and hand-bound stand-ins inside the test that made them (#687, #688) - #693

Merged
JarryShaw merged 1 commit into
mainfrom
fix/test-state-containment-687-688
Sep 23, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/test-state-containment-687-688

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner
  • Searched for similar pull requests
  • Followed the coding style (make pylint, make mypy, make isort)
  • make test passes, and a test case covers the change
  • Added a changelog entry under docs/source/changelog/ and regenerated CHANGELOG.md, if the change is user-visible

What is the purpose of your pull request?

  • fix — corrects a defect
  • feat — adds a feature
  • perf — changes performance, not behaviour
  • refactor — changes neither behaviour nor performance
  • test — tests only
  • docs — documentation only
  • ci — workflows or build tooling
  • chore — anything else

The subject line reads fix(tests): rather than test:, matching #662 and #686
the two immediately comparable changes — while the label is test alone, as on
both of those. Zero pcapkit/ lines change here, so fix would put a
test-infrastructure change into the release notes as a defect in the shipped
package.


Description of your pull request and other information

Fixes #687 and Fixes #688. Both were found while auditing #674 and both were
deliberately left out of #686; they are the two remaining ways a test in this suite
leaves the process worse than it found it.

Why one pull request and not two

They are independent mechanisms in different files, which argues for two. Against
that, decisively: tests/project/test_module_isolation.py has to carry a case for
each of them, so two pull requests would both edit that file and the second would
rebase over the first — and, worse, the new UnmaskedOrderTests case for #687 only
passes once tests/utilities/test_compat.py is fixed, and the case for #688 only
once tests/cli/test_main.py is. Split, each pull request lands a test that fails
until its sibling merges. They also share three docstring corrections, each of which
names both files.

Both defects reproduce on f0999858e, on a route tests/conftest.py cannot mask

I re-ran the measurements rather than taking the issues' word for them (both say the
numbers were not independently re-run). python 3.14.7, worktree at sys.path[0],
pcapkit.__file__ asserted inside the worktree, exit codes read from .rc files.

route before (f0999858e) after
python -m unittest tests.utilities.test_compat tests.project.test_public_api rc=1, Ran 14, FAILED (errors=10) rc=0, Ran 15, OK
pytest -p no:randomly --noconftest -q tests/utilities/test_compat.py tests/project/test_public_api.py rc=1, 10 failed, 4 passed rc=0, 15 passed, 432 subtests passed
python -m unittest tests.cli.test_main tests.project.test_public_api rc=1, Ran 16, FAILED (errors=10) rc=0, Ran 16, OK
pytest -p no:randomly --noconftest -q tests/cli/test_main.py tests/project/test_public_api.py rc=1, 10 failed, 6 passed rc=0, 16 passed, 432 subtests passed

All ten of #687's errors were the one the issue names, verbatim:
AttributeError: 'TransportProtocol' object has no attribute '__set_name__', raised
at aenum/_enum.py:1640 from pcapkit/const/reg/apptype.py:26. #688's ten split
nine/one: ImportError: cannot import name 'show_flag_values' from 'pcapkit.utilities.compat' (unknown location) and the same for SeekError from
pcapkit.utilities.exceptions. Each file alone passes before and after, which is
what makes them ordering defects rather than broken tests.

#687: which of the three suggested directions, and why not the other two

Chosen: direction 1, warm the caches before the fake — but with the ordering
asserted rather than assumed
, which is the stated weakness of that direction
turned into a checked precondition. load_compat_as_python35 imports
WARM_BEFORE_FAKING under the true version, snapshots frozenset(sys.modules),
installs the fake, and then asserts that the set of names added during the window is
empty. That generalises past aenum: it catches any module first-imported under
the lie, which matters because the blast radius is wider than one package. Measured
in a bare interpreter, the faked window imports 33 modules including typing,
typing_extensions, decimal, datetime, sqlite3, inspect, dis, ast,
annotationlib, threading and numbers alongside the five aenum modules. After
warming ('aenum', 'decimal', 'threading', 'typing', 'typing_extensions') the delta
is [].

Direction 3, reload aenum._common, is dead on measurement. aenum/_enum.py:2
is from ._common import *, so pyver is copied into every aenum module's
globals, and _enum.py:1640 branches on its own copy. Measured:
importlib.reload(aenum._common) gives _common.pyver == (3, 14) and leaves
_enum.pyver == (3, 5) — i.e. it repairs the reading nobody uses and leaves the one
that breaks.

Direction 2, a subprocess, is rejected on coverage and on diagnosability.
[tool.coverage.run] here is source = ["pcapkit"], branch = true, with no
parallel/concurrency setting and no COVERAGE_PROCESS_START .pth. coverage keys
on file path, so today's in-process load_module('…compat_py35', 'pcapkit/utilities/compat.py') is what credits the < 3.6, < 3.8, < 3.9 and
< 3.11 branches of pcapkit/utilities/compat.py; moving that to a child process
drops the measurement, which is coverage going backwards for a test-only change.
Separately, the existing test asserts on live objects — compat.Collection.__subclasshook__
against locally defined classes, cached_property.__set_name__ raising TypeError,
decimal.getcontext().prec inside the context manager — none of which crosses a
process boundary; they would become a child script reporting pass/fail as text, and a
failure would be captured stdout instead of a named assertion.

The aenum._common.pyver assertion

assert_aenum_is_not_poisoned is registered as a cleanup in setUp, before
anything else, so — cleanups being LIFO — it runs after the module-table restore
load_module arranges and after the test body however it ended. It sweeps every
aenum module holding a tuple pyver out of sys.modules rather than naming two,
because the copies can disagree; on 3.14.7 that is aenum, aenum._common,
aenum._constant, aenum._enum and aenum._tuple. test_faking_the_version_leaves_aenum_reading_the_real_one
additionally refuses to pass vacuously (it asserts the sweep is non-empty) and builds
a two-member aenum.IntFlag subclass, which is what
pcapkit/const/reg/apptype.py:26 does and what raises the AttributeError when the
cache is wrong — so the symptom is asserted, not a proxy for it, and without needing
import pcapkit in a unit-tier module.

#688: what the writes were routed through

tests._support.isolate_modules(self, ISOLATED) in setUp, replacing the
hand-rolled purge loop that used to sit at the top of _load_cli_module. The
stand-in construction is unchanged — that part was never the defect — so the diff is
"the purge loop becomes an isolation that also restores". ISOLATED is
ISOLATED_PREFIXES + ('emoji',), which fixes a second leak nobody had filed: the
faked emoji was outside the prefixes tests/conftest.py covers, so it was not
merely unrestored but unmasked, surviving a normal pytest run and leaving whatever
imported emoji next with a two-attribute SimpleNamespace — or, from one test, a
class whose emojize raises UnicodeEncodeError.

assert_module_table_restored compares snapshot_modules(ISOLATED) taken in setUp
against the region after the restore, in all three directions a restore can be wrong:
added, removed and rebound by identity. Absence is checked as well as presence, which
is the direction #674 actually was.

tests/_support.py is not touched. Nothing needed extending: isolate_modules
already takes a prefixes argument, and snapshot_modules and ISOLATED_PREFIXES
are already public.

Yes, the docstrings needed correcting — three of them, not one

  • tests/conftest.py said "The three known leaks are also fixed at their call sites,
    with isolate_modules". On f0999858e that was true of two. The third was
    tests/cli/test_main.py, which this pull request fixes, so the sentence becomes
    true; the paragraph now says which one it was and why a guard that repairs a leak
    also hides it.
  • tests/project/test_module_isolation.py is the worse one, and the one the issue's
    "irony worth checking" points at. It recorded tests/cli/test_main.py as
    "deliberately left as it was, as the standing witness that the guard covers a file
    which has not opted into anything" — a decision to leave the very defect tests/cli/test_main.py leaves hand-written sys.modules stand-ins behind: 10 errors when paired with test_public_api #688 is
    about. That trade is now reversed and the reasoning written down: the guard healed
    the leak on every run, which is exactly why it took an audit rather than a red test
    to notice. The witness role is deliberately left vacant rather than replaced by a
    test whose job is to leave sys.modules broken. test_a_polluter_outside_the_helpers_is_covered
    is renamed test_the_cli_polluter_order_passes, since it no longer covers a
    polluter outside the helpers.
  • tests/const/test_const_enum_lookup.py justified its own setUp purge with
    "depending on suite order, that stub can still be sitting there when this module
    runs", naming tests/cli/test_main.py. Now it cannot. The purge stays — the
    convention is worth keeping, and the next stand-in written without the helpers
    would be a live hazard — but the claim is corrected.

A test that fails without each fix, exit codes read from files

Each fix was disabled in isolation (leaving the new assertions in place) and the run
re-measured:

fix disabled route result
WARM_BEFORE_FAKING = () python -m unittest tests.utilities.test_compat rc=1, Ran 5, FAILED (failures=5)
WARM_BEFORE_FAKING = () python -m unittest tests.utilities.test_compat tests.project.test_public_api rc=1, FAILED (failures=5, errors=10)
isolate_modules → old purge loop pytest -q tests/cli/test_main.py rc=1, 6 failed
isolate_modules → old purge loop python -m unittest tests.cli.test_main tests.project.test_public_api rc=1, FAILED (failures=6, errors=10)

Note the asymmetry, and it is worth knowing rather than glossing. #688's assertion
fails under a plain pytest run
, because the cleanup is registered ahead of the
isolation and therefore runs before restore_module_table gets to heal anything.
#687's does not: pytest_sessionstart imports pcapkit, and so aenum, before
the first test, so there is no cold cache left to poison and both guards pass whether
or not the fix is in place — measured, 5 passed with the warm-up emptied. That is
why UnmaskedOrderTests exists: it runs the pairing under the stdlib unittest
runner in a subprocess, which is the only automated route on which a #687 regression
is visible at all.

Coverage does not move

coverage run -m pytest -p no:randomly -q over
tests/utilities/test_compat.py tests/cli/test_main.py tests/const/test_const_enum_lookup.py tests/project/test_public_api.py tests/project/test_module_isolation.py,
run against this branch and against f0999858e (the five changed files swapped out of
git show HEAD:…, then swapped back and git diff --stat re-checked):

diff before.report.txt after.report.txt   →  exit 0   (byte-identical)
TOTAL   40315 statements   16780 missed   9434 branches   266 partial   48%

What moved is the test count: 37 passed → 40 passed, 566 subtests unchanged. The
three are one new test in test_compat.py and two in test_module_isolation.py.

What the local evidence proves, and what it does not

CI cannot see either defect, and could not before this change either — the
unit-tests.yml jobs run plain pytest, which is exactly the selection that masks
both. So no claim is made here about CI being green on the defect: what CI will now
exercise is UnmaskedOrderTests, which spawns the unmasked runs itself, and #688's
in-file assertion. The measurements above are local, on CPython 3.14.7, and are the
only evidence offered.

On the first two checkboxes, so they are not taken for more than they are worth:
make pylint, make mypy and make isort all take pcapkit as their only target,
and this change touches no pcapkit/ file, so the coding-style box is ticked
vacuously — the import ordering here follows the convention the neighbouring test
modules already use.

make test runs the suite bar the integration tier, which needs tens of gigabytes of
resident memory on this machine, and a full pass would in any case prove nothing
here — that selection is precisely what hides both defects. The test checkbox is
ticked for the narrower selections actually run: tests/project, tests/cli,
tests/utilities
(231 passed, 571 subtests), tests/const (40 passed, 798 subtests),
tests/test_support_helpers.py + tests/interface + tests/corekit/test_protochain.py
(51 passed, 17 subtests), plus the eight pairing runs tabulated above.

No changelog entry

Following #686's reasoning, and checked rather than assumed: every entry in
docs/source/changelog/1.5.0.rst corresponds to a pcapkit/ change, and neither
#660/#662 nor #674/#686 appears there. Zero pcapkit/ lines change here, so the
shipped package is byte-identical and there is nothing user-visible to record.

…test that made them (#687, #688)

Two test-isolation defects found while auditing #674, neither of which a normal
`pytest` run can see: `tests/conftest.py` repairs one of them after every test and
warms the cache the other one needs cold.

* `tests/utilities/test_compat.py` faked `sys.version_info` around a real
  `import`, so the `< 3.11` branch of `pcapkit/utilities/compat.py` performed the
  first `import aenum` of the process under the lie. `aenum/_common.py` memoises
  `pyver = sys.version_info[:2]` at import time, and every other aenum module
  takes a *copy* through `from ._common import *`, so `aenum/_enum.py:1640` then
  called `__set_name__` on every member of every later enumeration. Restoring
  `sys.version_info` undoes none of that -- the value is already memoised. It now
  imports what the faked branches need *before* installing the fake, and asserts
  that nothing at all was first-imported under it, so the ordering is checked
  rather than relied on. Reloading `aenum._common` was rejected as a fix because
  it repairs `_common.pyver` and leaves `_enum.pyver`, the copy that decides the
  branch; a subprocess was rejected because coverage keys on file path, so the
  in-process load is what credits compat.py's `< 3.6` lines today and this repo
  wires up no subprocess coverage.
* `tests/cli/test_main.py` wrote nine names into `sys.modules` -- seven stand-ins,
  a bare parent package and `pcapkit.__main__` -- and restored none, so the next
  test to import the real library got a `pcapkit.utilities.compat` carrying one
  name. It now goes through `tests._support.isolate_modules`, widened to cover
  `'emoji'`: the faked `emoji` was outside `ISOLATED_PREFIXES`, so it was not even
  masked and survived a normal pytest run.
* Both files now assert the invariant they used to break, in a cleanup registered
  ahead of the restore so it runs after it: aenum's `pyver` readings across every
  aenum module, and the `sys.modules` region unchanged in all three directions --
  added, removed and rebound -- so absence is checked as well as presence.
* `tests/project/test_module_isolation.py` gains `UnmaskedOrderTests`, which runs
  both pairings under the stdlib `unittest` runner. For #688 that is the
  cross-file half of a claim the file itself already makes; for #687 it is the
  only place a regression is caught at all, since `pytest_sessionstart`'s eager
  `import pcapkit` warms aenum and makes the file's own guards pass either way.
* Three docstrings corrected, all of which had become false. `tests/conftest.py`
  claimed all three known #660 leaks were fixed at their call sites;
  `tests/project/test_module_isolation.py` recorded `tests/cli/test_main.py` as
  "deliberately left as it was", the standing witness for the conftest guard --
  which is precisely why the leak survived unnoticed; and
  `tests/const/test_const_enum_lookup.py` named that file as a live hazard.

Measured on CPython 3.14.7. `python -m unittest tests.utilities.test_compat
tests.project.test_public_api` goes from `FAILED (errors=10)` to `OK`, and
`tests.cli.test_main tests.project.test_public_api` likewise; both pairings also
go from `10 failed` to all passing under `pytest --noconftest`. With the warm-up
list emptied the compat file reports `FAILED (failures=5)` under `unittest`, and
with the isolation replaced by the old purge loop the CLI file reports `6 failed`
under plain `pytest`. No `pcapkit/` line changed, and `coverage report` over the
touched selection is byte-identical before and after (`diff` exit 0); the same
selection goes from 37 to 40 tests with 566 subtests unchanged.

Fixes #687
Fixes #688
@JarryShaw JarryShaw added the test Pull requests that add or correct tests (test: subject prefix) label Sep 23, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Cross-interpreter measurement, and a correction to #687's premise

The description above measured only CPython 3.14.7. Repeated on 3.10.21, 3.11.15 and
3.12.13 (mise builds, throwaway venvs carrying nothing but aenum and
typing-extensions, tests._support.__file__ asserted inside this worktree on each,
exit codes read from .rc files):

interpreter unittest tests.utilities.test_compat unittest tests.cli.test_main aenum warm after the real branch? delta under the fake, warm-up off pyver after, warm-up off delta, warm-up on aenum.IntFlag builds addCleanup LIFO
3.10.21 rc=0, 5 OK rc=0, 6 OK yes aenum's own body crashes first [] yes yes
3.11.15 rc=0, 5 OK rc=0, 6 OK no 15 names (3, 5) [] yes yes
3.12.13 rc=0, 5 OK rc=0, 6 OK no 15 names, byte-identical to 3.11.15 (3, 5) [] yes yes
3.14.7 rc=0, 5 OK rc=0, 6 OK no 33 names (3, 5) [] yes yes

Three things worth recording.

WARM_BEFORE_FAKING is complete on all four interpreters — the delta under the fake
is empty on each, which is the invariant the new assertion enforces. 3.11.15 and
3.12.13 pull in exactly the same 15 names with the warm-up off; 3.14.7 pulls in 33
(it adds annotationlib, ast, dis, inspect, opcode, token, tokenize,
textwrap, weakref and friends). The extras in every case are transitive imports of
aenum and decimaldecimal registering with the numbers ABC, aenum reaching
datetime and sqlite3 — so warming the five names warms them too, which the empty
delta proves directly rather than by argument.

addCleanup is LIFO on all four, which the whole design rests on: the checking
cleanup is registered before the restore it checks, so it must run after it.

#687's premise needs a correction, and it makes the fix more load-bearing, not less.
The issue says the poisoning happens "on Python ≥ 3.11", because below that
compat.py's real branch is itself from aenum import StrEnum and so warms the cache
honestly. That is true of this test filesetUp executes the real compat.py
before every test, so on 3.10 aenum is always already warm when the fake goes in, and
the file passed on 3.10 before this change. It is not true of the underlying
hazard. Faking the version around aenum's first-ever import in a fresh 3.10.21 process
does not silently poison a cache, it takes the process down: aenum's own module body
raises ImportError: cannot import name 'FlagBoundary' from 'enum' and then
AttributeError: 'FlagBoundary' object has no attribute '__set_name__' from its own
fallback definition of that name.

So 'aenum' in WARM_BEFORE_FAKING is load-bearing on 3.10 as well, not merely
defensive for 3.11 and later — the only thing protecting 3.10 today is that setUp
happens to import the real module first, which is exactly the kind of incidental
ordering the empty-delta assertion exists to stop depending on. I will fold this into
WARM_BEFORE_FAKING's own comment so the next reader does not have to rediscover it,
and I will note it on #687.

One non-blocking observation found on the way, about #686's code rather than this
change: tests._support.load_module's test= parameter is honoured by load_module
itself but not by the ensure_package call beneath it, which re-derives the running
test through _running_test_case(). That is arguably correct — _require_arranged_restore
is asking about the running test, which is what the nested case in
tests/test_support_helpers.py needs — but the parameter's docstring reads as though
passing it were sufficient, and a synthetic stand-in passed as test= gets a
RuntimeError it cannot satisfy. Nothing in the suite hits it; worth a docstring note
if anyone is in there next.

@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO MERGE

Independent, read-only cross-review performed on a different model from the one that authored this change (per this repo's cross-review convention: disagreement is the value, not confirmation). No files edited, no mutating git, nothing pushed, no formal review state — this is a comment only.

Verified from a temporary local clone (/local/home/jarryx/GitHub/PyPCAPKit, PR head 39552a51e vs base f0999858e), confirming pcapkit.__file__ resolved to the clone before trusting any result.

  1. aenum poisoning is contained, including a mid-test raise — CONFIRMED. tests/utilities/test_compat.py:150 (self.addCleanup(self.assert_aenum_is_not_poisoned)) is the first line of setUp, so it fires per unittest's documented contract even if setUp itself later raises. I injected a throwaway test that fakes the version, imports aenum cold (skipping the warm-up), then raises RuntimeError before any explicit check — the run reported FAILED (failures=1, errors=1): the RuntimeError as an error and assert_aenum_is_not_poisoned independently failing with the poisoned pyver values. mock.patch.object(sys, 'version_info', ...) in load_compat_as_python35 also restores via __exit__ regardless of exception.

  2. test_main.py stand-ins are restored on every exit path — CONFIRMED. setUp (tests/cli/test_main.py:93) registers assert_module_table_restored before isolate_modules(self, ISOLATED) (:104) — deliberate LIFO ordering so the restore-cleanup runs first and the check runs second. isolate_modules (tests/_support.py:718, pre-existing infrastructure this PR did not touch, already hardened for twelve tests/project/ tests fail with TypeError: type 'ProtocolBase' is not subscriptable depending on what ran first: the fake-module helpers purge on entry and never restore on exit #660/tests/_support.py leaves bare stub modules in sys.modules, so tests/corekit + tests/project in one process fails three test_public_api tests #674) restores via a full snapshot diff registered through addCleanup, so it undoes whatever was bound regardless of where in _load_cli_module a failure occurs.

  3. New tests fail without the fix — CONFIRMED empirically, with an honest limitation the PR itself documents. I reproduced both defects directly: reverting test_main.py to its base ad-hoc purge loop and running python -m unittest tests.cli.test_main tests.project.test_public_api gave FAILED (errors=10) with the exact ImportError: cannot import name 'show_flag_values' from 'pcapkit.utilities.compat' (tests/cli/test_main.py leaves hand-written sys.modules stand-ins behind: 10 errors when paired with test_public_api #688). Emptying WARM_BEFORE_FAKING and running python -m unittest tests.utilities.test_compat tests.project.test_public_api gave FAILED (failures=5, errors=10) with the exact pyver-poisoning AttributeError symptom (tests/utilities/test_compat.py poisons aenum._common.pyver for the whole process, breaking a later import pcapkit #687). With the fix restored, both pairings pass. The documented limitation: under a normal pytest -q tests/utilities/test_compat.py, emptying WARM_BEFORE_FAKING still gives 5 passedconftest.py's pytest_sessionstart warms aenum first, so the in-file assertion alone is vacuous under pytest. What actually catches the regression under a normal pytest tests/ run is UnmaskedOrderTests (tests/project/test_module_isolation.py:256, methods at :282 and :286), which shells out to a bare python -m unittest subprocess specifically to bypass conftest. I confirmed that class passes with the fix in place. Whether CI's collection actually includes it was the one thing I flagged as unverified in my own review — the coordinator has since checked and confirmed it, and I'm relaying that as resolved rather than as an open item: .github/workflows/unit-tests.yml runs a bare python -m pytest -q at lines 86, 156 and 239 — no -m, no --deselect, no --ignore — and pyproject.toml:279-285 sets only testpaths = ["tests"] / python_files = ["test_*.py"] with no addopts and no markers configured anywhere. test_module_isolation.py matches that pattern under that path, so UnmaskedOrderTests is collected and runs in CI as-is.

  4. conftest.py change does not weaken the guard — CONFIRMED. The entire diff to tests/conftest.py is docstring prose; zero executable lines changed in restore_module_table. It documents that tests/cli/test_main.py leaves hand-written sys.modules stand-ins behind: 10 errors when paired with test_public_api #688 was the previously-unfixed third leak this fixture was quietly healing, now also fixed at its call site — it does not touch the fixture's behavior.

  5. Test-only, no library files touched — CONFIRMED. git diff --name-only f0999858e...39552a51e -- pcapkit/ is empty. Also checked: zero changelog files (CHANGELOG.md or changelog .rst) touched anywhere in the diff.

  6. tests/utilities/test_compat.py poisons aenum._common.pyver for the whole process, breaking a later import pcapkit #687 and tests/cli/test_main.py leaves hand-written sys.modules stand-ins behind: 10 errors when paired with test_public_api #688 are genuinely independent, soundly bundled — CONFIRMED. The tests/utilities/test_compat.py poisons aenum._common.pyver for the whole process, breaking a later import pcapkit #687 fix lives entirely in test_compat.py (warm-up list, aenum_pyver_readings, cleanup, load_compat_as_python35); the tests/cli/test_main.py leaves hand-written sys.modules stand-ins behind: 10 errors when paired with test_public_api #688 fix lives entirely in test_main.py (isolate_modules, assert_module_table_restored). test_module_isolation.py bundles both only at the shared-infrastructure level (run_unittest/child_environ/UNMASKED_ORDERS), with two independent test methods, each running its own subprocess pairing against its own distinct symptom string. Either fix could be reverted without touching the other's code or tests.

No unguarded failure paths, dropped state, or missing bookkeeping found elsewhere in the diff. No pcapkit/ files, no changelog edits, no from inspect/importlib import ...-style imports introduced.

GOOD TO MERGE

@JarryShaw
JarryShaw merged commit 9d7890d into main Sep 23, 2026
13 of 25 checks passed
@JarryShaw
JarryShaw deleted the fix/test-state-containment-687-688 branch September 23, 2026 03:35
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO — cross-review verdict (Sonnet), with one non-blocking finding now fixed in #698

This pull request was raised by an agent, so it got an independent cross-review on a
different model, briefed to falsify rather than to confirm: a verdict per load-bearing
claim, each with evidence the reviewer obtained itself. It ran read-only and reported
back rather than editing anything. Its verdict landed after this merged, so it is
recorded here and its one finding is fixed in #698.

Verdict: GOOD TO GO, with one substantive non-blocking finding.

Confirmed independently, each with its own derivation rather than by reading the
description:

  • Both defects reproduce on f0999858e and not on 39552a51e, on all four routes,
    with the counts matching this description exactly. The reviewer produced the "before"
    half without editing the tree, by exec-ing git show f0999858e:… content into
    throwaway sys.modules names in a scratch script: route 1 gave testsRun=14, errors=10 with the 'TransportProtocol' object has no attribute '__set_name__' text
    at aenum/_enum.py:1640, route 3 gave testsRun=16, errors=10 with the 9/1
    show_flag_values/SeekError split.
  • aenum._enum.pyver really is a separate copy. It found that the two names are
    initially the same tuple object, and that importlib.reload(aenum._common) rebinds
    only _common's name — leaving _enum.pyver == (3, 5) — after which a two-member
    IntFlag raised AttributeError: 'Demo' object has no attribute '__set_name__'.
    Direction 3 is dead.
  • The 33-name figure for the un-warmed window reproduces exactly, including every
    module named.
  • Coverage cannot move: source = ["pcapkit"], branch = true, no
    parallel/concurrency/dynamic contexts, so hit-once semantics keyed on pcapkit/
    paths; the new test re-executes branches already covered; the two new subprocess runs
    propagate no COVERAGE_PROCESS_START and there is no auto-start .pth. Its own
    coverage run on this branch gave TOTAL 40315 16780 9434 266 48% — matching.
  • addCleanup is LIFO on 3.14.7, and cleanups registered before a raise in setUp
    still run.
  • The tests/utilities/test_compat.py poisons aenum._common.pyver for the whole process, breaking a later import pcapkit #687-vacuous / tests/cli/test_main.py leaves hand-written sys.modules stand-ins behind: 10 errors when paired with test_public_api #688-not asymmetry is real, verified from source rather than
    taken on trust: pcapkit/const/reg/apptype.py:16 imports aenum unconditionally and
    tests/conftest.py:87 imports pcapkit in pytest_sessionstart. It then
    demonstrated that UnmaskedOrderTests's assertion would fail without the fix, by
    running this file's own source in-memory with WARM_BEFORE_FAKING overridden to
    (): FAILED (failures=5) alone and FAILED (failures=5, errors=10) paired, with
    the symptom string present ten times.
  • No fourth stale docstring: it grepped for test_main, the issue numbers, "purge
    loop", "rolls its own", "deliberately left", "standing witness". The only other hits
    are tests/integration/test_cli_subprocess.py:4, which describes what this file
    tests and makes no claim about its isolation, and an unrelated list in
    tests/test_docstring_contract.py.
  • No dangling reference to the renamed test_a_polluter_outside_the_helpers_is_covered
    anywhere, .github/ included. tests/_support.py untouched, no pcapkit/ file
    changed.
  • run_unittest's child does import this tree's pcapkit, not the editable install:
    sys.meta_path[4] is the editable finder and PathFinder at index 3 answers from
    sys.path first, with pcapkit.__file__ resolving into the worktree under
    child_environ()'s exact combination.

The finding, and it is a real one. WARM_BEFORE_FAKING held the modules observed
in the window, not the ones compat.py imports there. pathlib is imported
unconditionally at line 63 and was missing, as were collections.abc, contextlib and
enum. With pathlib forced cold the window leaks ['pathlib', 'pathlib._os']. It
cannot fire today — site and tests/_support.py import all four first — which is
exactly why it is worth fixing: "another file imports it for us" is the unenforced
ordering this change claimed to have removed. Reproduced independently before acting on
it, and fixed in #698, where the list is read off compat.py's guards instead.

Two things the reviewer could not verify, stated rather than glossed: a byte-for-byte
"before" coverage report, since producing one needs the tree edited, so it attacked the
reasoning instead; and the supplementary tests/project / tests/const figures quoted
above, which it deprioritised in favour of the load-bearing claims. The pathlib gap
aside, it raised nothing else, blocking or stylistic.

One last honest note, unchanged from the description: CI cannot see either defect,
and could not before this change either — the unit-tests.yml jobs run plain pytest,
which is the selection that masks both. What CI now exercises is UnmaskedOrderTests,
which spawns the unmasked runs itself.

JarryShaw added a commit that referenced this pull request Sep 23, 2026
…se's imports

`WARM_BEFORE_FAKING` in `tests/utilities/test_compat.py` held the five modules that
were *observed* to be imported inside the faked-version window, not the ones
`pcapkit/utilities/compat.py` actually imports there. Line 63 of that file is an
unconditional `import pathlib` on every branch reachable at `(3, 5)`, and `pathlib`
was not on the list; nor were `collections.abc` (line 34), `contextlib` (153) and
`enum` (179).

The assertion did not fire because another module imports all four first:
`collections.abc`, `contextlib` and `pathlib` at module scope in
`tests/_support.py`, and `enum` in `test_compat.py` itself. It is not the runtime --
under `python -S` none of the four is in `sys.modules` at all -- so this really was
one file's correctness resting on another file's import list, which is precisely the
unenforced ordering #687's fix claimed to have removed. Measured: with `pathlib`
forced cold and the old five-name list, the window leaks
`['pathlib', 'pathlib._os']`; with all nine, it is empty even with five of the nine
forced cold.

The list is now read off `compat.py`'s guards rather than tuned until the assertion
stopped complaining, and its docstring says which four never surface today and why
they are there anyway. Warming an already-imported module is a `sys.modules` lookup,
so the four cost nothing.

Also records, in the same docstring, that warming `aenum` is load-bearing below 3.11
as well. #687 reads as a `>= 3.11` defect and that is true of this file -- below 3.11
`setUp`'s real load warms aenum honestly first -- but faking the version around
aenum's first import in a fresh CPython 3.10.21 process raises `ImportError: cannot
import name 'FlagBoundary' from 'enum'` outright rather than poisoning a cache.

`pytest -q tests/utilities/test_compat.py tests/cli/test_main.py
tests/project/test_module_isolation.py` gives `17 passed`, and `python -m unittest
tests.utilities.test_compat tests.project.test_public_api` gives `Ran 15 tests` /
`OK`. Measured on 3.10.21, 3.11.15, 3.12.13 and 3.14.7. No `pcapkit/` line changes,
and `coverage report` over the same five-file selection #693 used is byte-identical
against `f0999858e` (`diff` exit 0), at 40 tests and 566 subtests.

Follow-up to #693, found by cross-review after that pull request had merged.
Refs #687
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test Pull requests that add or correct tests (test: subject prefix)

Projects

None yet

1 participant