Skip to content

test_stacklevel asserts an exact warning count, so a leaked in.pcap handle fails it on unrelated PRs #606

Description

@JarryShaw

tests/utilities/test_stacklevel.py::StacklevelAttributionTests::test_a_truncated_traceback_limit_does_not_blind_the_walk asserts an exact warning count of 1 inside a catch_warnings(record=True) block with simplefilter('always'). Any unrelated warning that happens to be emitted in that window fails it. A leaked in.pcap file handle from an earlier test supplies exactly such a warning, so the test fails for reasons that have nothing to do with the code under review.

Observed

It has now failed two unrelated pull requests:

FAILED tests/utilities/test_stacklevel.py::StacklevelAttributionTests::
       test_a_truncated_traceback_limit_does_not_blind_the_walk
E  AssertionError: 2 != 1 : ["unclosed file <_io.BufferedReader
   name='…/examples/captures/in.pcap'>",
   'Probe: info class has been finalised; now skipping']

#596 changes pcapkit/const/**, pcapkit/vendor/**, ftp.py and httpv1.py — none of which this test exercises. The red mark is noise against that PR.

Two separate defects

1. The assertion is too strict for what it is testing. The test's subject is stacklevel attribution — that a truncated traceback limit does not blind the warning walk. It does not need the window to contain only its own warning; it needs its own warning to be present and attributed correctly. Asserting len(records) == 1 makes every unrelated ResourceWarning in the process a failure of this test.

2. A leaked in.pcap handle is the warning that trips it. Nineteen test files reference in.pcap, and most show no sign of closing what they open:

tests/_tiers.py                                        refs=2   close-ish=0
tests/foundation/engines/test_new_engine_parity_runtime.py  refs=3   close-ish=0
tests/foundation/engines/test_scapy_engine.py          refs=5   close-ish=0
tests/integration/test_cli_subprocess.py               refs=4   close-ish=0
tests/integration/test_engine_parity.py                refs=4   close-ish=0
tests/integration/test_engine_runtime.py               refs=6   close-ish=0
tests/integration/test_runtime_extract.py              refs=2   close-ish=0
tests/interface/test_core.py                           refs=4   close-ish=0
tests/interface/test_misc.py                           refs=12  close-ish=0
tests/protocols/internet/test_ip_runtime.py            refs=2   close-ish=0
tests/protocols/link/test_link_runtime.py              refs=1   close-ish=0
tests/test_tier_guard.py                               refs=10  close-ish=0

("close-ish" counts .close(), with open, and closing( — a crude proxy, so a zero is a candidate rather than a proof. I have not isolated which file leaks the specific handle.)

Because the warning is emitted when the handle is garbage-collected, which test trips is a function of GC timing and collection order, not of any one test's code. That is why it appears and disappears across runs and across unrelated PRs, and why #577 "cleared" after main moved without anything being fixed.

Why this is worth fixing rather than tolerating

A merge-blocking check that fails on unrelated changes trains everyone to ignore red marks. It has already produced one PR where the red had to be explained away in a comment, and a second where it will have to be again.

Fixing either defect alone breaks the coupling:

  • relax the assertion to "my warning is present and correctly attributed" rather than "mine is the only warning"; or
  • close the leaked handles, so the ResourceWarning is never emitted.

Both are worth doing, and the first is the one that makes the test robust against the next unrelated warning rather than only this one.

Notes

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testPull requests that add or correct tests (test: subject prefix)

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions