Repository navigation
tests: seven more skipUnless gates have never executed on any CI path (106 methods) #738
Description
Activity
Blocked on two things, recorded so nobody dispatches into a conflict:
- The fix needs
.github/workflows/unit-tests.yml, which open PR ci(unit-tests): install DPKT so the dpkt-gated tests actually run #737 owns. ci(unit-tests): install DPKT so the dpkt-gated tests actually run #737 is ✅ and awaiting merge; editing the same install lines now would conflict. This clears the moment it merges — exactly how tests: dpkt-gated tests have never run on any CI path (28 methods, 6 files) #729 waited on ci: parallelise pytest with xdist and drop 3.15 from the blocking matrix #725. - 55 of the 106 methods (
HAS_VENDOR_DEPS41,HAS_CRAWLER_DEPS14) need your ruling, not a fix. They exercise the vendor crawlers over the network, and Vendor crawlers: three take a Wikipedia 403 on the default User-Agent, three point at a dead IETF URL #518 records four Wikipedia 403s plus a dead IETF URL — so installingrequestswould make them run and possibly fail on upstream availability rather than on our code. Deliberately dark may be the right answer there.
The 48-method subset needs no ruling and is ready to dispatch once #737 lands:
HAS_CRYPTO14,HAS_EMOJI14,HAS_PYCRATE10,HAS_PYPCAPFILE10 — all pure-Python or wheel-shipping, no network.tests/integration/test_cli_subprocess.pyis already selected wholesale by theintegrationjob, soHAS_EMOJIis the identical one-line edit #737 made.- The fix needs
What's needed exactly from me?
Fair — my earlier comment buried the ask. One question:
Should the 55 vendor-crawler test methods run in CI at all?
HAS_VENDOR_DEPS(41) andHAS_CRAWLER_DEPS(14) gate on thevendorextra, i.e.requests. Adding it to the install lines is a one-line change and they would start running. The reason I did not just do it:- They fetch from IANA and Wikipedia over the network. Vendor crawlers: three take a Wikipedia 403 on the default User-Agent, three point at a dead IETF URL #518 records four Wikipedia 403s and a dead IETF URL, so these tests would go red on upstream availability rather than on our code — turning a required check into a flake.
- The three options: (a) leave them dark, (b) run them but off the blocking matrix, like the old 3.15 leg, (c) run them blocking and accept the flake.
I would take (b) if you want the coverage and (a) if you do not; (c) seems wrong given #518. Your call.
Nothing else here needs you. The other 48 methods —
HAS_CRYPTO14,HAS_EMOJI14,HAS_PYCRATE10,HAS_PYPCAPFILE10 — are pure-Python or wheel-shipping with no network, and a PR for them is already in flight.HAS_PYPCAP(4) andHAS_PCAP_CT(1) I left out as C extensions needing libpcap headers; say so if you want those too and I will treat the header install as its own decision.I agree on (b).
Ruling recorded: (b) — the vendor-crawler tests run, but off the PR-blocking matrix.
So the remaining scope on this issue is a non-blocking leg carrying the
vendorextra, coveringHAS_VENDOR_DEPS(41) andHAS_CRAWLER_DEPS(14) = 55 methods. The precedent to follow is how 3.15 was handled inb7f51401b: the job exists and reports, but is absent from ruleset23497679's 15 required checks, so a red there informs without gating.Two things whoever implements it must get right, both learned the hard way here:
- The new job's name must not collide with a required context. The 15 required are
Python 3.10–3.14,Integration Python 3.10–3.14,Compat Python 3.10–3.14. Reusing one of those names silently makes the new leg gating — the opposite of this ruling. - Skips must be provably resolved, not merely absent. CI runs
pytest -qwith no-r, so skip reasons never print and "0 occurrences ofrequests not installed" reads identically whether or not the fix worked. The skip delta is the only usable signal — ci(unit-tests): install DPKT so the dpkt-gated tests actually run #737 established this.
Expect real upstream flake: #518 records four Wikipedia 403s and a dead IETF URL. That is the cost being accepted deliberately, and it is the reason the leg is non-blocking.
needs: decisionremoved — nothing here waits on the maintainer now. The 48-method subset is already in flight as #740 (Part of #738), which leaves this issue tracking the 55, and separatelyHAS_PYPCAP(4) andHAS_PCAP_CT(1) which remain excluded as C extensions needing libpcap headers — say the word if those should be revisited.- The new job's name must not collide with a required context. The 15 required are
- added a commit that references this issue
on Sep 24, 2026 Correction to this issue's remaining scope, found while writing #745's guard: it is 41 methods, not 55.
HAS_CRAWLER_DEPS(14) asks only forrequestsandbs4, and #507 put both in thetestextra — so those 14 have been running on all three pytest jobs ever since, and need no leg. The two source files say so themselves (test_crawler_reachability_unit.py:91,test_ipx_socket_unit.py:48).HAS_VENDOR_DEPS(41) is still dark, but for a narrower reason than "needsrequests": it additionally wantshtml5lib, which onlybeautifulsoup4[html5lib]provides — i.e. thevendororallextra.requestsandbs4are already installed. Note the one exception the guard also turned up:tests/vendor/test_vendor_dest_path_unit.py(4 methods, added by #741) gates on aVENDOR_DEPStuple withouthtml5lib, so those 4 already run.Both facts are now asserted rather than remembered —
DependencyGateCoverageTests.test_the_crawler_dependencies_are_satisfied_by_the_test_extrafails ifHAS_CRAWLER_DEPSever goes dark again, and theHAS_VENDOR_DEPSallowlist entry fails if its missing-package set changes.Independently confirmed, with the scopes named so the two figures do not read as contradicting.
My AST sweep over
tests/on5c0df9248(class-level and method-levelskipUnless):gate methods dark? HAS_VENDOR_DEPS45 total — test_request_prompt_unit.py16,test_user_agent_unit.py11,test_ipx_packet_unit.py9,test_ftp_return_code_unit.py5,test_vendor_dest_path_unit.py441 dark, the 4 in test_vendor_dest_path_unit.pyalready runHAS_CRAWLER_DEPS14 — test_ipx_socket_unit.py9,test_crawler_reachability_unit.py5none; both deps are in the testextraSo 45 gated, 41 dark — the comment's 41 is the remaining-scope figure and mine is the total. Both
right at their own scope, and it flagged the 4-method exception itself.The
html5libmechanism holds.VENDOR_DEPS = ('requests', 'bs4', 'html5lib')in four files, and
html5libships only viabeautifulsoup4[html5lib]—pyproject.toml:203(vendor) and:226
(all).:256says it outright: thetestextra is "Plain, without thevendorextra's[socks]
and[html5lib]". So one package, not three, is what stands between these 41 and a green leg.That narrows the ruling you owe this issue. The earlier framing was "55 methods, most of them
hitting the network, needing a call given #518's Wikipedia 403s". It is now: 41 methods, one missing
package. Whether they belong on a non-blocking leg is still yours — the network reachability concern
is unchanged and real — but the cost of the (b) leg is smaller than this issue has been recording.One caveat on the guard's own assertion: it pins that
HAS_CRAWLER_DEPSstays satisfied by thetest
extra, which is the right thing to pin. It does not evaluate environment markers, so it cannot see
a gate that is dark only on some Python versions — the same limitation noted forHAS_PYPCAPFILEin #751.3 remaining items
- added 3 commits that reference this issue
on Sep 25, 2026 Unblocked. #755 merged, so
.github/workflows/unit-tests.ymlandtests/test_tier_guard.pyare free. Dispatching a worker now.Worth carrying across: #755 landed
ambiguous_satisfactions()andcontested_imports(), which derive the correct providing distribution from a flag's negated probe rather than from a hand-maintained table. So the guard can now tellPyPCAPfromPCAP_CT, and a gate satisfied through the wrong half of an ambiguous import name is reported rather than passed through. Any fix here should use that machinery rather than adding a parallel exclusion list.- added 5 commits that reference this issue
on Sep 25, 2026
#729 fixed
HAS_DPKT. Seven more gates are in the identical never-executes state — the dependency lives in an extra no CI job installs, so the tests report as skips, which read as passes.CI installs
.[test]on thetestlegs and.[test,Scapy]onintegration/gate. Thetestextra ispytest,pytest-xdist,typing-extensions— nothing else. Counts are my own AST pass overtests/, resolving class-level and method-levelskipUnlessgates, againstorigin/mainat55513f69e.HAS_VENDOR_DEPSvendor(requests)tests/vendor/*HAS_CRYPTOcrypto(cryptography)protocols/internet/test_esp_unit.pyHAS_EMOJIcli(emoji)integration/test_cli_subprocess.pyHAS_CRAWLER_DEPSvendortests/vendor/*HAS_PYCRATENGAP(pycrate)protocols/application/test_ngap_unit.pyHAS_PYPCAPFILEPyPCAPFiletoolkit/test_pypcapfile_unit.pyHAS_PYSHARKPySharktsharkis separate)foundation/engines/test_pyshark_engine.py106 methods, against #729's 28.
Two nuances that make this a judgement call rather than a pure defect:
HAS_VENDOR_DEPSandHAS_CRAWLER_DEPS(55 of the 106) exercise the vendor crawlers, which reach the network. Those may be deliberately dark in CI — see Vendor crawlers: three take a Wikipedia 403 on the default User-Agent, three point at a dead IETF URL #518, where four Wikipedia 403s and a dead IETF URL make them flaky by nature. Installingrequestswould make them run and possibly fail on upstream availability rather than on our code. This wants your ruling, not a blanket fix.HAS_SCAPY(14) is not in this list —Scapyis installed on theintegrationandgatelegs, so those run. They are dark only on thetestleg.HAS_PYPCAP(4) andHAS_PCAP_CT(1) are excluded as defensibly dark: C extensions needing libpcap headers.The cheap, uncontroversial subset is
HAS_CRYPTO+HAS_EMOJI+HAS_PYCRATE+HAS_PYPCAPFILE= 48 methods, all pure-Python or wheel-shipping, andtest_cli_subprocess.pyis already selected wholesale by theintegrationjob — so that one is literally the same one-line install-list edit #737 made.test_esp_unit.pyis real ESP crypto correctness, which is the highest-value of the set.Why the whole class survived:
Pipfile [dev-packages]carriesdpkt,cryptographyandemoji, so localmake testhas always run these. Only CI was ever blind.