test(tiers): make ALL a curated opt-in tier (#1958) by JarryShaw · Pull Request #1978 · JarryShaw/PyPCAPKit

JarryShaw · 2026-10-20T14:01:50Z

Searched for similar pull requests
Followed the coding style (make pylint, make mypy, make isort)
make test passes, and a test case covers the change
Changelog entry: N/A — centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657

What is the purpose of your pull request?

test — tests only

Description of your pull request and other information

Stacked on #1976. Implements the ruling on #1963 that ALL should be a tailored list. A module joins ALL only through an explicit __tier__ = 'ALL' marker with a reason; everything else defaults to LATEST. The criterion is behaviour that can differ by interpreter or version (compat gates, GC/finalisation, int/struct limits, pickling, metaclass/ABC/enum, IO, threading, engines), plus the ruling-B round trips and a smoke slice per protocol family. The criterion is written in _tiers.py and workflows.rst. The guard also fails any module that compares sys.version_info/sys.implementation without stating a tier.

CPython 3.14, unit part of `ALL`	Before	After
Test ids	3787	2207
Runtime	852 s	474 s
`pcapkit/` line coverage	92.8%	85.3%

Latest CPython still runs everything. Judgement calls for review:

Scapy's engine and toolkit tests are ALL like the other engines; they skip on test legs that do not install Scapy.
Six vendor tests declare LATEST with a reason: the vendor extra is installed only there, and the interpreter-sensitive vendor parts stay ALL.
The interpreter guard is syntactic, best-effort, and says so. It flags sys.version_info, sys.implementation, sys.flags, sys.maxsize, sysconfig and probes of sys.
The modules fix(corekit): copy SeekableReader's octets and feed MultiDict a dict, and gate GraalPy's limits (#1963) #1974 gates for GraalPy are already marked ALL here; the two PRs merge cleanly.

tests/project/test_curated_all_tier_1958.py fails on #1976's head and passes here.

Refs #1958, #1963

- test, integration: a step on the latest leg alone installs the whole suite's extras (Scapy, PyShark, vendor, docs, cli on top of the base); every other leg keeps its install line and runs the ALL tier. - project-tests, gate: install the same set. - tests/_dependency_gates.py: read a latest-only install step as that leg's extras; drop the exclusions it closes (HAS_SCAPY, HAS_PYSHARK, HAS_RUNTIME, HAS_VENDOR_DEPS, HAS_SPHINX, HAS_DOCUTILS, HAS_FURO). - tests/project/test_whole_suite_on_latest_1958.py pins the lines, the leg, the gates closed and the docs; tier-guard, #1732 and #1954 pins follow the new reading. - workflows.rst describes it; unit-tests.yml line citations re-anchored. Selection steps and tests/_tiers.py are unchanged.

…1958) - test, integration: the install comments said cli, and crypto and NGAP, are not installed; the latest leg's whole-suite step installs them. - workflows.rst, releasing.rst: unit-tests.yml line citations re-anchored.

- tests/_tiers.py: module_tier() is the declared tier, else DEFAULT_TIER (LATEST); rule_tier() goes. all_tier_paths() lists ALL, and branches_on_interpreter() flags a module that compares the interpreter itself. The docstring states the curation criterion (#1963). - Curated modules opt in with `__tier__ = 'ALL'` and a reason: version gates, GC/weakref, int/struct limits, pickling and copying, metaclass and enum machinery, IO, threading and hooks, engines whose releases vary by interpreter, the hand-picked round trips, and a smoke slice per protocol family. Six vendor markers with no such reason are gone. - tests/test_tier_guard.py: ALL is opt-in, every ALL module declares it, and a module that branches on the interpreter must state its tier; the mixed-directory split test no longer assumes tests/integration is all ALL. test_sweeps_on_latest_only_1732 compares what the ALL fixture selection drops with the LATEST modules it reaches. - tests/project/test_curated_all_tier_1958.py pins the opt-in rule, the strict subset, the ruling-B round trips and the docs. - workflows.rst "Test Tiers" and the `test` job's `Run unit tests` comment describe it. Tier guard, #1732, #1954 and #1958 pins pass on CPython 3.14.

- add __tier__ = 'ALL': the module pins GraalPy-only semantics, so it has to keep running on every interpreter leg once ALL becomes opt-in (#1978) Refs #1963

JarryShaw · 2026-10-20T14:10:21Z

NEEDS CHANGES at b733cd042. A Sonnet review found interpreter-sensitive modules that fell to LATEST, against the PR's own criterion:

Pickling / copying: protocols/internet/test_mh_timestamp_info_1886_unit (pickles at every protocol; fix(corekit): copy SeekableReader's octets and feed MultiDict a dict, and gate GraalPy's limits (#1963) #1974 gates it for GraalPy's pickle 0/1), protocols/test_to_dict_repeated_exthdr_1453_unit, corekit/test_fields_numbers_port_option_no_mint_unit, corekit/test_fields_numbers_port_enum_field_peek_1774_unit, corekit/test_fields_misc, protocols/internet/test_ipv6_option_helpers_1519_unit
Warnings and threading hooks: protocols/test_trial_parse_warnings_1580_unit (free-threaded context_aware_warnings), corekit/test_enum_get_exception_provenance_923_unit (threading.excepthook)
Engines: Scapy is LATEST while the other engines stay ALL. This needs an explicit reason, or a move to ALL.
Vendor: the six vendor tests that lost their markers need a stated reason in the code.
"Before" count: the review reproduced 3677 unit ids, not 3787.

Non-blocking: say that the version/implementation guard is syntactic only. It misses aliases, getattr/hasattr probes, and sys.flags/Py_GIL_DISABLED reads.

…#1958) - Opt in the modules the review found on LATEST despite the criterion: pickling and copying, the warning machinery and exception hooks, struct and int boundary tests, and the modules #1974 gates on GraalPy: fallback cycle, MH timestamp and the two #1541 overrun tests that compare against sys.maxsize. - Opt Scapy's engine and toolkit modules into ALL like the other engines, and PyShark's toolkit and non-packet modules with them. - State each demoted vendor module's LATEST tier and reason. - branches_on_interpreter() also sees sys.flags, sys._is_gil_enabled, sys.maxsize comparisons, sysconfig.get_config_var() and hasattr/getattr probes of sys; its docstring and the guard's message say the check is syntactic and best-effort. - _tiers.py and workflows.rst "Test Tiers" give the Scapy and vendor reasons. Tier guard, #1732, #1954, #1958 pins and every touched module pass on CPython 3.14.

JarryShaw · 2026-10-20T14:51:13Z

Pushed d963c39bf.

Into ALL: the eight flagged modules, more warnings and struct-boundary modules, the Scapy engine and toolkit tests, and the GraalPy-gated modules from fix(corekit): copy SeekableReader's octets and feed MultiDict a dict, and gate GraalPy's limits (#1963) #1974.
Vendor: the six vendor tests declare LATEST, each with a reason.
Guard: it is now documented as best-effort, and it also catches sys.flags, sys.maxsize, sysconfig and hasattr/getattr probes of sys.
Counts: ALL has 2207 ids, runs in 474 s and covers 85.3%. The worker re-measured the "before" count as 3787 in two venvs. The reviewer's 3677 likely reflects different extras.
fix(corekit): copy SeekableReader's octets and feed MultiDict a dict, and gate GraalPy's limits (#1963) #1974: the merge with it is clean, and the guard passes on the merged tree.

A delta review is running.

JarryShaw · 2026-10-20T15:02:49Z

GOOD TO GO at d963c39bf. A Sonnet delta review found:

Tiers: every flagged module is now ALL. A full rescan of modules that moved to LATEST finds only incidental matches.
Scapy: its tests skip cleanly where Scapy is missing (47 skipped, 0 failed).
Guard: it catches each new pattern, and every module it flags declares a tier.
Counts: 3787 before and 2207 after, re-derived the way the workflow selects them.
fix(corekit): copy SeekableReader's octets and feed MultiDict a dict, and gate GraalPy's limits (#1963) #1974: the merge with it passes the guard.

Non-blocking: the criterion names recursion limits, but the guard does not detect sys.getrecursionlimit. CI runs once this PR targets main, after #1976.

… and gate GraalPy's limits (#1963) (#1974) * fix(corekit): copy SeekableReader's octets and feed MultiDict a dict, and gate GraalPy's limits (#1963) - SeekableReader copies buffered octets out of its view: GraalPy's BytesIO read/read1/readline from an exported buffer's start can return bytes sharing its memory, which the next buffer slide overwrote - MultiDict hands dict.__init__ and dict.update a dict, not pairs: GraalPy stores pairs through the overridden __setitem__, nesting each value list - new test_graalpy_semantics_1963_unit reproduces both on CPython with stand-ins for GraalPy's behaviour - gate in the tests: pickle protocols 0/1 (typing.Generic is not a heap type), the >sys.maxsize PCAP claim, resource.setrlimit and _abc_impl; bound the huge BytesField length by sys.maxsize Refs #1963 * test(corekit): keep only the cases that fail without the GraalPy fixes (#1963) - drop the OrderedMultiDict copy and the plain-dict state cases, which passed without the fixes - say why the stand-in check stays: it guards the stand-in, not the fix - drop the count from the module docstring - correction to the first commit's review: werkzeug 2.3.8's MultiDict.__init__ and __setstate__ match pcapkit's apart from the annotations and the l -> v rename Refs #1963 * fix(corekit): upgrade the MultiDict port to werkzeug 3.1.9 (#1963) - port MultiDict, OrderedMultiDict and iter_multi_items from werkzeug 3.1.9 rather than 2.3.8: dict reached through super(), any Mapping and set values accepted, | and |= added, setdefault/setlistdefault simplified - keep the local adaptations: dict base without TypeConversionDict, MissingKeyError/UnsupportedCall, get() and getlist() without type=, OrderedMultiDict.deepcopy order fix, and no DeprecationWarning on OrderedMultiDict - werkzeug 3.1.9 still feeds pairs to dict.__init__ and update, which GraalPy routes through __setitem__; keep the two dict-building lines as the only behavioural divergence, noted in place - the GraalPy stand-in now sits in the MRO behind MultiDict, since super() no longer reads a module-global dict - test the new Mapping/set handling and the | operators tests/corekit and every test module importing the multidict pass on CPython; the #1963 and multidict tests pass on GraalPy 25.4.4. Refs #1963 * fix(corekit): keep the MultiDict port verbatim to werkzeug 3.1.9 (#1963) - drop the versionchanged notes and explanatory prose this PR added - restore werkzeug 3.1.9's docstrings, comments, overloads, annotations (as strings) and code layout for the ported classes - keep pcapkit's adaptations: dict + Generic base, MissingKeyError and UnsupportedCall, no type= on getlist, local get() and _missing, the #1295 OrderedMultiDict.deepcopy, poplist's [] default, and no OrderedMultiDict deprecation warning - keep the two GraalPy dict-building lines, with shorter comments tests/corekit passes on CPython; the multidict and #1963 modules pass on CPython and GraalPy 25.4.4. Refs #1963 * test(project): know the useless-parent-delegation pragma (#1963) - add useless-parent-delegation (W0246) to KNOWN_MESSAGES: the pragma on MultiDict.__iter__ keeps upstream's CPython 87412 fast-path workaround Refs #1963 * test(corekit): declare the GraalPy semantics module ALL-tier (#1963) - add __tier__ = 'ALL': the module pins GraalPy-only semantics, so it has to keep running on every interpreter leg once ALL becomes opt-in (#1978) Refs #1963

JarryShaw · 2026-10-20T15:22:06Z

GOOD TO GO at e864673d2. The commit only merges main: its tree is identical to the automatic merge of the reviewed d963c39bf with main (1ebf31aad). On it, the tier guard, the curated and whole-suite pins, and the three GraalPy-gated modules pass (184 passed). CI is running for the first time, now that the PR targets main.

github-actions · 2026-10-20T15:53:55Z

Coverage: 91.52% (unit tier, Python 3.15, e864673d2, Unit Tests run success)

Package	Statements	Missed	Branches	Partial	Cover
`pcapkit (top level)`	104	4	20	2	95.16%
`pcapkit/const`	19597	881	2618	867	92.11%
`pcapkit/corekit`	2524	70	800	36	96.57%
`pcapkit/dumpkit`	276	3	100	3	98.40%
`pcapkit/foundation`	2916	23	1016	25	98.78%
`pcapkit/interface`	138	4	52	3	96.32%
`pcapkit/protocols`	17411	224	4580	189	98.10%
`pcapkit/toolkit`	872	36	298	10	95.21%
`pcapkit/utilities`	522	2	158	4	99.12%
`pcapkit/vendor`	4431	1995	1018	167	51.73%

Per-file detail: the coverage-html artifact of this run.

JarryShaw added 3 commits October 20, 2026 00:30

JarryShaw added test Pull requests that add or correct tests (test: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 20, 2026

JarryShaw moved this to In review in PyPCAPKit Oct 20, 2026

JarryShaw added this to PyPCAPKit Oct 20, 2026

JarryShaw mentioned this pull request Oct 20, 2026

fix(corekit): copy SeekableReader's octets and feed MultiDict a dict, and gate GraalPy's limits (#1963) #1974

Merged

5 tasks done

JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 20, 2026

JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Oct 20, 2026

JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 20, 2026

Base automatically changed from ci/1958-whole-suite-latest to main October 20, 2026 15:16

Merge branch 'main' into ci/1958-curated-all

e864673

JarryShaw mentioned this pull request Oct 20, 2026

chore: cite issues as #NNN in plain comments #1982

Merged

5 tasks done

JarryShaw merged commit 43aa794 into main Oct 20, 2026
55 checks passed

JarryShaw deleted the ci/1958-curated-all branch October 20, 2026 16:07

github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 20, 2026

JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 20, 2026

JarryShaw mentioned this pull request Oct 20, 2026

test: sort the remaining GraalPy-only failures into defects and gates #1963

Closed

JarryShaw mentioned this pull request Oct 20, 2026

ci: categorise the tests into tiers and run each tier where it belongs #1958

Closed

JarryShaw added this to the PyPCAPKit v1.5 milestone Oct 26, 2026

Navigation Menu

Repository navigation

test(tiers): make ALL a curated opt-in tier (#1958) - #1978

test(tiers): make ALL a curated opt-in tier (#1958)#1978
JarryShaw merged 5 commits into
mainfrom
ci/1958-curated-all

JarryShaw commented Oct 20, 2026 •

edited

Loading

Uh oh!

JarryShaw commented Oct 20, 2026

Uh oh!

JarryShaw commented Oct 20, 2026

Uh oh!

JarryShaw commented Oct 20, 2026

Uh oh!

JarryShaw commented Oct 20, 2026

Uh oh!

github-actions Bot commented Oct 20, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

JarryShaw commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is the purpose of your pull request?

Description of your pull request and other information

Uh oh!

JarryShaw commented Oct 20, 2026

Uh oh!

JarryShaw commented Oct 20, 2026

Uh oh!

JarryShaw commented Oct 20, 2026

Uh oh!

JarryShaw commented Oct 20, 2026

Uh oh!

github-actions Bot commented Oct 20, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

JarryShaw commented Oct 20, 2026 •

edited

Loading