Repository navigation
HAS_PYPCAPFILE stays dark on 3.12-3.14 regardless of install line: pypcapfile is marker-limited #776
Description
Activity
Decision needed: is this worth doing, given most of it is already in place and the rest needs a larger change?
Measured at
83b58ebda. 15 methods, matching the current figure — 9 intests/toolkit/test_pypcapfile_unit.py(class-level gate) and 6 intests/foundation/engines/test_new_engine_parity_runtime.py(per-method), by AST walk cross-checked against grep and against the guard's own prose.The version-bump route is dead, but only just.
pypcapfile's latest PyPI release is 0.12.0, uploaded 2016-09-08 (18 releases, none since). Upstream has fixed it —kisom/pypcapfilecommit0aeb29e, 2024-05-29, swapsimport impforimportlib, closing its own issue #56 — but no release was ever cut.pcapfile/linklayer.pyimportsimpat module scope andsavefile.pyimportslinklayerat module scope, so it is unavoidable forpcapkit/foundation/engines/pypcapfile.py:165-167. Consuming the fix today would mean a VCS direct reference (pypcapfile @ git+…@0aeb29e) — materially unlike every other extra, which resolves from PyPI with version and marker specifiers only. It would add a GitHub-availability dependency to CI installs, pin an untagged commit of a project idle since 2024, and probably need handling in the guard'sRequirementparser, which is built for PEP 508name; markerrather thanname @ url; marker.Option 1 is already substantially in place — landed by #755, before this issue was filed.
DEPENDENCY_GATE_EXCLUSIONS['HAS_PYPCAPFILE']'sreasonalready states in prose thatengine-tests/pypcap-parityinstall the extra across the full 3.10-3.14 matrix while the marker only resolves on 3.10/3.11, and that the guard "cannot see that only two of five legs run for real … so that partial coverage is recorded here in prose rather than modelled". The module docstring attests/_dependency_gates.py:46-51separately names environment markers as deliberately unmodelled, using this exact extra as its worked example.What is genuinely missing is the structured half:
Exclusion.darklists onlytest/integration/gate, not thatengine-testsandpypcap-parityare partially dark on 3 of 5 legs each. And it cannot record that without matrix-awareness —pytest_jobs()returns oneJobper job name with no per-Python-version dimension, sopython_version < '3.12'is meaningless to it until the whole job × version matrix is modelled. That is a much larger widening than "teach the guard markers" suggests, and is precisely what the docstring's refusal was written to avoid.So three ways, none obviously right:
- Close as already-documented. The fact is recorded where a reader of the guard will find it; only the structured assertion is absent.
- Extend
darkto record the partial legs, accepting it is prose-adjacent rather than matrix-modelled. Needs coordination with whattest_each_exclusion_still_describes_a_gap_that_is_really_thereasserts aboutdark's shape — and ci(deps): install the vendor extra on engine-tests, closing HAS_VENDOR_DEPS (#738) #774 owns those files until it merges. - Model the job × Python-version matrix so markers become evaluable and this reports as a real gap. Correct, and the largest of the three.
Blocked on #774 either way for options 2 and 3.
Not entirely sure the context but we definitely want full coverage (all possible Python versions being tested). And we should bump the deps version if new versions are available and fixing some blockers for us.
Ruling recorded, in the maintainer's words: "Not entirely sure the context but we definitely want full coverage (all possible Python versions being tested). And we should bump the deps version if new versions are available and fixing some blockers for us."
That settles the preference and supersedes my three options — the answer is "fix the dependency", not "document the gap". But the two halves of the ruling collide, because there is no version to bump to. Measured:
pypcapfile 0.12.0 2016-09-08 requires_python=(unset) <- the only release, 18 total pcapfile not on PyPI python-pcapfile not on PyPI pypcapfile-ng not on PyPI pcapfile-py not on PyPINo newer release, and no maintained fork exists on PyPI either. Upstream has fixed it —
kisom/pypcapfilecommit0aeb29e, 2024-05-29, swappingimport impforimportliband closing its own issue #56 — but no release was ever cut, and the project has had no activity since.pyproject.toml:199-200's existing comment already says exactly this.So full coverage has exactly one path: a VCS direct reference,
pypcapfile @ git+https://github.com/kisom/pypcapfile@0aeb29e. What that costs, stated plainly because it is a real change in kind:- Every other extra in
pyproject.tomlresolves from PyPI with version and marker specifiers only. This would be the first direct reference. - It adds a GitHub-availability dependency to every CI install that includes the extra.
- It pins an untagged commit of a project idle since 2024, with no release process to fall back on.
- The dependency-gate guard's
Requirementparser is built for PEP 508name; marker;name @ url; markerwould likely need handling there too. - PyPI rejects direct references in uploaded distributions, so this is only viable if the extra is never part of a published sdist/wheel's metadata — worth confirming against how
pyproject.toml's extras are packaged before committing to it.
One decision left: take the VCS reference to get full coverage, or keep the marker and accept 3.10/3.11 only? The bump you asked for is not available, so those are the two real options. I have not implemented either. Re-applying
needs: decisionfor that one question; everything else here is settled.For completeness, the coverage at stake is 15 methods — 9 in
tests/toolkit/test_pypcapfile_unit.py(class-level gate) and 6 intests/foundation/engines/test_new_engine_parity_runtime.py— currently running on 2 of 5 legs.- Every other extra in
Oh in this case, let's keep marker and accept the limit of the 3P module's released versions.
When i say "cover as much as possible", it only applies to actual releases, not git commits based distro.
Ruling recorded, in the maintainer's words: "Oh in this case, let's keep marker and accept the limit of the 3P module's released versions." and "When i say 'cover as much as possible', it only applies to actual releases, not git commits based distro."
So: no VCS direct reference.
pyproject.toml:201keeps"pypcapfile; python_version < '3.12'"unchanged, and the 15 gated methods run on 3.10/3.11 only — 9 intests/toolkit/test_pypcapfile_unit.py, 6 intests/foundation/engines/test_new_engine_parity_runtime.py. The earlier "we definitely want full coverage" is scoped by this to coverage reachable from released distributions, which forpypcapfilemeans 0.12.0 (2016-09-08) and nothing more.Closing as resolved. What this issue asked for was either the methods executing on every leg, or the reason they cannot being visible. The first is now explicitly declined, and the second is already in place — landed by #755, before this issue was filed:
DEPENDENCY_GATE_EXCLUSIONS['HAS_PYPCAPFILE']'sreasonstates thatengine-tests/pypcap-parityinstall the extra across the full 3.10-3.14 matrix while the marker only resolves on two legs, andtests/_dependency_gates.py:46-51names environment markers as deliberately unmodelled using this exact extra as its worked example.pyproject.toml:197-200records the cause and that upstream fixed it without releasing.One thing deliberately not done, and it would be a different issue if wanted:
Exclusion.darkstill lists onlytest/integration/gate, so the partial darkness ofengine-tests/pypcap-parityis narrated rather than asserted. That cannot be modelled as things stand —pytest_jobs()returns oneJobper job name with no per-Python-version dimension, sopython_version < '3.12'is meaningless to it until the whole job × version matrix is modelled, which is precisely what that docstring's refusal avoids. If the structured assertion is wanted later it is a guard-modelling issue, not this one.Filing note for the record: I opened this issue on the premise that the fact was undocumented. It was already documented. The measurement that was genuinely new is that no maintained fork exists on PyPI —
pcapfile,python-pcapfile,pypcapfile-ngandpcapfile-pyall 404 — so there was never a bump available to take.
Describe the bug
HAS_PYPCAPFILE's gated tests stay dark on Python 3.12, 3.13 and 3.14 no matter which job installs the extra, because the dependency is marker-limited:The reason is recorded and legitimate (
pyproject.toml:197): "pypcapfile 0.12.0'slinklayermodule importsimp, removed in Python 3.12". So.[...,PyPCAPFile]resolves to nothing on three of the five matrix legs, and the 15 gated methods report as skips there.Expected behavior
Either the gated methods execute on every leg, or the fact that they cannot is visible to the dependency-gate guard rather than only to a reader of
pyproject.toml.Additional context
This is residual #738 scope. #738 is "seven more
skipUnlessgates have never executed on any CI path", and #774 closes the last of the seven by name — butHAS_PYPCAPFILEis only partially closed:engine-testsandpypcap-parityinstall the extra, so the methods run on 3.10 and 3.11 and are dark on 3.12-3.14. Filing separately so #774'sFixes #738.can close cleanly instead of burying this.tests/_dependency_gates.pydeliberately does not evaluate environment markers — its own docstring says so — sodependency_gate_gaps()reports no gap here and cannot. That is a defensible scope decision rather than a guard defect; the consequence is that this particular hole is invisible to the machinery built to find exactly this class of hole, which is what makes it worth its own issue.Options, none chosen: pin a
pypcapfilefork or a patched build that does not importimp; teach the guard to evaluate markers so the 3.12-3.14 darkness is reported as a gap and then explicitly excluded with this reason; or accept it and record it inDEPENDENCY_GATE_EXCLUSIONSas a marker-limited partial, which at least puts it where the next reader of the guard will find it. Surfaced by the cross-review of #774.