Repository navigation
fix(toolkit): pypcapfile reads IP over loopback and raw-IP link types, up to the frame layer bound (#1609, #1667) - #1682
Merged
JarryShaw merged 2 commits intoOct 13, 2026
Conversation
…, up to the frame layer bound (#1609, #1667) - pypcapfile decodes Ethernet only, so the adapters handed no TCP, trace or IPv4 input over on NULL, LOOP, RAW, IPV4 and IPV6. They now read the IP packet there themselves (READ_LINKTYPES): the loopback family by Loopback's table and family_byteorder, RAW by its version nibble. TCP over IPv6 stays declined with the per-capture warning; 12 and LINUX_SLL stay unread, and only those still get the engine's "unrecognised link layer protocol" warning. - The VLAN, tunnel and extension-header walks ran past FRAME_LAYER_LIMIT. Each VLAN tag, IP header, AH and IPv6 extension header now counts one layer, as in the default engine, and nothing past layer 64 is read or warned of. _upper_layer, which pyshark imports, is unchanged. - packet2dict reported an unread frame's hex text (raw_len 94 for 47 octets); it now reports the octets. New tests fail on main and pass here on 3.11 and 3.14. Closes #1667 Part of #1609
… frame layer bound (#1609, #1667) - protocol.rst listed pypcapfile with pyshark as not applying FRAME_LAYER_LIMIT. It now stands with dpkt and scapy; pyshark's sentence is otherwise unchanged. - The toolkit note names the link types the adapters read and the layer bound, and documents READ_LINKTYPES. The engine note says the toolkit reads the IP packet of loopback and raw-IP frames. - The previous commit's message says 12 and LINUX_SLL are the only link types still warned of. Every link type but Ethernet and the five READ_LINKTYPES still is; the engine test now checks PPP (9), 105 and 147 as well. The pypcapfile tests pass on 3.11 and the touched tests on 3.14; the engine test still fails on main on both. Closes #1667 Part of #1609
JarryShaw
added
fix
Owner
Author
|
Verdict: GOOD TO GO at What it does. The pypcapfile adapters now read the IP packet on NULL, LOOP, RAW, IPV4 and IPV6 link types (#1667). They read it with pypcapfile's own What the reviewer measured. 481 of its own captures: 307 disagreements with the default engine on Round 2. Left as you may want them: these five link types now give no engine warning, and an undecodable IPv4 packet on them does not warn either. |
Contributor
|
Coverage: 89.48% (unit tier, Python 3.14,
Per-file detail: the |
JarryShaw
deleted the
fix/1609-1667-pypcapfile-linktypes-and-layer-bound
branch
October 13, 2026 21:53
JarryShaw
removed
the
review: good-to-go
This was referenced Oct 13, 2026
JarryShaw
added a commit
that referenced
this pull request
Oct 14, 2026
…1721) Third-party engines are best effort (#1687), so the pypcapfile code that exists only to match the default engine comes out: - revert #1682, #1600, #1575 and #1535, and the pypcapfile hunks of #1655, #1608 and #1605: reading loopback and raw-IP link types, the frame layer bound, the VLAN and tunnel walks, keying tunnelled TCP by the innermost header, stepping over AH and extension headers, and checking options with the default engine's parsers - keep the helpers the dpkt and scapy adapters import, unchanged, until their own reverts drop them; their docstrings no longer say pypcapfile asks the default engine's parsers too - re-land #1576, #1577, #1590 and #1591 as guards in _network and _transport, #1513's once-per-capture warning for TCP straight over IPv6, and #1667's packet2dict fix: an undecoded frame is reported as its octets, not as hex text twice as long - keep #1569's record reader and #1690's packet2timestamp - delete the three test modules those reverts added, the pypcapfile timestamp agreement class, and #1682's link-type warning test; the re-landed guards' tests assert constants - drop what the reverts removed from the docs, READ_LINKTYPES included, and say that TCP behind a VLAN tag, in a tunnel or behind an extension header (an IPv6 one, or AH in IPv4) is now left out without a warning The pypcapfile modules, test_new_engine_parity_runtime.py, test_engine_runtime.py, the timestamp module and the project guards pass on 3.11 (pypcapfile 0.12.0) and 3.14.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please follow the guide below
You will be asked some questions, please read them carefully and answer honestly
Put an
xinto all the boxes [ ] relevant to your pull request (like that [x])Use Preview tab to see how your pull request will actually look like
Searched for similar pull requests
Followed the coding style (
make pylint,make mypy,make isort)make testpasses, and a test case covers the change — tests cover it; only the toolkit, engine and pypcapfile-related selections were run, not the whole suiteAdded a changelog entry under
docs/source/changelog/and regeneratedCHANGELOG.md, if the change is user-visible — N/A — centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657What is the purpose of your pull request?
Tick the commit type your subject line carries.
fix— corrects a defectfeat— adds a featureperf— changes performance, not behaviourrefactor— changes neither behaviour nor performancetest— tests onlydocs— documentation onlyci— workflows or build toolingrelease— bumps the version or rolls up a distributionchore— anything elseDescription of your pull request and other information
The pypcapfile adapters now read what the default engine reads:
IP(data, 0). Loopback families come fromLoopback's table infamily_byteorder's byte order, and RAW is split by its version nibble. TCP over IPv6 stays declined with the per-capture warning. Every other link type but Ethernet (e.g. PPP 9, 12, 105, LINUX_SLL 113, 147) stays unread, as default leaves it, and still gets the "unrecognised link layer protocol" warning; only those five stop getting it.packet2dictreports an unread frame's octets (raw_len47, not 94).FRAME_LAYER_LIMIT, warning included._upper_layer,_past_extension_headersand_walk_extension_headersbehave as before for pyshark, dpkt and scapy.protocol.rstnow lists pypcapfile with dpkt and scapy as stopping at the bound (pyshark is left to fix(toolkit): pyshark traces no TCP inside encapsulations the default engine leaves raw (#1639) #1670), and the toolkit note names the link types read and the bound.I measured 154 captures (the issue rows, the #1609 depth rows on Ethernet and on the new link types, controls, and the sample corpus) with every engine on 3.11.15 and default, dpkt and scapy on 3.14.8, on
4673cffa9. The branch is rebased onto2025da5e5;257580444and2025da5e5touch the Conda workflow and its release-gate test, the version, and HIP/MH constant enumerations only, and were not re-measured. 94 pypcapfile cells changed, and no cell of any other engine did. pypcapfile now matches default field for field on every PCAP capture, and its TCP-over-IPv6 warning fires exactly where default finds TCP over IPv6. The six PCAP-NG refusals are unchanged, by design.Two new test files, plus one engine test, which also checks PPP, 105 and 147 still warn. All 10 new test methods fail on
main(the agreement module's two inheritedRoundTripBasetable checks pass anywhere) and pass with the fix, on 3.11 and on 3.14, where the real-engine comparison is skipped. Under the pypcapfile tests on 3.14, coverage ofpcapkit/toolkit/pypcapfile.pygoes from 98.93% to 100.00%.Closes #1667
Part of #1609