Repository navigation
fix(toolkit): dpkt and scapy check IPv4 headers as the default engine does (#1607) - #1614
Merged
Merged
Conversation
… does (#1607) - dpkt and scapy decoded an IPv4 header the default engine rejects (version not 4, IHL below 5 or past the capture, options its parser rejects) and still handed on IPv4 and TCP input. Each IPv4 header is now asked of the default engine's own parser via pcapkit.toolkit.pypcapfile._default_accepts, outer and tunnelled alike; version 4 with IHL 5 skips the parser. - Scapy hands the parser at least the 20 fixed octets, so an IHL below 5 warns of nothing the default engine does not. New tests in tests/toolkit/test_dpkt_scapy_ipv4_header_checks_1607_unit.py fail without the fix.
Contributor
|
Coverage: 89.70% (unit tier, Python 3.14,
Per-file detail: the |
Owner
Author
|
Verdict: GOOD TO GO at The review built its own corpus of about 61,000 frames:
Results:
Non-blocking:
|
This was referenced Oct 10, 2026
This was referenced Oct 10, 2026
JarryShaw
added a commit
that referenced
this pull request
Oct 14, 2026
Third-party engines are best effort (#1687), so the scapy code that exists only to match the default engine comes out: - revert #1683, #1663, #1652 and #1631, and the scapy hunks of #1614: the link-type and loopback-family walk, the frame and extension-header bounds, the IPv6 chain walk over the octets, the stops at GRE, VXLAN, PPPoE and the other layers the default engine leaves raw and at ICMP quotes, and the IPv4 header checks - keep #1624's walk (TCP behind AH or an IPv6 extension header in IPv4, and the later-fragment stop), #1605's keying by the carrying IP header and #1589's trailer cut; the walk calls _walk_extension_headers, which the pypcapfile revert keeps - re-land as guards: a zero-length frame is no IPv4 input and has no packet2dict octets (#1676); IPv4 in IPv6 ends at the Payload Length (#1623); a packet built by hand is read off its octets (#1676); ipv6_reassembly takes only its own IPv6 layer's Fragment header, and none in a later IPv4 fragment - delete the four scapy agreement modules and test_scapy_unit's default-engine checks; the guards' tests assert constants - docs: scapy applies the extension-header bound only behind IPv4 and the frame-layer bound nowhere, and its silent gaps are listed where gaps are said to be announced The scapy toolkit and engine modules, the shared modules that run scapy, and the project and tier guards pass on 3.14 (scapy 2.7.0) and 3.11 (scapy 2.8.0).
3 of 13 tasks
JarryShaw
added a commit
that referenced
this pull request
Oct 14, 2026
Third-party engines are best effort (#1687), so the scapy code that exists only to match the default engine comes out: - revert #1683, #1663, #1652 and #1631, and the scapy hunks of #1614: the link-type and loopback-family walk, the frame and extension-header bounds, the IPv6 chain walk over the octets, the stops at GRE, VXLAN, PPPoE and the other layers the default engine leaves raw and at ICMP quotes, and the IPv4 header checks - keep #1624's walk (TCP behind AH or an IPv6 extension header in IPv4, and the later-fragment stop), #1605's keying by the carrying IP header and #1589's trailer cut; the walk calls _walk_extension_headers, which the pypcapfile revert keeps - re-land as guards: a zero-length frame is no IPv4 input and has no packet2dict octets (#1676); IPv4 in IPv6 ends at the Payload Length (#1623); a packet built by hand is read off its octets (#1676); ipv6_reassembly takes only its own IPv6 layer's Fragment header, and none in a later IPv4 fragment - delete the four scapy agreement modules and test_scapy_unit's default-engine checks; the guards' tests assert constants - docs: scapy applies the extension-header bound only behind IPv4 and the frame-layer bound nowhere, and its silent gaps are listed where gaps are said to be announced The scapy toolkit and engine modules, the shared modules that run scapy, and the project and tier guards pass on 3.14 (scapy 2.7.0) and 3.11 (scapy 2.8.0).
JarryShaw
added a commit
that referenced
this pull request
Oct 14, 2026
Third-party engines are best effort (#1687), so the dpkt code that exists only to match the default engine comes out: - revert #1662, #1655, #1624, #1622, #1614 and the dpkt hunks of #1605: the octet walk and its link-type stash, IPv6 in IPv4 and IPv4 in IPv6, refusing MPLS and ISL, the tunnel and extension-header walks, the IPv4 header checks and the layer bounds - keep #1690's timestamps, #1594's parser choice, #1621 and #1653 - re-land as guards: a layer dpkt's own lengths misplace (#1646) is re-serialised, not refused with ProtocolError; a Total Length below the IHL keeps the header whole (#1616), and a header the capture cuts is no input; packet2dict and packet2chain take a str payload and a raw frame - delete the modules #1662 and #1655 added; the 1615/1616 module keeps only what its #1616 tests use, as no scapy test imports it now; dpkt's comparisons with the default engine assert constants or the capture's octets, or are dropped - drop pypcapfile's VLAN_TAGS, VLAN_TAG_LEN and IPV6_EXTENSION_HEADERS, which only dpkt imported, and the 1668 module's _Agreement base, which nothing subclasses now - the docs drop the depth claims and name what dpkt leaves out silently The dpkt modules, the shared modules that run dpkt, tests/foundation/engines and the project guards pass on 3.14 and 3.11 (dpkt 1.9.8, pypcapfile 0.12.0).
JarryShaw
added a commit
that referenced
this pull request
Oct 14, 2026
Third-party engines are best effort (#1687), so the dpkt code that exists only to match the default engine comes out: - revert #1662, #1655, #1624, #1622, #1614 and the dpkt hunks of #1605: the octet walk and its link-type stash, IPv6 in IPv4 and IPv4 in IPv6, refusing MPLS and ISL, the tunnel and extension-header walks, the IPv4 header checks and the layer bounds - keep #1690's timestamps, #1594's parser choice, #1621 and #1653 - re-land as guards: a layer dpkt's own lengths misplace (#1646) is re-serialised, not refused with ProtocolError; a Total Length below the IHL keeps the header whole (#1616), and a header the capture cuts is no input; packet2dict and packet2chain take a str payload and a raw frame - delete the modules #1662 and #1655 added; the 1615/1616 module keeps only what its #1616 tests use, as no scapy test imports it now; dpkt's comparisons with the default engine assert constants or the capture's octets, or are dropped - drop pypcapfile's VLAN_TAGS, VLAN_TAG_LEN and IPV6_EXTENSION_HEADERS, which only dpkt imported, and the 1668 module's _Agreement base, which nothing subclasses now - the docs drop the depth claims and name what dpkt leaves out silently The dpkt modules, the shared modules that run dpkt, tests/foundation/engines and the project guards pass on 3.14 and 3.11 (dpkt 1.9.8, pypcapfile 0.12.0).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please follow the guide below
You will be asked some questions, please read them carefully and answer honestly
Put an
xinto all the boxes [ ] relevant to your pull request (like that [x])Use Preview tab to see how your pull request will actually look like
Searched for similar pull requests
Followed the coding style (
make pylint,make mypy,make isort)make testpasses, and a test case covers the changeAdded a changelog entry under
docs/source/changelog/and regeneratedCHANGELOG.md, if the change is user-visible — N/A — centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657What is the purpose of your pull request?
Tick the commit type your subject line carries.
fix— corrects a defectfeat— adds a featureperf— changes performance, not behaviourrefactor— changes neither behaviour nor performancetest— tests onlydocs— documentation onlyci— workflows or build toolingrelease— bumps the version or rolls up a distributionchore— anything elseDescription of your pull request and other information
Closes #1607.
dpkt and scapy decode any IPv4 EtherType or tunnel payload as IPv4, whatever its version, and check none of its options, so a header the default engine rejects still produced IPv4 and TCP input.
Each now asks the default engine's own IPv4 parser, reusing
pcapkit.toolkit.pypcapfile._default_acceptsfrom #1600 rather than a third copy of the rules (pypcapfile.pyitself is unchanged):_ipv4_acceptedin each toolkit; a version-4, IHL-5 header skips the parser._tcp_carrierand scapy_carriercheck every IPv4 header between the frame and the TCP segment, so outer and tunnelled headers are both covered (fix(toolkit): key tunnelled TCP by the innermost IP header (#1581) #1605).ipv4_reassemblyof both checks its IPv4 layer too, as the default engine hands no IPv4 input for a rejected header.Fields checked: the default engine rejects a bad version, an IHL below 5 or past the capture, and bad options. It does not check the header checksum, so that already agreed. Across 42 crafted frames, the cases where IPv4, TCP or flow counts disagree went from 28 to 2. Both remaining cases are dpkt finding no IPv4 inside IPv6 for IPv4 reassembly. The cross-review found further pre-existing dpkt and scapy disagreements that are not about header validity, tracked separately.
New tests:
tests/toolkit/test_dpkt_scapy_ipv4_header_checks_1607_unit.py(4 tests, all fail onmain). The full suite was not run locally, only narrow selections.