Repository navigation
fix(toolkit): dpkt and scapy find TCP in IPv4 where the default engine does (#1606, #1617) - #1624
Conversation
|
Coverage: 89.67% (unit tier, Python 3.14,
Per-file detail: the |
|
Verdict: NEEDS CHANGES at Sonnet cross-review. The fix holds: on 216 hand-built frames, disagreements with the default engine fall from 390 to 3, and #1617's second cause (scapy binds AH to proto 51 without
Required: catch as the engine does in |
e0aff30 to
4edf009
Compare
…e does (#1606, #1617) - dpkt and scapy step over an AH or IPv6 extension header carried directly in IPv4 (protocols 0, 43, 44, 51, 60, 135, 139) by the default engine's own rules, via pypcapfile's _past_extension_headers, bounded at EXTENSION_HEADER_LIMIT; the TCP segment or IP-in-IP tunnel behind them is decoded from the captured octets. Behind such a header, TCP is read only as the default TCP parser accepts it. - dpkt keeps a tunnel it raises anything decoding undecoded, as its engine keeps such a frame raw, rather than ending the extraction. - dpkt's packet2dict and packet2chain treat only a packet-like payload as a layer; a str payload, such as VRRP's '', is leaf data and ends the chain. - scapy's carrier walk now runs top-down and stops at a later IPv4 fragment, which scapy still dissects IPv6 and AH in. - Drop the #1606 gap row from the #1604 chain-limit agreement test.
4edf009 to
6c7b4d0
Compare
|
Verdict: GOOD TO GO at Sonnet cross-review, round 2. Both dpkt crashes are gone.
I reproduced the three new tests' fail-before myself: 3 failed with |
Third-party engines are best effort (#1687), so the scapy code that exists only to match the default engine comes out: - revert #1683, #1663, #1652 and #1631, and the scapy hunks of #1614: the link-type and loopback-family walk, the frame and extension-header bounds, the IPv6 chain walk over the octets, the stops at GRE, VXLAN, PPPoE and the other layers the default engine leaves raw and at ICMP quotes, and the IPv4 header checks - keep #1624's walk (TCP behind AH or an IPv6 extension header in IPv4, and the later-fragment stop), #1605's keying by the carrying IP header and #1589's trailer cut; the walk calls _walk_extension_headers, which the pypcapfile revert keeps - re-land as guards: a zero-length frame is no IPv4 input and has no packet2dict octets (#1676); IPv4 in IPv6 ends at the Payload Length (#1623); a packet built by hand is read off its octets (#1676); ipv6_reassembly takes only its own IPv6 layer's Fragment header, and none in a later IPv4 fragment - delete the four scapy agreement modules and test_scapy_unit's default-engine checks; the guards' tests assert constants - docs: scapy applies the extension-header bound only behind IPv4 and the frame-layer bound nowhere, and its silent gaps are listed where gaps are said to be announced The scapy toolkit and engine modules, the shared modules that run scapy, and the project and tier guards pass on 3.14 (scapy 2.7.0) and 3.11 (scapy 2.8.0).
Third-party engines are best effort (#1687), so the scapy code that exists only to match the default engine comes out: - revert #1683, #1663, #1652 and #1631, and the scapy hunks of #1614: the link-type and loopback-family walk, the frame and extension-header bounds, the IPv6 chain walk over the octets, the stops at GRE, VXLAN, PPPoE and the other layers the default engine leaves raw and at ICMP quotes, and the IPv4 header checks - keep #1624's walk (TCP behind AH or an IPv6 extension header in IPv4, and the later-fragment stop), #1605's keying by the carrying IP header and #1589's trailer cut; the walk calls _walk_extension_headers, which the pypcapfile revert keeps - re-land as guards: a zero-length frame is no IPv4 input and has no packet2dict octets (#1676); IPv4 in IPv6 ends at the Payload Length (#1623); a packet built by hand is read off its octets (#1676); ipv6_reassembly takes only its own IPv6 layer's Fragment header, and none in a later IPv4 fragment - delete the four scapy agreement modules and test_scapy_unit's default-engine checks; the guards' tests assert constants - docs: scapy applies the extension-header bound only behind IPv4 and the frame-layer bound nowhere, and its silent gaps are listed where gaps are said to be announced The scapy toolkit and engine modules, the shared modules that run scapy, and the project and tier guards pass on 3.14 (scapy 2.7.0) and 3.11 (scapy 2.8.0).
Third-party engines are best effort (#1687), so the dpkt code that exists only to match the default engine comes out: - revert #1662, #1655, #1624, #1622, #1614 and the dpkt hunks of #1605: the octet walk and its link-type stash, IPv6 in IPv4 and IPv4 in IPv6, refusing MPLS and ISL, the tunnel and extension-header walks, the IPv4 header checks and the layer bounds - keep #1690's timestamps, #1594's parser choice, #1621 and #1653 - re-land as guards: a layer dpkt's own lengths misplace (#1646) is re-serialised, not refused with ProtocolError; a Total Length below the IHL keeps the header whole (#1616), and a header the capture cuts is no input; packet2dict and packet2chain take a str payload and a raw frame - delete the modules #1662 and #1655 added; the 1615/1616 module keeps only what its #1616 tests use, as no scapy test imports it now; dpkt's comparisons with the default engine assert constants or the capture's octets, or are dropped - drop pypcapfile's VLAN_TAGS, VLAN_TAG_LEN and IPV6_EXTENSION_HEADERS, which only dpkt imported, and the 1668 module's _Agreement base, which nothing subclasses now - the docs drop the depth claims and name what dpkt leaves out silently The dpkt modules, the shared modules that run dpkt, tests/foundation/engines and the project guards pass on 3.14 and 3.11 (dpkt 1.9.8, pypcapfile 0.12.0).
Third-party engines are best effort (#1687), so the dpkt code that exists only to match the default engine comes out: - revert #1662, #1655, #1624, #1622, #1614 and the dpkt hunks of #1605: the octet walk and its link-type stash, IPv6 in IPv4 and IPv4 in IPv6, refusing MPLS and ISL, the tunnel and extension-header walks, the IPv4 header checks and the layer bounds - keep #1690's timestamps, #1594's parser choice, #1621 and #1653 - re-land as guards: a layer dpkt's own lengths misplace (#1646) is re-serialised, not refused with ProtocolError; a Total Length below the IHL keeps the header whole (#1616), and a header the capture cuts is no input; packet2dict and packet2chain take a str payload and a raw frame - delete the modules #1662 and #1655 added; the 1615/1616 module keeps only what its #1616 tests use, as no scapy test imports it now; dpkt's comparisons with the default engine assert constants or the capture's octets, or are dropped - drop pypcapfile's VLAN_TAGS, VLAN_TAG_LEN and IPV6_EXTENSION_HEADERS, which only dpkt imported, and the 1668 module's _Agreement base, which nothing subclasses now - the docs drop the depth claims and name what dpkt leaves out silently The dpkt modules, the shared modules that run dpkt, tests/foundation/engines and the project guards pass on 3.14 and 3.11 (dpkt 1.9.8, pypcapfile 0.12.0).
Please follow the guide below
You will be asked some questions, please read them carefully and answer honestly
Put an
xinto all the boxes [ ] relevant to your pull request (like that [x])Use Preview tab to see how your pull request will actually look like
Searched for similar pull requests
Followed the coding style (
make pylint,make mypy,make isort)make testpasses, and a test case covers the changeAdded a changelog entry under
docs/source/changelog/and regeneratedCHANGELOG.md, if the change is user-visible — N/A — centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657What is the purpose of your pull request?
Tick the commit type your subject line carries.
fix— corrects a defectfeat— adds a featureperf— changes performance, not behaviourrefactor— changes neither behaviour nor performancetest— tests onlydocs— documentation onlyci— workflows or build toolingrelease— bumps the version or rolls up a distributionchore— anything elseDescription of your pull request and other information
Closes #1606.
Closes #1617.
dpkt.ah.AH, and keeps 43/44/60/135/139 as octets. scapy keeps them all asRaw, except AH, behind which it dissects no TCP. Both now step over these headers withpypcapfile._past_extension_headers, which uses the default engine's own rules and stops atEXTENSION_HEADER_LIMIT. The TCP segment or tunnel behind the headers is decoded from the captured octets. Behind such a header, TCP counts only if the default TCP parser accepts it, as in pypcapfile's_transport.inet6.py:4233in 2.7.0,:4256in 2.8.0) and AH (ipsec.py:111) to IPv4 withoutfrag=0. Its carrier walk now runs top-down and stops at a later IPv4 fragment._decode_ipnow treats any exception as "not decoded", as the engine does for a whole frame (AttributeErrorfrom dpkt'sip6.py:96, fix(dpkt): extract(engine='dpkt') lets a raw AttributeError escape on an IPv6 Fragment header followed by ESP or Destination Options #1351).packet2dictandpacket2chain(verbose mode) now treat only packet-like payloads as layers, so VRRP's''is leaf data and ends the chain. dpkt still reads no TCP in the first frame; that is fix(toolkit): dpkt, scapy and pyshark drop TCP and IPv4 behind some IPv6 extension-header chains #1626.I measured 45 crafted frames for TCP/trace input counts and records. Before the fix, 26 or 27 of them disagreed with default: the cross-review counts 26 on these same frames (dpkt 22, scapy 25) on 3.11 and 3.14, and the author counted 27 on 3.11 (dpkt 1.9.8, scapy 2.8.0). After the fix, 0 of the 45 disagree there or on 3.14 (scapy 2.7.0). pypcapfile still differs only on TCP over IPv6, which it declines by design. This PR drops the #1606
Gaprow intest_ext_chain_limit_agreement_1604_unit.py.