Repository navigation
test(foundation): reassembly, traceflow, engine and registry round trips (#1202) - #1508
Merged
Merged
Conversation
Contributor
|
Coverage: 90.07% (unit tier, Python 3.14,
Per-file detail: the |
1 of 5 tasks
JarryShaw
force-pushed
the
test/1202-foundation-roundtrips
branch
from
October 9, 2026 21:08
cfed47a to
e781499
Compare
This was referenced Oct 9, 2026
Closed
Closed
Closed
JarryShaw
force-pushed
the
test/1202-foundation-roundtrips
branch
2 times, most recently
from
October 9, 2026 21:28
7f9c349 to
4775b2c
Compare
This was referenced Oct 9, 2026
Owner
Author
|
Verdict: GOOD TO GO at
Follow-ups being applied, so this will need one more push:
I corrected the #1515 body: CI hit the harness bug first, so the parity jobs decide that row. |
JarryShaw
force-pushed
the
test/1202-foundation-roundtrips
branch
from
October 9, 2026 21:53
4775b2c to
26c370f
Compare
…ips (#1202) - reassembly: IPv4, IPv6 and TCP payloads cut, reordered, duplicated and overlapped come back as one COMPLETE datagram with the original octets; a late duplicate IP fragment surfaces as a second PARTIAL one (#1507) - traceflow: every flow's PCAP file reads back as exactly its frames, over built captures (PCAP and PCAP-NG, both byte orders and resolutions) and over every sample capture - engines: dpkt, scapy, pyshark, pypcap, pcap_ct and pypcapfile agree with default field for field; an engine that falls back, or a documented refusal, is skipped - registry: override then restore leaves every public registry as shipped - failing cases are recorded per root cause in KNOWN_FAILURES, checked in both directions: #1500-#1505, #1507, #1512-#1515, #1520 Refs #1202
JarryShaw
force-pushed
the
test/1202-foundation-roundtrips
branch
from
October 9, 2026 21:55
26c370f to
f2cb71f
Compare
Owner
Author
|
Verdict: GOOD TO GO at
Non-blocking:
|
JarryShaw
added a commit
that referenced
this pull request
Oct 9, 2026
- toolkit/dpkt, toolkit/scapy: tcp_traceflow reports a FrameRecord, the engine's packet2dict mapping as before, carrying the captured octets and the PCAP record header that only the PCAP trace dumper reads. - engines/dpkt: the PCAP and PCAP-NG readers yield a Timestamp (float) or, for a nanosecond PCAP, a DecimalTimestamp: the value dpkt yields, carrying the exact timestamp, the resolution and the original length. Under a dpkt without the reader internals this relies on, it reads with dpkt's own loop and warns. - engines/scapy: captures are read through PcapReader subclasses that stamp each packet with its PCAP or PCAP-NG interface resolution, and close once. - foundation/traceflow/data: new FrameRecord, a dict with packet and frame_info attributes, picklable with every protocol. - foundation/extraction, interface/misc: dpkt and scapy leave the trace_format='pcap' -> JSON fallback; #1508's Gap(1507) keeps pypcapfile. Traced PCAP files are byte-identical to the default engine's, and JSON, plist and tree output byte-identical to main's.
This was referenced Oct 9, 2026
Closed
JarryShaw
added a commit
that referenced
this pull request
Oct 10, 2026
- toolkit/dpkt, toolkit/scapy: tcp_traceflow reports a FrameRecord, the engine's packet2dict mapping as before, carrying the captured octets, the PCAP record header and the exact time_epoch that only the PCAP trace dumper reads. - engines/dpkt: the PCAP and PCAP-NG readers yield a Timestamp (float) or, for a nanosecond PCAP, a DecimalTimestamp: the value dpkt yields, carrying the exact timestamp, the resolution and the original length. Under a dpkt without the reader internals this relies on, it reads with dpkt's own loop and warns. - engines/scapy: captures are read through PcapReader subclasses that stamp each packet with its PCAP or PCAP-NG interface resolution, and close once. - foundation/traceflow/data: new FrameRecord, a dict with packet, frame_info and time_epoch attributes, picklable with every protocol. - foundation/extraction, interface/misc: dpkt and scapy leave the trace_format='pcap' -> JSON fallback; #1508's Gap(1507) keeps pypcapfile. Traced PCAP files are byte-identical to the default engine's at both trace resolutions, and JSON, plist and tree output byte-identical to main's.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please follow the guide below
You will be asked some questions, please read them carefully and answer honestly
Put an
xinto all the boxes [ ] relevant to your pull request (like that [x])Use Preview tab to see how your pull request will actually look like
Searched for similar pull requests
Followed the coding style (
make pylint,make mypy,make isort)make testpasses, and a test case covers the changeAdded a changelog entry under
docs/source/changelog/and regeneratedCHANGELOG.md, if the change is user-visible — N/A — centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657What is the purpose of your pull request?
Tick the commit type your subject line carries.
fix— corrects a defectfeat— adds a featureperf— changes performance, not behaviourrefactor— changes neither behaviour nor performancetest— tests onlydocs— documentation onlyci— workflows or build toolingrelease— bumps the version or rolls up a distributionchore— anything elseDescription of your pull request and other information
The foundation slice of #1202, test-only (unit modules feed
extractan in-memoryio.BytesIO, which #1509 made possible): reassembly recovers the payload, traceflow output re-reads to exactly its frames, every offline engine agrees withdefault, and override→restore through the 36 publicpcapkit.foundation.registryregistrars is symmetric (class-level registrars without a wrapper are out of scope). Failures sit in per-moduleKNOWN_FAILURES, checked in both directions.tests/foundation/…)reassembly/test_reassembly_payload_roundtrip_unit.pytraceflow/test_traceflow_reread_roundtrip_unit.pytraceflow/test_traceflow_reread_roundtrip_runtime.pyengines/test_engine_agreement_runtime.pyregistry/test_registry_symmetry_unit.pyGaps: #1500, #1501, #1520 (dpkt engine drops PCAP-NG SPBs), #1502, #1503, #1504, #1505, #1507 (dpkt/scapy/pypcapfile JSON trace), #1512 (pypcapfile reads no big-endian PCAP), #1513 (pypcapfile drops TCP over IPv6), #1514 (pyshark
int('True')on TCP flags), #1515 (pyshark counts a journal export block as a frame). The late duplicate IP fragment is asserted as PARTIAL per the #1507 ruling.Ran sequentially under a 200 MiB file cap after
make_samples.py: the five modules,tests/foundation,tests/project,tests/test_tier_guard.py,tests/test_docstring_contract.pyandutil/run_unittest_leg.py foundation, all passing; the engine module also passes in a Python 3.11 venv with Scapy, DPKT, PyShark and PyPCAPFile (PyPCAP would not build: nopcap.h).Refs #1202