Repository navigation
fix(toolkit): dpkt and pypcapfile round a PCAP timestamp twice, so a small ts_sec hands over a float 1 ulp off the default engine's #1668
Description
Describe the bug
The default engine hands reassembly and the tracer float(Decimal(ts_sec) + Decimal(frac) / res), which is the double nearest the exact timestamp (pcapkit/protocols/misc/pcap/frame.py:265, :267 → pcapkit/toolkit/pcap.py:234, :298, :344, :383). dpkt on a microsecond PCAP, and pypcapfile on any PCAP, hand over float ts_sec + frac / divisor instead. That rounds twice and gives 1.0036909999999999 for 1.003691. The IPv4, IPv6 (dpkt only), TCP and trace inputs all differ, and so do the trace flow label and its file name (pcapkit/foundation/traceflow/tcp.py:230, :274). Every engine already hands over a float. #1501 fixed the type, not the value.
- dpkt:
pcapkit/foundation/engines/dpkt.py:292builds the float and:297yieldsTimestamp(value, exact, …). The exact value (:296) is read only bypacket2frame(pcapkit/toolkit/dpkt.py:318).float(timestamp)atpcapkit/toolkit/dpkt.py:1108,:1216,:1306,:1364isvalue. The_upstreamfallback does the same sum (dpkt/pcap.py:399). - pypcapfile:
pcapkit/toolkit/pypcapfile.py:216, read at:1196,:1285,:1349. - Not affected: dpkt on nanosecond PCAP (
engines/dpkt.py:294) and PCAP-NG (:182), scapy (scapy/utils.py:1586), and pyshark (pcapkit/toolkit/pyshark.py:452).
Only a small ts_sec trips it. Wall-clock captures are safe, but a capture whose clock counts from boot, or one shifted to the epoch, is not. The generated samples sit at EPOCH = 1500000000.0 (examples/generators/pcap.py:118), so no parity test sees it.
Reproduction
main at 85e4198c8, which is a version bump on edada31dd and touches no engine. Linux, CPython 3.14.8 (dpkt 1.9.8, scapy 2.7.0) and 3.11.15 (dpkt 1.9.8, scapy 2.8.0, pypcapfile 0.12.0, pyshark 0.6, tshark 4.6.9). The captures were built with struct at ts_sec=1 and hold Ethernet/IPv4 (DF clear)/TCP frames and Ethernet/IPv6/Fragment frames. Values were read off ReassemblyBase.__call__ and TraceFlowBase.__call__, and both interpreters agree.
capture, ts_sec=1 |
default | dpkt | scapy | pypcapfile | pyshark (trace only) |
|---|---|---|---|---|---|
| µs PCAP, frac 3691: IPv4, TCP, trace | 1.003691 |
1.0036909999999999 |
1.003691 |
1.0036909999999999 |
1.003691 |
| µs PCAP, frac 3691: trace file | …_80-1.003691.json |
…_80-1.0036909999999999.json |
…-1.003691.json |
…-1.0036909999999999.json |
…-1.003691.json |
| ns PCAP, frac 3691000: IPv4, TCP, trace | 1.003691 |
1.003691 |
1.003691 |
1.0036909999999999 |
1.003691 |
| µs PCAP, 415 fracs: frames off, per input | — | 28 | 0 | 28 | 0 |
| ns PCAP, 415 fracs | — | 0 | 0 | 25 | 0 |
| PCAP-NG µs and ns, 415 fracs | — | 0 | 0 | reads no PCAP-NG | 0 |
How often ts_sec + frac / divisor misses the correctly rounded double:
ts_sec |
µs, all 10⁶ fracs | ns, 10⁶ random fracs |
|---|---|---|
| 0 | 0 | 0 |
| 1 | 166,633 | 166,783 |
| 2 | 83,283 | 83,355 |
| 1000 | 336 | 321 |
| 4095 | 64 | not measured |
| 4096, 8191, 8192, 8193, 16384 | 0 | not measured |
| 86400 | 0 | 3 |
| 2¹⁹−1, 2¹⁹, 2²⁰−1, 2²⁰, 2²¹ | not measured | 0 |
| 10⁹, 1.5×10⁹, 1.7×10⁹, 1.76×10⁹, 2³¹−1, 2³²−1 | 0 | 0 |
| 150 random in [8192, 2³²) | 0 | not measured |
Expected behavior
At every ts_sec, dpkt and pypcapfile hand over the float the default engine hands over. The Timestamp docstring (pcapkit/toolkit/dpkt.py:113) says a stored frame keeps dpkt's own value, so the dpkt fix belongs in the four adapters (float(timestamp.exact)), not in the reader. The _upstream path yields a plain float that has no exact. For pypcapfile, (sec * divisor + frac) / divisor rounds once. That int/int division matched float(Decimal(…)) on 200,000 random samples.
Metadata
Metadata
Assignees
Labels
Projects
- StatusShow more project fieldsDone
Activity
JarryShaw commentedon Oct 13, 2026
Blocked until #1662 (dpkt) and the PR for #1667 (pypcapfile) merge, because both edit the files this fix touches:
pcapkit/toolkit/dpkt.py's adapters andpcapkit/toolkit/pypcapfile.py:216.JarryShaw commentedon Oct 13, 2026
Unblocked: #1662 (dpkt) merged, and #1682, the PR for #1667 (pypcapfile), merged at 21:53Z. No open PR touches
pcapkit/toolkit/dpkt.py,pcapkit/toolkit/pypcapfile.pyorpcapkit/foundation/engines/dpkt.py. A worker is on it.fix(toolkit): dpkt and pypcapfile hand over the correctly rounded tim…