Skip to content

fix(toolkit): dpkt and pypcapfile round a PCAP timestamp twice, so a small ts_sec hands over a float 1 ulp off the default engine's #1668

Closed

Description

@JarryShaw

Describe the bug
The default engine hands reassembly and the tracer float(Decimal(ts_sec) + Decimal(frac) / res), which is the double nearest the exact timestamp (pcapkit/protocols/misc/pcap/frame.py:265, :267 → pcapkit/toolkit/pcap.py:234, :298, :344, :383). dpkt on a microsecond PCAP, and pypcapfile on any PCAP, hand over float ts_sec + frac / divisor instead. That rounds twice and gives 1.0036909999999999 for 1.003691. The IPv4, IPv6 (dpkt only), TCP and trace inputs all differ, and so do the trace flow label and its file name (pcapkit/foundation/traceflow/tcp.py:230, :274). Every engine already hands over a float. #1501 fixed the type, not the value.

  • dpkt: pcapkit/foundation/engines/dpkt.py:292 builds the float and :297 yields Timestamp(value, exact, …). The exact value (:296) is read only by packet2frame (pcapkit/toolkit/dpkt.py:318). float(timestamp) at pcapkit/toolkit/dpkt.py:1108, :1216, :1306, :1364 is value. The _upstream fallback does the same sum (dpkt/pcap.py:399).
  • pypcapfile: pcapkit/toolkit/pypcapfile.py:216, read at :1196, :1285, :1349.
  • Not affected: dpkt on nanosecond PCAP (engines/dpkt.py:294) and PCAP-NG (:182), scapy (scapy/utils.py:1586), and pyshark (pcapkit/toolkit/pyshark.py:452).

Only a small ts_sec trips it. Wall-clock captures are safe, but a capture whose clock counts from boot, or one shifted to the epoch, is not. The generated samples sit at EPOCH = 1500000000.0 (examples/generators/pcap.py:118), so no parity test sees it.

Reproduction
main at 85e4198c8, which is a version bump on edada31dd and touches no engine. Linux, CPython 3.14.8 (dpkt 1.9.8, scapy 2.7.0) and 3.11.15 (dpkt 1.9.8, scapy 2.8.0, pypcapfile 0.12.0, pyshark 0.6, tshark 4.6.9). The captures were built with struct at ts_sec=1 and hold Ethernet/IPv4 (DF clear)/TCP frames and Ethernet/IPv6/Fragment frames. Values were read off ReassemblyBase.__call__ and TraceFlowBase.__call__, and both interpreters agree.

capture, ts_sec=1 default dpkt scapy pypcapfile pyshark (trace only)
µs PCAP, frac 3691: IPv4, TCP, trace 1.003691 1.0036909999999999 1.003691 1.0036909999999999 1.003691
µs PCAP, frac 3691: trace file …_80-1.003691.json …_80-1.0036909999999999.json …-1.003691.json …-1.0036909999999999.json …-1.003691.json
ns PCAP, frac 3691000: IPv4, TCP, trace 1.003691 1.003691 1.003691 1.0036909999999999 1.003691
µs PCAP, 415 fracs: frames off, per input — 28 0 28 0
ns PCAP, 415 fracs — 0 0 25 0
PCAP-NG µs and ns, 415 fracs — 0 0 reads no PCAP-NG 0

How often ts_sec + frac / divisor misses the correctly rounded double:

ts_sec µs, all 10⁶ fracs ns, 10⁶ random fracs
0 0 0
1 166,633 166,783
2 83,283 83,355
1000 336 321
4095 64 not measured
4096, 8191, 8192, 8193, 16384 0 not measured
86400 0 3
2¹⁹−1, 2¹⁹, 2²⁰−1, 2²⁰, 2²¹ not measured 0
10⁹, 1.5×10⁹, 1.7×10⁹, 1.76×10⁹, 2³¹−1, 2³²−1 0 0
150 random in [8192, 2³²) 0 not measured

Expected behavior
At every ts_sec, dpkt and pypcapfile hand over the float the default engine hands over. The Timestamp docstring (pcapkit/toolkit/dpkt.py:113) says a stored frame keeps dpkt's own value, so the dpkt fix belongs in the four adapters (float(timestamp.exact)), not in the reader. The _upstream path yields a plain float that has no exact. For pypcapfile, (sec * divisor + frac) / divisor rounds once. That int/int division matched float(Decimal(…)) on 200,000 random samples.

Activity

  1. added
    fixPull requests that fix a defect (fix: subject prefix)
    blockedDeferred pending another issue or decision; see the last comment for what unblocks it
    on Oct 13, 2026
  2. JarryShaw commented on Oct 13, 2026

    @JarryShaw
    OwnerAuthor

    Blocked until #1662 (dpkt) and the PR for #1667 (pypcapfile) merge, because both edit the files this fix touches: pcapkit/toolkit/dpkt.py's adapters and pcapkit/toolkit/pypcapfile.py:216.

  3. moved this to Blocked in PyPCAPKiton Oct 13, 2026
  4. JarryShaw commented on Oct 13, 2026

    @JarryShaw
    OwnerAuthor

    Unblocked: #1662 (dpkt) merged, and #1682, the PR for #1667 (pypcapfile), merged at 21:53Z. No open PR touches pcapkit/toolkit/dpkt.py, pcapkit/toolkit/pypcapfile.py or pcapkit/foundation/engines/dpkt.py. A worker is on it.

  5. added
    wipWork in flight - a covering PR is open or an agent is actively on it
    and removed
    blockedDeferred pending another issue or decision; see the last comment for what unblocks it
    on Oct 13, 2026
  6. moved this from Blocked to WIP in PyPCAPKiton Oct 13, 2026
  7. removed
    wipWork in flight - a covering PR is open or an agent is actively on it
    on Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    fixPull requests that fix a defect (fix: subject prefix)

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions