Skip to content

fix(generators): make_samples builds fixtures from the shared checkout, not the tree it runs in #1343

Closed

Description

@JarryShaw

examples/generators/make_samples.py builds its fixtures from the shared checkout's pcapkit, not from the tree it is run in, so a worktree's own changes are invisible to the captures it generates.

Mechanism. The venv's editable install hard-codes an absolute path:

.venv/.../__editable___pypcapkit_1_4_1_post2_finder.py
MAPPING = {'pcapkit': '/local/home/jarryx/GitHub/PyPCAPKit/pcapkit'}

make_samples.py lives in examples/generators/, so sys.path[0] is that directory and the tree root is never on sys.path — cwd is irrelevant, since a script's sys.path[0] is the script's own directory. PathFinder finds no pcapkit, the editable finder answers, and the generators import the shared checkout. make samples behaves the same way.

Measured. Same worktree, same generator, same head, same minute; the only change is whether PYTHONPATH points at the tree:

PYTHONPATH pcapkit.__file__ result
set to the tree the worktree 266 capturable case(s) … 0 case(s) left out, options-transport.pcap sha1 e0e869fd09c5, 44 frames
unset the shared checkout 257 capturable case(s) … 7 case(s) left out, sha1 e03f56271cb8, 43 frames

Three fresh worktrees at the same head, run in parallel, produced byte-identical output with PYTHONPATH set, across all 23 captures — and PYTHONHASHSEED=0 vs 1 and a second consecutive run made no difference. The generator is deterministic; the import path is not.

The seven dropped cases here are httpv2-frame/PRIORITY, hip-parameter/HOST_ID, hopopt-option/Quick_Start, ipv4-option/QS, ipv6-opts-option/Quick_Start, tcp-option/Quick_Start_Response and tcp-option/User_Timeout_Option. They fail only on the shared checkout's stale working tree, which lacks dbb30c4a3 (#1138). They span five captures, so options-transport.pcap is not special.

Why it matters beyond a frame count. Every round-trip result measured in a worktree is measured against main's library, so a branch's own fix is invisible to the captures generated there, and "this case still fails" can be a statement about main rather than about the branch. That is the same class of error as a pinned count, and it silently weakens EXPECTED_FAILURES in tests/protocols/test_option_roundtrip_unit.py.

The failure is silent. options.py:generate() records a failing case in unreachable and continues, so a stale library just yields a smaller fixture set. The only signal is the N case(s) left out line.

Fix, tested. Two lines in make_samples.py after ROOT is defined and before any generator loads:

if str(ROOT) not in sys.path:
    sys.path.insert(0, str(ROOT))

PathFinder precedes the appended editable finder in sys.meta_path, so this wins. Verified in a fresh worktree with no PYTHONPATH and no PYTHONSAFEPATH: pcapkit.__file__ resolved inside the tree, 0 case(s) left out, sha1 e0e869fd09c5, from both the tree root and cwd=/.

Two gaps, both deliberate:

  • python examples/generators/options.py standalone — which its own docstring advertises — still regresses, because the insert is only in make_samples.py. Each sibling generator needs the same two lines, or a shared _path.py they all import. Only the make_samples.py half is tested.
  • Making generate() exit non-zero when unreachable is non-empty would have turned this silent 43-frame fixture into a failed make samples. Untested, so proposed rather than asserted.

Found while reconciling a 43-vs-44 frame-count disagreement between #1335's author and its cross-review. Both counts were real, in different trees.

Activity

  1. added
    fixPull requests that fix a defect (fix: subject prefix)
    pendingAccepted and unstarted - nobody is on it and nothing blocks it
    on Oct 7, 2026
  2. added
    wipWork in flight - a covering PR is open or an agent is actively on it
    and removed
    pendingAccepted and unstarted - nobody is on it and nothing blocks it
    on Oct 7, 2026
  3. removed
    wipWork in flight - a covering PR is open or an agent is actively on it
    on Oct 8, 2026
  4. added this to the PyPCAPKit v1.5 milestone on Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    fixPull requests that fix a defect (fix: subject prefix)

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions