fix(dumpkit): a nameless flag member no longer renders as Type::None [0] (#648) - #670
Conversation
… [0]` (#648) BREAKING CHANGE to dumped output. `make_dumper`'s `object_hook` interpolated `o.name` unguarded, and a `Flag` value composed entirely of undeclared bits has `name is None` rather than a string -- so the literal four characters `None` landed in the name half and `pcapkit.const.tcp.flags.Flags(0)` rendered as `Flags::None [0]` in `json`, `tree`, `text`, `txt`, `plist` and `xml`, out of both `Extractor` and `TraceFlow`. * Three sites, not one, all with character-for-character the same interpolation: the `MultiDict`/`OrderedMultiDict` *key* path, the `addon` branch's `'enum'` key, and the scalar return. They now share one `render_enum` helper, so the guard cannot be applied to two of three. * The fallback is the value's own decimal spelling -- `Flags::0 [0]`, `Flags::8 [8]`. That is what the enumeration libraries already use for an undeclared residue: `Flags(2057).name` is `'ACK|9'`, naming the declared bit and giving the leftovers as one number, so a wholly-undeclared value is the same rendering with no declared bit in front. It also cannot be mistaken for a member name, since a Python identifier may not begin with a digit -- none of the 1867 identifiers under `pcapkit/const` is a bare decimal. `'None'` could be mistaken for one, and `NONE` is a real declared name elsewhere. * Not limited to zero, so the guard is on `name is None` and not on the value. `Flags(1)`, `Flags(8)`, `Flags(9)` and `Flags(65536)` are equally nameless; a fix keyed on zero would have left four of five cases emitting `None`. * Not an `aenum` quirk either -- a stdlib `enum.IntFlag` answers `name is None` identically on CPython 3.14.7, so the guard belongs at the interpolation rather than in a choice of enumeration library. * Five of the seven flag registries in the library are nameless at zero, not one: `Flags` plus the four Mobility Header flag registries. The new test discovers them rather than naming them. * #634 pinned the old string in `test_tcp_udp_unit.py` and said it was "left to its own change"; that assertion and its prose now read `Flags::0 [0]`. A flagless TCP segment is the library's only wire-reachable producer of a nameless member, which makes it the test that notices if the guard goes away. `examples/captures/out.json`, `out.plist` and `out.txt` do not move: regenerating all three from `in.pcap` with and without the change gives byte-identical output, because `render_enum` is the identity on every member that has a name and no committed fixture contains a nameless one. `tests/dumpkit/ tests/foundation/ tests/protocols/transport/test_tcp_udp_unit.py` passes 256 tests and 397 subtests. The new module is 5 tests and 21 subtests and fails on `main` with `'Flags::None [65536]' != 'Flags::65536 [65536]'` among 18 failures; the amended #634 assertion fails there with `'Flags::None [0]' != 'Flags::0 [0]'`. `pcapkit/dumpkit/common.py` holds 100% statement and branch coverage across the change, 71 statements and 32 branches before, 76 and 34 after. One pre-existing failure is unrelated and unchanged by this commit: `test_tcp_runtime.py::…::test_sample_capture_reassembles_every_stream_byte_exactly` raises `FileNotFoundError` for a generated fixture in a tree where `make samples` has not been run, identically with and without the change. Fixes #648
…s in #670 and #671 Two bullets, not one, because the two defects are unrelated: one changes what the dumpers emit, the other only the text of four exception messages. They share a file only by accident of being found in the same pass. The #648 bullet leads with the output change and says so in bold, because that is what a reader upgrading needs to see: a flag value with no declared bits dumped as `Type::None [0]` in all six textual format names, out of both `Extractor` and `TraceFlow`. It then justifies the *replacement* rather than just stating it, since "render it as its decimal value" looks arbitrary until you know the enumeration libraries already spell an undeclared residue that way -- and that a decimal cannot collide with a member name where `None` can, `NONE` being a real declared name elsewhere. Three things a reader would otherwise get wrong are recorded: three sites carried the interpolation and not one, the guard is on `name is None` rather than on zero because the defect never was about zero, and it is not an `aenum` quirk since stdlib `enum.IntFlag` behaves identically. It also corrects the issue on a point of fact. #648 said `Flags` was the only registry nameless at zero; a sweep of all seven finds five, the four Mobility Header flag registries included. And it states that the committed example dumps do not move, which was measured by regenerating all three with and without the change rather than assumed -- a reader of a bullet this emphatic will otherwise wonder whether `examples/captures/` drifted. The #649 bullet says "cosmetic" in its second sentence so nobody reads it as a behavioural change, then gives the one reason it was worth doing at all: it is the text a user sees when an option is rejected. It names all four sites, and the 28-against-4 count in the same file, because that count is what makes the correct form a fact about the module rather than a preference. Neither bullet claims a guard it does not have: #648's third site, the `addon` branch, is not reachable from any registry in the library today, and the bullet does not imply otherwise. `CHANGELOG.md` regenerated with `util/changelog_md.py`, not edited by hand. `--check` exits 0 and `tests/project/` is green at 96 passed, 469 subtests. Each `` literal is kept on one line, since the generator rejects one spanning a line break with `ResidualMarkupError`. Committed from a detached HEAD on d14577d and pushed to the branch ref, because `docs/changelog-1.5.0` is checked out in another agent's worktree and could not be taken here. Note d14577d, not the 69a6e13 I was given: the branch had already moved on with #665's, #651's, #661's and #652/#650's entries. Refs #648 Refs #649
Cross-review verdict: GOOD TO GOIndependent cross-review, run on a different model (Sonnet) from the one that authored the change, in its own worktree, briefed to falsify rather than bless and to treat the issue's own measurements as claims rather than ground truth. It was read-only with respect to this PR: no amend, no push, no commit to the branch. Tree it measured: Per-claim verdicts
Structural checks also passed: one commit, author What it disputed, and what I changedTwo findings, both acted on rather than argued away:
Nothing in A–L was left unverified. |
…idth (#702) `tests/dumpkit/test_nameless_enum_rendering_unit.py` swept the literal `NAMELESS_VALUES = (0, 1, 8, 9, 65536)` against `pcapkit.const.tcp.flags.Flags`, and `65536` is `0x10000` -- one bit past the sixteen-bit field that registry's `_missing_` bounds itself to. The guard is right, so the probe was what had to move: a value a registry is correct to refuse cannot also be a value the dumper is expected to render. `main` has been failing on the `library='aenum'` subtest since #670 introduced the file, and every PR whose Unit Tests job reached it failed for reasons that were not its own. Swept all seven flag registries under `pcapkit/const/` before choosing a direction, since aborting at the first failure hid the rest: * All seven bound `_missing_` to their own field, at four distinct widths -- three bits for `CommandType`, four for `TransportProtocol`, eight for `BindingACKFlag`, `HandoverACKFlag` and `HandoverInitiateFlag`, sixteen for `BindingUpdateFlag` and `Flags`. Exempting the width-guarded registries would therefore have exempted every one of them and left this half of the sweep with nothing in it, which is why the probe is derived per registry instead. * `65536` is out of range for all seven, not only for `Flags`. And `8` and `9` are *named* in `BindingACKFlag` and `TransportProtocol` rather than nameless, so one fixed literal was never right for more than one registry. * For all seven the bound is exactly the smallest all-ones mask covering every declared bit, which `TransportProtocol` already spells for itself as `max(cls.__members__.values()) * 2 - 1`. `_field_mask` derives that, so it needs no table to maintain and cannot drift from the guard. * `_nameless_values` returns the values a registry admits that no member names: no bits at all, each undeclared bit alone, and every undeclared bit at once. The last takes over `9`'s multi-bit role and is the largest nameless value the field holds, which is what the out-of-range literal was reaching for. * `test_no_flag_registry_renders_the_literal_none` sweeps those values rather than `registry(0)` alone. It was broad across registries and one value deep in each, so the four Mobility Header registries only ever saw a single value. * `test_a_value_past_the_field_is_refused_rather_than_rendered` asserts that rejection instead of tripping over it, and pins the derivation itself: the widest in-field value is accepted and the next one up is not, which holds only if the derived mask is the bound each registry declared. No `pcapkit` line changed. The file goes from `1 failed, 5 passed, 16 subtests passed` to `6 passed, 64 subtests passed`, and `tests/dumpkit/` is 15 passed. Coverage rises rather than falls, which deleting the literal on its own would not have managed: `tcp/flags.py:91` was reached only by this file failing on it, and the same `raise` in the other six registries was reached by nothing at all. All seven are now covered -- `ftp/command.py:76`, `mh/binding_ack_flag.py:72`, `mh/binding_update_flag.py:87`, `mh/handover_ack_flag.py:60`, `mh/handover_initiate_flag.py:63`, `reg/apptype.py:65` and `tcp/flags.py:91` -- for six fewer missed statements and six fewer partially-covered branches across those modules. Against the #648 guard reverted the sweep fails 27 subtests and the scalar test 21; against `Flags`' width widened to `0xFFFFFFFF` the new test fails on `value=65536`, which is the trade #702 warns against. Fixes #702
Fixes #648.
The defect
make_dumper'sobject_hookrenders every enumeration member asType::name [value], interpolatingo.namewith no guard. AFlagvalue composed entirely of undeclared bits hasname is Nonerather than a string, so the literal four charactersNonelanded in the name half:The three sites
All three carried character-for-character the same interpolation, and all three are in
pcapkit/dumpkit/common.py::204MultiDict/OrderedMultiDictkey path{'Flags::None [0]': [...]}{'Flags::0 [0]': [...]}:213addonbranch's'enum'key{'enum': 'Sub::None [0]', ...}{'enum': 'Sub::0 [0]', ...}:216'Flags::None [0]''Flags::0 [0]'They now share one
render_enumhelper, so the guard cannot be applied to two of three, and a fourth site cannot be added without it.render_enumis documented under Internal Definitions indocs/source/pcapkit/dumpkit/common.rst, alongsideDumperBase— it is deliberately not added to__all__.What a nameless member renders as, and why
The value's own decimal spelling.
Flags(0)→'Flags::0 [0]',Flags(8)→'Flags::8 [8]'.Three reasons, each measured on this tree under CPython 3.14.7:
Flags(2049).nameis'ACK|1'andFlags(2057).nameis'ACK|9'— the declared bit is named and the leftovers are given as one decimal number. A wholly-undeclared value is that same rendering with no declared bit in front, so the fallback continues an existing convention instead of inventing one.pcapkit/constis a pure decimal.'None'had no such protection, andNONEis a real declared name elsewhere in the library (Integrity(0).name == 'NONE'), so a consumer splitting the rendering on::genuinely could not tell "no flags set" from a member so named.Alternatives considered and rejected:
repr(o)gives'Flags::<Flags: 0> [0]', which is redundant and nests brackets inside the bracketed half; an explicit token (NONE,UNNAMED,<unnamed>) reintroduces exactly the collision problem in (2); and dropping the::namehalf to giveFlags [0]changes the shape every consumer parses, which is a larger break than changing one field's content.Two corrections the issue established, both reproduced
Not limited to value 0. The guard is on
name is None, not on the value, because any all-undeclared value is nameless:A fix keyed on zero would have left four of those five still emitting
None. The test covers all five, so it cannot pass a zero-only fix.Not an
aenumquirk. A stdlibenum.IntFlagbuilt from the same members answersname is Noneidentically on 3.14.7, so the guard belongs at the interpolation rather than in a choice of enumeration library. The test asserts both libraries side by side.One thing the issue undercounted
Five of the seven flag registries are nameless at zero, not one —
pcapkit.const.tcp.flags.Flagsplus the four Mobility Header flag registries (BindingACKFlag,BindingUpdateFlag,HandoverACKFlag,HandoverInitiateFlag). The remaining two (TransportProtocol,CommandType) declare an explicitundefined = 0and were always fine. The new test discovers the registries rather than naming them, so an eighth cannot slip past, and it asserts the nameless count is at least five so it cannot silently stop exercising the fix.Interaction with #634
#634 has merged, so a flagless TCP segment now reaches this path and
connectionbecame a string where it had been the JSON number0. #634's reviewer judged that an improvement because the field is now consistently a string instead of switching type with the flag bits — that judgement is unaffected here. What changes is only the name half of that string:"Flags::None [0]"becomes"Flags::0 [0]". The field stays a string, so the consistency #634 bought is kept; what it gains is that the string no longer claims a member name ofNone. #634's own test comment said theNonewas "left to its own change" — this is that change, and its assertion and prose are updated here.Blame puts all three lines in
dfac5d1767(2023-04-28), three years before #634: pre-existing, not introduced by it.The committed fixtures do not move
examples/captures/out.json,out.plistandout.txtare unchanged, and this was measured rather than assumed. All three were regenerated fromin.pcapwith and without the change and the two sets are byte-identical —render_enumis the identity on every member that has a name, and no committed fixture contains a nameless one (grep '::None \['returns nothing in all three).A side-finding, reported and deliberately not fixed here: the committed fixtures have already drifted from what the current tree produces, independently of this change. That drift is substantial, not cosmetic — an earlier draft of this description called it "a
+00:00timezone suffix and different packet-byte offsets", which undersold it, and the cross-review was right to push back. It also includes whole frames whose hex payloads differ in content rather than offset,protocol -> NILbecomingprotocol -> 5001, andpacket -> NILbecoming real WireGuard-secrets bytes.None of it is caused by this PR — the A/B regeneration above isolates that question and answers it byte-identically — but it means
examples/captures/out.*no longer reflect what the library emits, which is worth its own change and is not one I have made here.Failing, then passing
Both runs used the same harness:
__editable__*stripped fromsys.meta_path, the worktree atsys.path[0],pcapkit.__file__asserted and printed before any other import, and pytest's exit code read from a file rather than a pipeline — the wrapper reportedrc=0while pytest exited1, which is exactly the trap.On
main(fix reverted, tests present):With the fix:
That 15 is the whole of
tests/dumpkit/— the pre-existingtest_common_unit.py(9 tests) plus the new module (5) — and the one amended tcp test. Spelled out because it is not the new module's own figure: alone the new module is 5 passed / 17 subtests, and with the amended tcp test 6 passed / 21 subtests. The cross-review could not reconcile 15 against a module-only run, which is the ambiguity this paragraph removes.Wider scoped run —
tests/dumpkit/ tests/foundation/ tests/protocols/transport/test_tcp_udp_unit.py:That one failure is
test_tcp_runtime.py::TCPReassemblyRuntimeTests::test_sample_capture_reassembles_every_stream_byte_exactly, which raisesFileNotFoundErrorfor a generated fixture in a tree wheremake sampleshas not been run. It fails identically with the fix reverted, so it is pre-existing and unrelated — the fixture-dependent tier, exactly astests/_tiers.pydocuments.The suite was never run whole; every run was scoped.
Coverage
pcapkit/dumpkit/common.py, statement and branch, measured withcoveragedirectly:Coverage does not go backwards: it holds at 100% while the file grows by 5 statements and 2 branches, and the new
name is Noneguard is covered on both arms.Labels —
fix+breakingfixis uncontested. Onbreaking, arguing it both ways as asked:For. This changes user-visible output in all six format names (
json,tree,text,txt,plist,xml) out of bothExtractorandTraceFlow. Since #634 merged it is reachable from a real capture — any flagless segment, an nmap NULL scan being the canonical example — so it is not hypothetical. A consumer that string-matchedFlags::None [0]breaks..github/release.ymlreads PR labels only, so omitting the label is the only way this becomes a silent change to the library's principal output, and the repo's own precedent (#635,fix(pcap)+ aBREAKING CHANGEnote for a public-attribute meaning change) points the same way.Against. The label reads "Alters public API or wire output", and the wire bytes are untouched —
o.valueand the[value]half are unchanged, only the name half moves. Read narrowly, a dump-rendering change is neither the public API nor the wire. And the set of affected inputs is small: no registry other than the five nameless-at-zero flag enumerations can produce it.Applied, because
breakingis additive and the asymmetry is decisive: a false positive costs one honest line in the release notes, while a false negative ships a changed output format with nothing telling anyone. Happy to drop it if the label is meant strictly as "wire".One more thing found and deliberately not fixed
pcapkit.const.pcapng.option_type.OptionType(0)has a string-typed.value('opt_endofopt [0]'), so it already renders with nested brackets —'OptionType::opt_endofopt [opt_endofopt [0]]'. That member is named, sorender_enumtreats it identically to the old code and this PR neither causes nor worsens it. Surfaced by the cross-review; a latent oddity in the rendering format, left to its own change.Cross-review
Dispatched on a different model, briefed to falsify rather than bless; verdict posted as a comment below.