Skip to content

fix(protocol): refuse a construction keyword no signature declares (#617) - #640

Merged
JarryShaw merged 1 commit into
mainfrom
fix/617-make-rejects-undeclared-keywords
Sep 22, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/617-make-rejects-undeclared-keywords

Conversation

@JarryShaw

@JarryShaw JarryShaw commented Sep 22, 2026

Copy link
Copy Markdown
Owner

Fixes #617.

⚠️ This is a behaviour change to a public API

Constructing a protocol with a keyword that no signature declares now raises pcapkit.utilities.exceptions.UnsupportedCall instead of silently discarding it.

That will surface latent bugs in code that has been quietly losing a field. That is the point of the change, and it is stated here rather than buried: anyone who has a misspelled keyword in a make/constructor call will get an exception where they previously got wrong octets. It surfaced four such bugs in this repository alone (all fixed here) and three more it deliberately only warns about (below).

Parsing is unaffected. from_data warns rather than raises.

Where the silence was

pcapkit/protocols/protocol.py:284 (pre-fix) — Protocol.pack:

self.__header__ = self.make(**kwargs)

Every one of the 30 make implementations in the tree ends its signature with **kwargs: 'Any' and reads nothing out of it, so a keyword make does not declare is accepted, dropped, and the field it named keeps its default. No warning, no error.

The schema layer has never been so permissive — pcapkit/protocols/schema/schema.py:450, in Schema.__update__, which is Schema.__init__:

if key not in self.__buffer__:
    warn(f'{key!r} is not a valid field name', UnknownFieldWarning)
    continue

That asymmetry between the two halves of the same construction is what this closes.

The silent loss, measured

Reproduced against an immutable git archive snapshot of 6c3d1b0d9 in /tmp, with the #602 shape of examples/generators/options.py's TCP_BASEseq, ack_flag and urgent_pointer, none of which TCP.make declares (it spells them seq_no, ack, urgent):

MEASURED AGAINST: /tmp/i617/before/pcapkit/__init__.py
declared seq = 1   built info.seq = 0
declared ack_flag = True   built info.flags.ack = False
declared urgent_pointer = 7   built info.urgent_pointer = 0
wire bytes = b'\x00P\x1f\x90\x00\x00\x00\x00\x00\x00\x00\x00P\x00\xff\xff\x00\x00\x00\x00'
warnings captured: []

Three fields asked for, three fields lost, warnings: [] under simplefilter('always'). The same construction now:

MEASURED AGAINST: .../worktrees/agent-a5b8f94d363c7e05d/pcapkit/__init__.py
RAISED: UnsupportedCall: TCP: unexpected keyword(s): 'ack_flag', 'seq' (did you mean 'seq_no'?), 'urgent_pointer' (did you mean 'urgent'?)
warnings captured: []

Why a naive fix is wrong, and what was done instead

#617 asked for the design to be decided rather than bolted on, and it was right to. Rejecting on type(self).make's signature alone breaks three legitimate internal patterns — found by surveying every call site, then confirmed by measurement (54 test failures on the naive version):

1. read-only keywords travel through make. ProtocolBase.__post_init__ hands one kwargs dict to both halves:

if file is None:
    _data = self.pack(**kwargs)      # -> self.make(**kwargs)
...
self._info = self.unpack(length, **kwargs)   # -> self.read(length, **kwargs)

So a keyword only read declares still passes through make. HIP is the live case from the issue, and it checks out exactly: HIP.read (hip.py:493) declares extension, HIP.make (hip.py:567) does not, and HIP.__post_init__ (hip.py:682) re-forwards it to both — which examples/generators/options.py's _hip_build depends on. Seven protocols do this with extension, AH/ESP/MH also with version, and Raw with error/alias.

2. HTTP.make is a version dispatcher. It declares only version and forwards everything else to HTTPv1.make or HTTPv2.make depending on that value (http.py:150), so no fixed set of names is correct for it.

3. from_data spreads a machine-generated dict. ProtocolBase.__init__ injects packet=self.packet.payload into every parsed _info, and the default _make_data is data.to_dict(), so from_data hands packet to a make that usually does not declare it — starting with NoPayload, reached for the innermost layer of every packet.

The design

Checked in ProtocolBase.__init__ — the one point every producer passes through — and only when constructing (file is None). The accepted set is the union of every keyword-taking parameter of make, read, pack, unpack, __post_init__ and __init__ anywhere in the MRO, read from inspect.signature and cached per class. That is #617's third option, and it accepts HIP's extension while still rejecting TCP's seq.

Two escapes for shapes a signature cannot express, both opt-in per class via a new ProtocolBase.__keywords__:

  • a set — for a keyword read out of **kwargs by name, as ESP.read does with kwargs.get('packet'). Unioned down the MRO.
  • None — for a dispatcher that cannot enumerate its own keywords. HTTP is the only user. Not inherited, so HTTPv1/HTTPv2 stay checked; inheriting it would have exempted the only two HTTP classes that can be checked, which is what the first attempt did and what a regression test now pins.

packet joins _layer, _protocol, __context__ and __packet__ as out-of-band, because the library injects it rather than a caller passing it.

from_data warns UnknownFieldWarning instead of raising. Its keywords came out of _make_data, not out of anybody's editor, so a mismatch is a disagreement between two of the protocol's own mappings and the caller who meets it cannot fix it.

The exception

UnsupportedCall, not a new class. The library already answers "you passed a keyword I do not accept" with it, in this same message shape, twice: ProtocolBase.__init_subclass__ (protocol.py:761 pre-fix) and Dumper.__init_subclass__ (dumpkit/common.py:142), both as f'{cls.__name__}: unexpected class keyword(s): {unexpected}'. Recorded deliberately in a test: it carries AttributeError rather than the TypeError a reader expecting the stdlib's unexpected keyword argument would reach for, and consistency with the two existing precedents was preferred to adding a second answer to one question.

Latent bugs surfaced

Four fixed here, all #602 residue — each passing seq/ack_flag/urgent_pointer, so each was building segments with sequence number 0 where the mapping read 1:

  • examples/generators/dispatch.py _TCP_BASE
  • tests/protocols/transport/test_tcp_mptcp_{length_arithmetic,subtype,capable_length}_unit.py

(plus test_tcp_mptcp_join_flag_ordering_unit.py's one reproduction call, whose seq=0 had to become seq_no=0.)

Three reported, not fixed_make_data returns a key no signature of the same protocol declares, so from_data has been dropping that field in silence. Each is a defect in its own protocol rather than in this mechanism, and two need a decision about naming rather than a rename:

protocol _make_data returns make declares lost
Frame (misc/pcap/frame.py:433) ts_src ts_sec the frame timestamp
L2TPv2 (link/l2tpv2.py:367) prio priority the priority bit
Header (misc/pcap/header.py:308) magic_number byteorder/bigendian/… the capture byte order

They are now audible as UnknownFieldWarning, and recorded as an expected-failure table in ReconstructionTests.test_the_three_known_make_data_mismatches_are_recorded, so fixing any one of them turns that test red and the entry gets deleted rather than outliving the defect.

Also corrected while here: #617's own note that tests/test_docstring_contract.py "documents this forwarding pattern as intentional and correct" — flagged there as unverified — is only half right. That module explicitly skips every function with **kwargs (:476-481), calls the exclusion "a known cost … not to be mistaken for coverage" (:73), and asserts nothing about make at all. It does not pin the swallowing behaviour, and it passes unchanged here (7 passed, 12 subtests).

Evidence

A test that fails without the fix and passes with it. New tests/protocols/test_construction_keyword_check_unit.py, 17 tests / 30 subtests. Against an immutable git archive snapshot of 6c3d1b0d9 (library side hash-verified unmodified, 19c398194f73…), with only the test file copied in:

MEASURED AGAINST: /tmp/i617/before/pcapkit/__init__.py
FAILED ...::RejectionTests::test_a_near_miss_is_named_in_the_message
FAILED ...::RejectionTests::test_every_keyword_the_signature_declares_is_accepted
FAILED ...::RejectionTests::test_several_unexpected_keywords_are_all_reported
FAILED ...::RejectionTests::test_the_exception_is_the_one_the_library_already_uses
SUBFAILED(keyword='seq') ...::test_the_three_misspellings_of_the_defect_are_each_refused
SUBFAILED(keyword='ack_flag') ...::test_the_three_misspellings_of_the_defect_are_each_refused
SUBFAILED(keyword='urgent_pointer') ...::test_the_three_misspellings_of_the_defect_are_each_refused
FAILED ...::ForwardedKeywordTests::test_a_subclass_inherits_the_declarations_of_its_parents
FAILED ...::ForwardedKeywordTests::test_hip_declares_extension_on_read_and_not_on_make
FAILED ...::ForwardedKeywordTests::test_keywords_declared_by_a_class_attribute_are_accepted
FAILED ...::ScopeTests::test_a_dispatcher_may_decline_the_check
FAILED ...::ScopeTests::test_out_of_band_keywords_are_accepted_while_constructing
FAILED ...::ReconstructionTests::test_from_data_warns_instead_of_raising
FAILED ...::ReconstructionTests::test_the_three_known_make_data_mismatches_are_recorded
FAILED ...::AsymmetryTests::test_the_schema_layer_still_warns_where_construction_now_raises
15 failed, 5 passed, 1 warning in 0.17s
PYTEST EXIT CODE: 1

and on this branch:

MEASURED AGAINST: .../worktrees/agent-a5b8f94d363c7e05d/pcapkit/__init__.py
.................                          [100%]
17 passed, 30 subtests passed in 0.06s
PYTEST EXIT CODE: 0

The 5 that pass on both are the no-false-rejection guards — a correct spelling still builds the right octets, a read-only keyword is still accepted, the parse path still tolerates anything. A rejection test alone would be satisfied by a check that rejects everything.

Coverage does not go backwards. pcapkit/protocols/protocol.py, same scoped test set, before and after:

before   441 stmts   9 miss   142 branch   2 BrPart   98%   224, 760-761, 774-777, 975-978
after    497 stmts   9 miss   170 branch   2 BrPart   98%   463, 1056-1057, 1070-1073, 1271-1274

56 statements added, all covered: the missed-statement count is unchanged at 9 and every one is a pre-existing gap, shifted by the insertions. None falls inside an added line range.

EXPECTED_FAILURES: nothing moves. tests/protocols/test_option_roundtrip_unit.py's table cannot be grepped (it is built with ** unpacking), so it was imported and dumped, along with the round-trip outcome of every case, on both trees. 45 entries, 322 cases, and the diff of the full table is empty — no entry changes state and none was deleted, including the 6-octet UInt32Field pack asymmetry at schema/internet/ipv4.py:368.

Generated fixtures are byte-identical. examples/generators/make_samples.py runs clean on both trees (exit 0, 21 captures), all 27 files in examples/captures/ hash-identical, and the generator log diff is empty. No generator needed fixingexamples/generators/options.py was already corrected by #602/#609, which is precisely why the fix lands quietly there.

Existing tests whose behaviour changes — two, both in the direction the issue intended:

  • test_option_generator_tcp_base_unit.py::test_make_still_has_the_kwargs_that_hid_the_defect asserted the silence (assertEqual([...], []) after TCP(..., no_such_tcp_field=12345)). Its own docstring said a change to make "turns this red rather than passing silently" — it did, and it is now test_an_undeclared_keyword_is_now_refused_rather_than_absorbed, asserting the raise. It keeps the **kwargs assertion, because that is why the check has to live outside make.
  • test_protocol_base_unit.py's dummy protocol read value/next_protocol out of **kwargs and passed layer=/protocol= while constructing. Declared as parameters now, which is the pattern this change asks for. That edit passes on both trees, so it is not behaviour-coupled.

Full suite green, run in three scoped batches (never all at once — the whole suite in one process hit 41.4 GB RSS on this host):

batch result
tests/protocols tests/integration tests/test_docstring_contract.py tests/test_tier_guard.py 777 passed, 2 skipped, 1733 subtests, exit 0
tests/foundation tests/corekit tests/interface tests/toolkit tests/dumpkit 464 passed, 15 skipped, 810 subtests, exit 0
tests/cli tests/const tests/project tests/vendor tests/utilities 274 passed, 849 subtests, exit 0

1515 tests, 3392 subtests, zero failures. Exit codes read from a file rather than a shell pipeline. pcapkit/protocols/protocol.py hashed before and after the first batch (4b9666affba2…, unchanged) to prove the measurement was not taken against a tree that moved underneath it. python util/changelog_md.py --check exits 0.

Blast radius

pcapkit/protocols/protocol.py is the most central module in the tree, so, stated plainly:

  • Parsing is untouched. The check is behind if not parsing:. Dissection keywords — alias, packet, the parse limits — keep flowing as before, because a protocol cannot know which of its ancestors' keywords its parent chose to forward, and nothing was ever lost that way: a dropped parse keyword changes how a packet is read, not what its octets say.
  • Per-class cost is one MRO walk, cached. Not per packet.
  • Construction of anything correct is unchanged, verified by the byte-identical fixtures and the whole suite.
  • The only library file changed besides protocol.py is http.py, which gains __keywords__ = None and a comment. pcapkit/protocols/schema/schema.py is deliberately not changed: Schema.__update__ is the constructor of every schema and runs on the parse path too, so tightening it is a separate change with a far wider radius.

One commit, on top of 0c7f2b7c9.

Review follow-up: the absent-key sentinel is now a typed singleton

Addressing the review comment on pcapkit/protocols/protocol.py:95"prefer using an actual class
instance, None, NotImplemented or custom defined one, like NoValue"
— and the follow-up asking
for a library-wide sweep. _MISSING = object() is gone:

@final
class _AbsentType:
    """Type of :data:`_Absent`, the absent-key sentinel."""

    def __bool__(self) -> 'Literal[False]':
        return False

    def __repr__(self) -> 'str':
        return '<absent>'


_Absent = _AbsentType()

Why a sibling of NoValueType and not NoValue itself, per "maybe not exactly NoValue":

  • It answers a different question. NoValue is documented as "Default value for fields" and as
    the value of FieldBase.default — "was a value given?". This site asks "is this key in the class
    __dict__ at all?". One shared instance would let either site's marker satisfy the other's identity
    test, and NoValue does not stay put: it is returned from SwitchField.pre_process/post_process
    (corekit/fields/misc.py:444, :474), written into the packet context
    (protocols/schema/schema.py:851), and stored on a schema attribute
    (protocols/schema/internet/hopopt.py:709) before being translated back to None at
    protocols/internet/hopopt.py:1053.
  • It would be a new layering edge. protocols/protocol.py imports nothing from corekit/fields/
    — checked before and after this PR; it takes context, module and protochain from corekit and
    no field. There is no import cycle either way, so this is preference rather than constraint, but the
    protocol base not depending on the field system reads as deliberate.

Why not None: it is taken at this very site, which is why a sentinel is needed at all.
ProtocolBase.__keywords__ is Optional[frozenset[str]] and HTTP.__keywords__ = None
(protocols/application/http.py:54) is the opt-out for a dispatcher that cannot enumerate its
keywords, so keywords is None and "the key is absent" are two reachable, different answers. Had
None been available the class would not exist.

Why not NotImplemented: its semantics are defined — a binary operator declining an operand —
and bool(NotImplemented) has warned since 3.9 and is slated to raise. Borrowing it would be a
misuse and would turn a future if not keywords into a runtime error.

Kept module-private. _Absent is read in _declared_keywords and discarded there — never
returned, never cached, never in an error message — so it does not widen the public surface of a PR
already labelled breaking.

Four tests, in SentinelTests. Two pin the marker's shape: it has a type of its own, it is falsy
and @final exactly as NoValueType is (asserted against that class rather than a literal, so the
two cannot drift), its repr is <absent> rather than <object object at 0x…>, and it neither is
nor compares equal to NoValue, None, NotImplemented or Ellipsis. The other two check
behaviour instead, because the mistake this kind of swap makes is an identity comparison against the
wrong instance, which reads and typechecks fine. Mutating the default to a fresh _AbsentType()
fails both behaviour tests with TypeError: '_AbsentType' object is not iterable from
names.update(keywords), while both shape tests pass through that mutation unharmed — which is why
both kinds are present.

run result
SentinelTests, pre-change tree 2 failed, 2 passed, exit 1
SentinelTests, pre-change tree + wrong-instance mutation 2 failed, 2 passed, exit 1 (the other two)
SentinelTests, this tree 4 passed, 7 subtests, exit 0
whole file, this tree 22 passed, 37 subtests, exit 0
+ tests/protocols/test_protocol_base_unit.py 36 passed, 48 subtests, exit 0

Measured against pcapkit.__file__ = …/.claude/worktrees/agent-a40cd115a1e092f53/pcapkit/__init__.py,
with every __editable__* finder stripped from sys.meta_path first and exit codes read from a file.

Nothing else moves. pylint message counts are byte-identical before and after (repo Makefile
flags). mypy reports the same four pre-existing errors with the same codes, line numbers shifted by
exactly the six statements added (742→771, 764→793, 1408→1437, 1751→1780). isort -l100 -ppcapkit --check-only is clean. python util/changelog_md.py --check exits 0. The sentinel's six
statements are all covered by this test file alone — coverage reports nothing missed before line
442 — so the missed-statement count above is unchanged.

The sweep is #661. type('...', (), {})() has zero hits package-wide; nothing under
pcapkit/const/ or pcapkit/vendor/ holds an identity sentinel; and with this PR there is exactly
one bare object() left in the library — _NOT_FOUND at pcapkit/utilities/compat.py:73, inside
the sys.version_info < (3, 8) backport of CPython's functools.cached_property, where upstream uses
the same construct for the same reason. It and _missing in corekit/multidict.py:78-86 (already a
class instance, and reachable through two public pop() signatures and by pickle-by-name) are written
up in #661 to be decided rather than swept, which is also why they are not in this PR: #640 should not
wait on them. No changelog entry is added here — this is an internal marker with no user-visible
effect, and entries for this cycle go to #657.

@JarryShaw
JarryShaw force-pushed the fix/617-make-rejects-undeclared-keywords branch from 58a9f82 to b9220c2 Compare September 22, 2026 06:34
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO — after one required documentation fix, now applied

Cross-review by an independent Sonnet subagent (this PR was raised by an agent, so it was reviewed by a different model), briefed to falsify rather than to bless and run read-only. It worked from its own git archive 6c3d1b0d9 snapshot rather than trusting this PR's numbers or its measurement scripts. Its verdict line, verbatim:

GOOD TO GO — with one required documentation fix (see claim 2/blast-radius below); the core mechanism and the reported #617 fix are sound and independently reproduced.

What it disputed, and what changed as a result

1. A direct .make() call bypasses the check — TRUE, and the docs overclaimed. This was the one real finding. Reproduced by the reviewer and then again by me:

MEASURED AGAINST: .../worktrees/agent-a5b8f94d363c7e05d/pcapkit/__init__.py
object.__new__(IPv4).make(offst=5, protocl=6, payload=b'\xaa'*8)
  -> accepted;  schema.offset: MISSING

offst and protocl are misspellings of offset and protocol, and they are still silently discarded. The check lives in ProtocolBase.__init__, so it covers SomeProtocol(...) and the pack it leads to, but not object.__new__(cls).make(**kwargs) — an idiom used across this suite and by HTTP.make itself to reach its versioned implementation. The original wording in ProtocolBase.make's docstring and in docs/source/ext.rst said flatly that such a keyword "raises ... rather than discarding it", which was false for that path.

Fixed in the amended commit (b9220c2c3): both now scope the claim to construction through the constructor and carry an explicit warning naming the uncovered path and why closing it — interposing on each of the thirty make implementations rather than on the one place their keywords converge — is a larger change than #617. The changelog bullet says the same. The limitation is also pinned by a new test, ScopeTests::test_a_direct_make_call_is_not_checked, so if it is ever closed the test goes red and gets deleted rather than the gap outliving the note.

2. The reviewer's count of the UnsupportedCall precedent was better than mine — corrected. I cited two existing sites using this message shape; there are five: protocol.py:1064, dumpkit/common.py:142, foundation/reassembly/reassembly.py:580, foundation/engines/engine.py:313, foundation/traceflow/traceflow.py:514. Verified by grep. Its fair objection stands and is now recorded in the test that documents the choice: all five reject a class keyword at __init_subclass__ time, which is a narrower thing than a field value passed to a constructor, and UnsupportedCall carries AttributeError where a reader would reach for TypeError — and where an except AttributeError written for duck-typing could swallow it. The reviewer would have added a TypeError subclass. I kept UnsupportedCall for consistency with five precedents rather than adding a sixth spelling, but the trade is real and is now written down rather than implied.

Where I disagree with the reviewer

HTTPv1/HTTPv2 are subclasses of http.HTTP. The review says they "subclass a separate HTTPBase", making the non-inherited None guard "moot". That is wrong, and it matters, because it is the guard that keeps those two classes checked. Measured:

V1 subclasses http.HTTP: True
V2 subclasses http.HTTP: True
V1 MRO: ['...httpv1.HTTP', '...http.HTTP', '...application.Application', '...protocol.ProtocolBase']
_declared_keywords(http.HTTP) is None: True
_declared_keywords(httpv1.HTTP) is None: False
_declared_keywords(httpv2.HTTP) is None: False

They inherit from http.HTTP directly, so without the klass is cls guard they would have inherited __keywords__ = None and gone unchecked — which is exactly what the first attempt at this did, measured before the guard was added. The guard is load-bearing, not decorative, and test_a_dispatcher_may_decline_the_check pins it.

What it independently confirmed

  • The silent loss, re-derived with its own script against its own hash-verified 6c3d1b0d9 snapshot: info.seq=0 for seq=1, zero warnings; raising afterwards. Numbers match this PR's body exactly.
  • Positional smuggling is closedTCP(None, None, 1, True, 7) is a TypeError from __init__'s own signature.
  • The accepted set is wide but not vacuous — it measured 25 names for TCP, 41 for IPv4, 17 each for Frame and PCAPNG, and checked that the extras are legitimate namespace-override parameters rather than a generic leak.
  • No __reconstructing__ leak. It built a real nested Ethernet→IPv4→TCP→NoPayload chain from tcp.pcap and confirmed vars() carries no leftover flag, and reasoned out why no re-entrancy vector exists: each nested layer gets a fresh instance from cls.__new__(cls). It notes one theoretical fragility — a subclass that deletes the attribute itself would make the finally's del raise and mask the real exception.
  • The three _make_data mismatches are real and the list is complete, by two methods of its own: source inspection, and its own AST audit against the live _declared_keywords. Exactly three, with no **-spread keys that could hide a fourth. It also checked the 8 further candidates that inherit _make_data and found all 8 to be abstract bases that can never be instantiated. A live sweep of 1603 frames across all 23 captures fired only Frame/ts_src, 2382 times. It confirmed the retraction of the fourth claim (HTTPv2length) as correct.
  • EXPECTED_FAILURES does not move — imported from both trees with its own script, 45 entries each, empty diff.
  • Coverage does not regress — its own coverage run reproduced 497/9 after and 441/9 before, 56 statements added, and it read the content at all 9 after-tree gaps to confirm each is pre-existing and none is inside the new code. Its line numbers differ from mine by a constant offset, which it flagged as unreconciled and immaterial; the finding is identical.
  • Nothing weakened in the four misspelling fixes — it checked each diff for loosened assertions, deleted tests or bent expected values, and confirmed none of the three mptcp files asserts anything about seq that the 0→1 change could have silently satisfied.
  • It re-ran tests/protocols itself: 655 passed, 1587 subtests, exit 0.

What it could not verify

The 322-case round-trip total (it verified the 45-entry table instead), the "54 failures on the naive version" figure (a claim about an implementation never in this diff), L2TPv2/Header warning on a real capture (it lacked fixtures, and settled it by static analysis instead), and the suites outside tests/protocols. Those last ones are covered here: the full suite is green in three scoped batches — 777 + 464 + 274 passed, 3392 subtests, zero failures, all exit 0.

Its judgement on the two arguable calls

  • Exception type: it would have used a TypeError subclass. Overruled, with the reasoning now recorded in the test.
  • Warn vs. raise on from_data: "correct call ... that's honest design, not a hidden problem."

The amended commit is b9220c2c3. Still one commit, still unmerged and awaiting your review.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Rewritten: changelog entry moved out, rebased onto current main

This branch was force-pushed. Head is now fbbc1375c, one commit on top of
7b0df4a9c (origin/main at the time of the push).

What was dropped: this PR's bullet in docs/source/changelog/1.5.0.rst, and the
CHANGELOG.md regeneration that came with it. Nothing else. The code patch is
byte-identical to what was reviewed — verified by diffing the old and the new
patch restricted to the non-changelog paths, which came back empty. The commit
message is byte-identical too, Fixes #617. included.

Where the entry went: #657, the shared long-lived changelog PR for the 1.5.0
cycle, verbatim — not reworded, reflowed or trimmed. It merges last.

Why: five open PRs each added a bullet at the same anchor in the entry file. Any
one of them merging re-conflicted the other four on the two changelog files, even
though none of their code files overlapped at all. That cycle had already cost this
wave thirteen rebases. With the changelog out, this PR reports MERGEABLE and stays
that way when the others land.

This PR now touches pcapkit/protocols/protocol.py,
pcapkit/protocols/application/http.py, docs/source/ext.rst,
examples/generators/dispatch.py and six test files — 11 files, down from 13.

Heads-up on red CI, which this PR does not cause

Changelog drift and the matrix jobs fail here, and would fail on any branch cut from
current main. main itself is drifted: 375e9d411 (#638) hand-inserted three lines
into the generated CHANGELOG.md instead of running util/changelog_md.py, which left
a duplicated #630 bullet and a misplaced #631 one. python util/changelog_md.py --check
exits 1 on main and on this branch, and the two changelog files here are
byte-identical to main's — so the failure is inherited, not introduced.

The repair is the first commit of #657 and can be cherry-picked ahead of the rest of
that branch to turn main green.

@JarryShaw JarryShaw added the breaking Alters public API or wire output (apply alongside the type label) label Sep 22, 2026
Comment thread pcapkit/protocols/protocol.py Outdated
)

**Behaviour change to a public API.** Building a protocol through its constructor
with a keyword that names nothing now raises `UnsupportedCall` instead of
discarding it.

* Every `make` in the tree ends its signature with `**kwargs` and reads nothing
  out of it, so a misspelled keyword was accepted, dropped, and the field it
  named kept its default -- wrong octets, with nothing said. That is what #602
  cost: `seq=1` where `TCP.make` spells the parameter `seq_no`, and 25 fixture
  frames carried sequence number `0` against an empty `warnings` list. The schema
  layer already warns `UnknownFieldWarning` for a field it does not know; this
  closes the asymmetry from the other end.
* Checked in `ProtocolBase.__init__`, against the union of every keyword-taking
  parameter of `make`, `read`, `pack`, `unpack`, `__post_init__` and `__init__`
  across the MRO -- wider than `make` alone because `__post_init__` hands one
  `**kwargs` to the construction *and* the parse, so `HIP.read`'s `extension`
  legitimately travels through `HIP.make`. Parsing is untouched, and a direct
  `SomeProtocol.make(...)` call is not covered -- documented, and pinned by a test.
* New `ProtocolBase.__keywords__`: a set for a keyword read out of `**kwargs` by
  name, `None` for a dispatcher that cannot enumerate its own. `HTTP` is the one
  user of `None`; it is not inherited, so `HTTPv1`/`HTTPv2` stay checked.
* Reading that attribute needs an absent-versus-`None` marker, and it is
  `_Absent`, an instance of a `@final` `_AbsentType` carrying `__bool__` and
  `__repr__`, rather than a bare `object()`. `None` is unavailable because it is
  the opt-out above, and `NoValue` is not reused: it is documented as the value of
  `FieldBase.default`, and `protocol.py` imports nothing from `corekit.fields`
  today.
* `from_data` warns rather than raises -- its keywords come from `_make_data`,
  not from a caller. That makes three latent defects audible instead of fatal:
  `Frame` returns `ts_src` for `ts_sec`, `L2TPv2` `prio` for `priority`, and
  `Header` a `magic_number` `make` does not take.
* Fixed the four misspellings this surfaced, all #602 residue: `_TCP_BASE` in
  `examples/generators/dispatch.py` and three stale copies under
  `tests/protocols/transport/`.

Fixes #617.

New `tests/protocols/test_construction_keyword_check_unit.py`: 22 tests, 37
subtests, of which 15 fail on `main` for the keyword check and 2 more for the
sentinel. Two further sentinel tests check the behaviour rather than the marker's
shape, and catch the mistake a swap like this actually makes -- comparing against
a second instance of the right type fails them with `TypeError: '_AbsentType'
object is not iterable`. The sentinel's six statements are covered by that file
alone, so `protocol.py` holds 98% coverage with the missed-statement count
unchanged at 9. The 21 generated fixtures are byte-identical and the generator
log is unchanged.
@JarryShaw
JarryShaw force-pushed the fix/617-make-rejects-undeclared-keywords branch from c46f67b to 0a2d79b Compare September 22, 2026 17:15
JarryShaw added a commit that referenced this pull request Sep 22, 2026
The bullet #640 originally carried, moved here verbatim so that #640 touches only
`pcapkit/protocols/protocol.py`, `pcapkit/protocols/application/http.py`,
`docs/source/ext.rst`, `examples/generators/dispatch.py` and its six test files.

Covers: the behaviour change to a public API -- building a protocol through its
constructor with a keyword no signature declares now raises `UnsupportedCall`
instead of discarding it.

55 lines added to the entry file; `CHANGELOG.md` regenerated, not edited.
@JarryShaw

Copy link
Copy Markdown
Owner Author

Cross-review of the sentinel change — GOOD TO GO

The sentinel swap was made by an agent, so it went to an independent reviewer on a different model
(Claude Sonnet)
, briefed to falsify rather than to bless: to try to break each load-bearing claim
with its own evidence, and told that disagreements were the valuable output. It reviewed c46f67b22
read-only in its own throwaway worktree. Its verdict was GOOD TO GO, with two findings I have
since acted on — the head is now 0a2d79b72.

Measured tree, for both of us:
pcapkit.__file__ = …/.claude/worktrees/agent-a40cd115a1e092f53/pcapkit/__init__.py, after stripping
every __editable__* finder from sys.meta_path.

What it disputed, and what I changed

1. The @final assertion was vacuous on Python 3.10 — fixed. My test compared
getattr(_AbsentType, '__final__', None) against the same getattr on NoValueType. The reviewer
established that typing.final only records __final__ from 3.11 on, and then checked the CI
matrix rather than taking my word that 3.10 was out of scope: .github/workflows/unit-tests.yml
runs 3.10 through 3.14 plus 3.15. So on one of five exercised versions the assertion degraded to
None == None and would have passed with neither class decorated. That is a real hole and it was
mine. The two cases are now spelled out, with NoValueType as the probe for which one applies, so
the 3.11+ branch asserts _AbsentType.__final__ is True and the older branch asserts the attribute
is absent rather than accidentally agreeing.

2. Two of the four tests pass on the old code — now said plainly in the docstring. True, and by
design: test_the_absent_marker_never_reaches_the_accepted_names and
test_the_three_answers_stay_distinct_across_the_swap are the evidence that the swap changed
nothing, which a shape assertion cannot provide. But the class docstring implied more than they
deliver, so it now states the split outright — which pair fails on the bare object() (at the
import, making it a rename check with the substance behind the gate), which pair passes on it, what
the second pair does catch (the wrong-instance mutation, TypeError: '_AbsentType' object is not iterable), and that neither pair is sufficient alone. The reviewer independently reproduced the
pre-swap tree and confirmed the 2-fail/2-pass split, and separately tried and failed to construct a
mutation defeating all four (aliasing _AbsentType = NoValueType is caught by the repr assertion;
subclassing it is caught by assertNotIsInstance).

3. It also called one of my justifications wrong, and it is right. I said reusing NoValue
"would be a new layering edge". The reviewer did not argue the point — it added
from pcapkit.corekit.fields.field import NoValue as a real top-level import to a reconstructed
protocol.py, imported it fresh, and it worked, having first traced that nothing under
corekit/fields/ imports pcapkit.protocols at all. So there is no import cycle and layering is
a preference, not a constraint. My review reply above already said as much ("no import cycle either
way, so this is a preference rather than a constraint"), and the code docstring never claimed
otherwise — it rests only on the semantic argument, which the reviewer judged "sound and semantically
sufficient on its own". Recording it here so the weaker half of the argument is not left standing
unchallenged.

What it confirmed, with its own evidence

  • No singleton confusion. It enumerated every candidate — NoValue, multidict._missing,
    compat._NOT_FOUND, None, NotImplemented, Ellipsis — and showed none can reach the
    keywords variable, since the only __keywords__ assignment anywhere in the library is
    http.py:54's None. It went further than asked and checked instance multiplication empirically:
    copy.copy, copy.deepcopy and a pickle round trip each produce a distinct instance, and
    @final does not block subclassing at runtime. All of that is equally true of NoValueType
    already, and unreachable here because _Absent is never copied, pickled, stored or returned — so
    not a gap this swap introduces. I would rather have that written down than assumed.
  • None is genuinely unavailable. __keywords__ defaults to frozenset() at
    protocol.py:392, and None is the opt-out, so a None default would collapse "never touched it"
    and "explicitly opted out" into one answer.
  • The truthiness flip is inert. The only two use sites are identity checks; nothing reads
    keywords for truth, and _declared_keywords's result is tested with is None. It grepped the
    whole file rather than the two changed lines.
  • The sentinel cannot escape. It traced every path out of _declared_keywords and
    _check_construction_keywords, including the difflib.get_close_matches near-miss path and the
    _DECLARED_KEYWORDS cache.
  • Docs and lint. Both new cross-references resolve. No added line exceeds 120 characters; the
    long lines in protocol.py are all outside this PR's hunks.

What it could not verify, stated rather than glossed

A Sphinx build with nitpicky/-W, to see whether the file's self-referential :data:_Absent, `:func:`_declared_keywords and :attr:ProtocolBase.keywords`` refs resolve to links — they
point at private or module-level names protocol.rst does not autodoc. That pattern is inherited
from this PR's already-reviewed main subject rather than introduced by the sentinel, so I have not
changed it, but it is worth knowing. It also declined to run full `mypy`/`pylint`, on the grounds
that this repo's own `lint.yml` header calls them advisory with a ~5900-message baseline; I ran both
scoped instead and they are unchanged before and after.

One procedural note, in the interest of it not looking tidier than it was: the reviewer noticed
uncommitted cosmetic edits appear in the shared worktree partway through its run (mine — three nits
I found while it worked), correctly declined to act on them, and pinned every quote to
git show c46f67b22:<path> instead. Its findings are against the reviewed commit, and those nits
plus the two fixes above are what 0a2d79b72 now carries.

Re-verified after the fixes, on 0a2d79b72: 22 passed, 37 subtests, exit 0 for the test file;
29 passed, 49 subtests, exit 0 with tests/test_docstring_contract.py added;
util/changelog_md.py --check exits 0. Still unpublished work in the sense that matters here —
I have not resolved the review thread, which is yours to close.

JarryShaw added a commit that referenced this pull request Sep 22, 2026
The bullet #667 would otherwise have carried, kept here so that #667 touches only
`pcapkit/corekit/multidict.py` and `tests/corekit/test_multidict.py`.

One bullet, because it is one convention gap in one class: `_Missing` behind
`MultiDict.pop` and `OrderedMultiDict.pop` lacked the `@final` and the falsy
`__bool__` that `NoValueType` in `pcapkit.corekit.fields.field` sets as the
package's convention for a marker of this kind.

The bullet says plainly that no behaviour changes, and says why rather than
asserting it: both `pop()` implementations decide by identity, never by
truthiness, and `pop()` structurally cannot return the marker -- it returns
`default` only on the branch where `default is not _missing`. It also names the
one way the old truthiness was observable, which is what justifies touching it at
all: `inspect.signature(MultiDict.pop).parameters['default'].default` hands the
marker to any caller who asks, and `if default:` on it reported "a default was
supplied" where none had been.

It closes by recording the disposition of the other two sites from the #640
sweep, so the entry is the whole story: site 1 needed nothing, and `_NOT_FOUND`
in `pcapkit.utilities.compat` stays a bare `object()` deliberately, being a
verbatim line of CPython's `functools.cached_property` inside a
`sys.version_info < (3, 8)` branch no supported interpreter reaches. The
reasoning behind that one is on #661, not here.

`:obj:` roles had to come out: `util/changelog_md.py` rejects them with
`ResidualMarkupError`, since its six conversion rules do not cover interpreted
text and `CHANGELOG.md` would carry the role through as literal text. Double
backticks instead, which is what the rest of the entry file uses.

20 lines added to the entry file; `CHANGELOG.md` regenerated with
`util/changelog_md.py`, not edited. `--check` exits 0 and `tests/project/` is
green at 96 passed, 469 subtests.

Committed from a detached HEAD on e55ba36 and pushed to the branch ref, because
`docs/changelog-1.5.0` is checked out in another agent's worktree at a stale
f846523 and could not be taken here. Note e55ba36, not the 69a6e13 I was
given: the branch had already moved on with #665's and #651's entries.

Refs #661
@JarryShaw
JarryShaw merged commit c224298 into main Sep 22, 2026
26 checks passed
@JarryShaw
JarryShaw deleted the fix/617-make-rejects-undeclared-keywords branch September 22, 2026 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Alters public API or wire output (apply alongside the type label) fix Pull requests that fix a defect (fix: subject prefix)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

make silently discards undeclared keywords where schema construction warns, so a misspelling costs data rather than raising

1 participant