Skip to content

feat(foundation): make subclass registration opt-in for Engine, Reassembly, TraceFlow and Dumper (#514) - #547

Merged
JarryShaw merged 2 commits into
mainfrom
feat/514-optin-registration
Sep 20, 2026
Merged

JarryShaw merged 2 commits into
mainfrom
feat/514-optin-registration

Conversation

@JarryShaw

@JarryShaw JarryShaw commented Sep 20, 2026

Copy link
Copy Markdown
Owner

First of the separate PRs agreed on #514. Design and the owner's answers to nine questions are in that thread: design, answers 1-7, answers 6/8/9.

Scope, and what is deliberately not here

Per Q7 — "separate ones so that we can review individually and the size of PRs can be reasonable" — this is the four families whose __init_subclass__ already takes a registry-key keyword: Engine (name=), Reassembly (protocol=), TraceFlow (protocol=) and Dumper (fmt=). Q1 asked for all five including Dumper; Dumper is here.

Protocol is held back deliberately. It has no registry-key keyword today — its key is the class name, derived inside register_protocol — so making it opt-in means inventing a keyword, and what that keyword should mean is the subject of Q3's enum-type inference. It belongs in the registry PR rather than doubling this one.

Also not here, and each its own follow-up: unwinding the ~76 *Base as <Public> alias imports (Q6 — and note the ordering is load-bearing, the guard has to land first, or a built-in inheriting Protocol under today's fallback auto-registers under its own class name); the per-class registry ownership fix; and TransType.L2TP being registered nowhere.

The change

Four __init_subclass__ hooks had the shape

if protocol is None:
    protocol = cast('str', cls.name)      # invent a key...
Extractor.register_reassembly(protocol.lower(), cls)   # ...then register ALWAYS

and now guard instead, which is what EnumSchema.__init_subclass__ has done all along at schema.py:1044. Measured consequence, unchanged built-ins:

engines: ['dpkt', 'pcap_ct', 'pypcap', 'pypcapfile', 'pyshark', 'scapy']
reassembly: ['ipv4', 'ipv6', 'tcp']     traceflow: ['tcp']

In-tree blast radius is zero by construction: the public classes have 0 descendants each (against EngineBase 9, ReassemblyBase 5, TraceFlowBase 2, DumperBase 3, re-measured on this branch), because every built-in inherits the *Base class to decline registration. Nothing the library ships went through the changed code path.

Two silent failures made loud

Both are UnsupportedCall, the in-library exception already used for API misuse of this kind (Transport.register raises it for cls is Transport). No new exception class, which would have meant touching exceptions.__all__ — a public surface, and out of scope here.

  • An unrecognised class keyword. Engine spells the key name= while Reassembly and TraceFlow spell the same idea protocol=, so guessing wrong is the expected mistake. Verified on main: class WrongKeyword(Reassembly, name='deliberately_wrong') registered under 'wrongkeyword' — the wrong keyword swallowed by **kwargs, dropped by the bare super().__init_subclass__(), and the class-name fallback firing. No exception, no warning, no log. With registration now opt-in the same typo would instead skip registration silently, which is quieter still. On Python 3.10 this holds for every keyword except the four that collide with ABCMeta.__new__ — see the 3.10 section below; those are loud either way, just as TypeError rather than UnsupportedCall.
  • Dumper's ext= without fmt=. With registration keyed on fmt, an ext alone has nothing to attach to and would discard the caller's intent.

registry on three metaclasses

Mirrors EnumMeta.registry (schema.py:948-951). It has to live on the metaclass — a property in the class body would be an instance property, so Reassembly.registry would return the property object. Cheap, because EngineMeta and ReassemblyMeta already carry class-level properties (name, module, protocol).

Two honest asymmetries, both documented in the docstrings and the .rst: it is not per-class (every registration lands in the one Extractor table, so any subclass returns the same object), and it is class-only, where EnumSchema carries two registry properties and so also answers on an instance. Dumper gets none — it writes two registries (Extractor.__output__ and TraceFlow.__output__), so a single registry name would be ambiguous, and DumperBase has no pcapkit metaclass to hang it on.

A class statement no longer touches the filesystem

Dumper.__init_subclass__ used to infer fmt from the subclass' kind property. kind is an instance property, so it built one against a tempfile.NamedTemporaryFile while the class statement was still executing. Guarding on fmt removes that path; tempfile is no longer imported. The test asserts NamedTemporaryFile is not called, so the path is gone rather than merely unused.

The final substitute the *Base hierarchy was standing in for

Worth calling out because it answers the concern raised in Q6. The owner's reason for the *Base inheritance was to stop a built-in's own subclasses being registered again — a substitute for a final marker. That works, but it is a property of which base you inherited, so it cannot vary per subclass: a class is either registering-and-closed or non-registering-and-open, never both.

Keying on the keyword gives both at once. Measured on this branch:

class Parent(Engine, name='ParentEngine')   -> registered as 'parentengine'
class Derived(Parent)                       -> registers NOTHING
class DerivedOptIn(Parent, name='Derived')  -> registered as 'derived'

Under the old fallback the middle line registered itself as 'derived'. Pinned as test_registration_is_not_inherited_by_a_subclass.

Separately measured, and the reason nothing needs revisiting urgently: subclassing HTTP, L2TP, IP or any foundation built-in registers nothing today, because all of them descend from a *Base class and have no registering hook in their MRO.

Documentation

docs/source/ext.rst had two worked examples that relied on the fallback and would silently have produced unregistered classes: class MyScapy(Engine['Packet']) with __engine_name__ = 'Scapy' and no name=, and class MyReassembly(Reassembly[...]) with __protocol_name__ and no protocol=. Both now pass the keyword, with a note that the class attribute is not an opt-in — it sets the name a class reports about itself, registered or not.

Also fixed: the name:/protocol: Args: mismatch on Reassembly.__init_subclass__ and TraceFlow.__init_subclass__ (the real parameter is protocol), Reassembly's class docstring which read "Base flow tracing class" — copied from TraceFlow — and Dumper.__init_subclass__'s reference to pcapkit.foundation.extraction.Extraction, a class that does not exist. None of these is in KNOWN_DEFECTS in tests/test_docstring_contract.py, so no known-defect entry is invalidated.

Fails without the fix

Reverting only the four pcapkit/ files (git apply -R of the production half) and keeping the tests:

result
with the fix 14 passed, 4 subtests passed, exit 0 (engine + dumpkit), full affected set 69 passed, exit 0
production reverted 13 failed, 10 passed, 4 subtests passed, exit 1

Every one of the 13 new-or-changed tests fails, across all four families: the opt-in assertion, the unrecognised-keyword rejection, the registry property, the non-inherited registration, and ext-without-fmt.

Read from a file, not a pipe. echo $? > … then cat, because pytest 9.1.1's native subtests print a failing subtest's parent as PASSED and a wrapper's exit code is not pytest's. That is not hypothetical here: the background runner reported "exit code 0" for the run whose own exit file said 1.

Suite, lint, types

CI's unit-tier selection, verbatim from unit-tests.yml:

pytest tests --ignore=tests/integration --ignore-glob='*_runtime.py' --ignore-glob='*_regression.py'
  • pylint, project invocation, four changed files: 25 messages on origin/main → 20 here, and the difference is exactly the five C0415 import-outside-toplevel now carrying the inline # pylint: disable the codebase already uses for deferred imports. Every other message ID has an identical count — E1003 1, R0401 1, R0801 2, W0223 10, W0404 1, W0622 1, W1113 4 — so no new message of any kind, and no new cyclic import from engine.py's new runtime import of pcapkit.utilities.exceptions.
  • mypy, project flags: back to the 2 pre-existing errors and no others. My first attempt annotated registry against the *Base classes and produced three return-value errors, because Extractor.__engine__ and siblings are declared against the public class. The annotations now match what Extractor declares. Worth noting that those declarations are the narrower-than-reality kind — the tables really hold *Base subclasses — which is the same imprecision as the eleven RegistryError strings named in the design comment; correcting it belongs with the alias work.

Python 3.10, and a correction to an earlier claim in this description

The first revision of this PR failed the Python 3.10 and Integration Python 3.10 jobs, and the explanation I gave for the version guards was wrong. Corrected, and now measured on three interpreters rather than assumed:

3.10.21  ABCMeta.__new__(mcls, name, bases, namespace, **kwargs)     <- positional-OR-keyword
3.11.15  ABCMeta.__new__(mcls, name, bases, namespace, /, **kwargs)  <- positional-ONLY
3.14.7   ABCMeta.__new__(mcls, name, bases, namespace, /, **kwargs)

So on 3.10 exactly four class keyword names — mcls, name, bases, namespace — collide with ABCMeta.__new__'s own parameters and raise TypeError before __init_subclass__ is reached. From 3.11 they are positional-only and arrive at the hook normally. It has nothing to do with EngineMeta being Generic: measured, a Generic metaclass takes a non-colliding class keyword perfectly well on 3.10. That earlier claim was mine and it was unfounded.

Which reading: both, and the split matters

  • Test problem, fixed here. Using name= as the "unrecognised keyword" in the Reassembly and TraceFlow tests was a pathological pick — it is one of the four colliding names, so those tests were asserting UnsupportedCall against a TypeError the metaclass raised first. They now use a non-colliding keyword and run on every supported version instead of being skipped. The engine test that uses protocol= has had its skipIf removed for the same reason — verified reaching the guard on 3.10 against this tree.

  • Feature limitation, documented not fixed. Engine's registry keyword is name. So on Python 3.10, class MyEngine(Engine, name='my_engine') — the documented registration path — raises TypeError. Verified against this tree on 3.10.21, not inferred:

    class MyEngine(Engine, name=...) -> TypeError: ABCMeta.__new__() got multiple values for argument 'name'
    class Typo(Engine, protocol=...) -> UnsupportedCall: Typo: unexpected class keyword(s): protocol
    

    This predates the PR, but the PR makes it load-bearing: before, the cls.name fallback registered the engine anyway, so 3.10 worked by accident. Now the keyword is the only class-definition path, so on 3.10 an engine must be registered with Extractor.register_engine, which works on every version. Recorded in the Engine.__init_subclass__ docstring as a Warning:, in the changelog entry, and in the one remaining skipIf's reason.

    Not fixable from __init_subclass__, because the collision happens in the metaclass before the hook runs. Fixing it means either renaming Engine's keyword or intercepting in a metaclass __new__ — both API decisions rather than test hygiene, so I have raised it on Design: adopt the EnumMeta/EnumSchema opt-in registration pattern for Protocol, Engine, Reassembly and TraceFlow #514 rather than deciding it here.

Verified on 3.10 locally this time

I said in the first revision that I could not execute a pre-3.11 interpreter. That was wrong — I had not looked hard enough; python3.10 is present via mise, and a throwaway venv with the four runtime deps runs the suite. Results, exit codes read from files:

3.14.7 3.10.21
affected test files 23 passed, exit 0 22 passed, 1 skipped, exit 0
tests/foundation + tests/dumpkit, CI's ignore globs 220 passed, 12 skipped, exit 0
production reverted vs origin/main 13 failed, exit 1 12 failed, exit 1

Skips went from three to one; the one that remains is the test that must pass name= to make its point.

One process note worth recording, because it nearly produced a false green: my first attempt at the fails-without proof reverted git diff -- pcapkit, which after the rebase captured only the uncommitted docstring edits — the guard being already in the commit. It reported "0 failed", which for a proof that things should fail is the obvious tell. Redone against git diff origin/main -- pcapkit, which is the real delta.

@JarryShaw

Copy link
Copy Markdown
Owner Author

✅ GOOD TO MERGE -- confirmed by AST inspection that all four public classes (Engine, Reassembly, TraceFlow, Dumper) contain nothing but a docstring and __init_subclass__ (zero attributes, methods, or properties beyond that), so the opt-in guard cannot be silently dropping any real behaviour; independently constructed both the with-keyword and without-keyword case for all four families and confirmed correct registration in each direction; and confirmed the inverse wrong fix (registering exactly when the keyword is absent) is caught by the shipped tests at exit code 1.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Detailed review (independent verification, falsify-not-bless)

Head sha reviewed: 0cf8c4f38fbaaa1039146b3170da2a55094c5175.

"No behaviour change beyond registration" -- confirmed exhaustively via AST, not by reading the diff

Wrote a small AST script rather than trusting the docstrings: for each of Engine (pcapkit/foundation/engines/engine.py), Reassembly (pcapkit/foundation/reassembly/reassembly.py), TraceFlow (pcapkit/foundation/traceflow/traceflow.py) and Dumper (pcapkit/dumpkit/common.py), I walked the class body and listed every FunctionDef/AsyncFunctionDef/Assign/AnnAssign node. Result for all four: the class body is exactly ['Expr', 'FunctionDef'] -- the Expr is the docstring, the FunctionDef is __init_subclass__, and nothing else exists at any of the four. This is a stronger and more exhaustive check than reading the diff, since it rules out any attribute or property the diff's context lines might not have shown.

Zero in-tree descendants of the four public classes -- re-derived independently

Walked __subclasses__() recursively after importing every submodule under pcapkit.foundation and pcapkit.dumpkit: Engine 0 descendants (EngineBase 9), Reassembly 0 (ReassemblyBase 5), TraceFlow 0 (TraceFlowBase 2), Dumper 0 (DumperBase 3) -- matches the PR's numbers exactly, confirming the "in-tree blast radius is zero by construction" claim.

Opt-in behaviour, constructed both directions for all four families

For each family, defined a subclass with the registration keyword and one without it in the same session, then diffed the registry before/after:

  • Engine: class WithKeyword(Engine, name='my_engine_opt_in_test') registers 'my_engine_opt_in_test'; class WithoutKeyword(Engine) registers nothing ('withoutkeyword' absent from Extractor.__engine__).
  • Reassembly: same shape, 'my_reassembly_test' registers, 'rwithout' does not appear in Extractor.__reassembly__.
  • TraceFlow: 'my_traceflow_test' registers, 'twithout' does not appear in Extractor.__traceflow__.
  • Dumper: 'my_dumper_test' registers, 'dwithout' does not appear in Extractor.__output__ (the correct attribute name -- Extractor.__dumper__ does not exist, confirmed by reading pcapkit/foundation/extraction.py).

Unrecognised-keyword and ext-without-fmt rejection -- confirmed for real, not just read

  • class BadEngine(Engine, name2='oops') -> UnsupportedCall: BadEngine: unexpected class keyword(s): name2.
  • class BadReassembly(Reassembly, name='oops') (the exact cross-family typo the PR's own comment names as "the expected mistake") -> UnsupportedCall: BadReassembly: unexpected class keyword(s): name.
  • class BadDumper(Dumper, ext='.mine') -> UnsupportedCall: BadDumper: ext='.mine' given without fmt.

Falsification: the inverse wrong fix

Per the brief, I inverted the Engine guard's direction (if name is None: ... register(cls.__name__.lower()) in place of if name is not None: ... register(name.lower())) -- a guard that registers exactly when the keyword is absent, the "safe-looking direction" a careless refactor could produce. Ran tests/foundation/engines/test_engine_base.py: exit code 1, 2 failed / 3 passed, specifically test_engine_subclass_registration_is_opt_in and test_registration_is_not_inherited_by_a_subclass. This confirms the shipped tests genuinely discriminate direction rather than only checking the happy path. Reverted; git diff <head> --stat empty afterward.

Fails-without proof -- reproduced exactly

Reverted all four production files to main, kept all four test files, ran tests/dumpkit/test_common_unit.py tests/foundation/engines/test_engine_base.py tests/foundation/reassembly/test_reassembly_base.py tests/foundation/traceflow/test_traceflow_base.py: exit code 1, 13 failed, 10 passed, 4 subtests passed -- matches the PR's claimed table exactly, and the 13 named failures span all four families (opt-in assertion, unrecognised-keyword rejection, the registry property, non-inherited registration, and ext-without-fmt, one per family as claimed). Restored: exit code 0, 23 passed, 4 subtests passed for the same four files run together (narrower than the PR's own "69 passed" full-affected-set figure, which I did not reproduce -- see below).

pylint -- confirmed exactly, including the exact message-ID accounting

Ran the project's own Makefile:135 invocation over the four changed files:

  • main's versions: 25 real code messages -- C0415 x5, E1003 x1, R0401 x1, R0801 x2, W0223 x10, W0404 x1, W0622 x1, W1113 x4.
  • This PR's versions: 20 real code messages -- the same set minus all 5 C0415, every other ID's count unchanged.

Exactly matches the PR's claim that the only difference is the five now-suppressed C0415 (import-outside-toplevel), with no new message of any kind. (Ignored the CLI-noise messages -- E0013 plugin-load failure, R0022/W0012 deprecated-option warnings -- which are identical in both runs and are pylint-version artifacts of this venv, not code findings.)

mypy -- confirmed exactly

Ran the project's Makefile:138 invocation over the whole package both before and after. The full package reports 111 errors in 37 files either way (unrelated pre-existing noise -- scapy typing, vendor/ftp typing -- untouched by this PR), but filtered to just the four changed files: exactly 2 errors on both main and this PR's head, same messages (dumpkit/common.py: "Unused type: ignore comment"; traceflow.py: "Argument 1 to warn has incompatible type"), only shifted by the line-number offset the diff introduces (160->217, 244->273). Confirms "mypy unchanged at its 2 pre-existing errors" precisely.

Protocol held back -- the stated reason checks out

Read Protocol.__init_subclass__ (pcapkit/protocols/protocol.py:1412) directly: its keywords are schema= and data=, which specify types, not a registration key, and register_protocol() (pcapkit/foundation/registry/protocols.py:136) derives the registry key purely from protocol.__name__.upper() -- there is no keyword anywhere in this path that a guard could key an opt-in on today. This confirms the PR's reasoning is structurally sound rather than a convenient excuse: the other four families already had a natural keyword to guard on (name=, protocol=, protocol=, fmt=); Protocol does not, and inventing one is exactly the Q3 enum-type-inference question the PR defers to the registry-work follow-up. Holding it back reads as the right scope boundary, not an incomplete change.

CI status

Not run. Per standing instruction, verdict on local evidence only.

What remains unverified

  • The PR's "full affected set: 69 passed, exit 0" figure was not reproduced -- I ran only the four directly-changed *_base.py/test_common_unit.py test files (23 passed, 4 subtests, exit 0), not whatever broader set of engine/reassembly/traceflow/dumpkit test files the PR's "69" aggregates.
  • The full unit-tier suite (pytest tests --ignore=tests/integration ...) was not independently re-run in full, given this PR's narrow, already-exhaustively-verified scope and the time already spent on it plus a second PR (fix(corekit): reject a signed= that contradicts a field's fixed sign (#545) #549) awaiting review.
  • The pre-3.11 sys.version_info guard on three new engine tests (needed because EngineMeta is Generic[_T] and class keywords on a subscripted generic need 3.11+) was not independently checked against an actual pre-3.11 interpreter -- I do not have one available either, so this is carried forward as the PR's own stated limitation rather than something I could close.

…embly, TraceFlow and Dumper (#514)

* invert the fallback into a guard in all four `__init_subclass__` hooks: each
  registers if and only if its registry keyword is given (`name=` for Engine,
  `protocol=` for Reassembly and TraceFlow, `fmt=` for Dumper). Previously an
  absent keyword fell back to `cls.name`, so every subclass of the public class
  was registered and declining meant inheriting the `*Base` class under an alias.
* reject unrecognised class keywords with `UnsupportedCall` instead of letting
  `**kwargs` swallow them, and reject Dumper's `ext=` without `fmt=` likewise.
  Both previously registered the class under a key nobody chose, or now would
  silently skip registration.
* add a class-level `registry` property to `EngineMeta`, `ReassemblyMeta` and
  `TraceFlowMeta`, mirroring `EnumMeta.registry`. Dumper gets none: it writes two
  registries, so one `registry` name would be ambiguous.
* a Dumper subclass no longer touches the filesystem while its `class` statement
  runs -- inferring `fmt` from `kind` meant instantiating the class against a
  `NamedTemporaryFile` mid-definition.
* fix the two `ext.rst` examples that relied on the fallback and would otherwise
  have produced unregistered classes, the `name:`/`protocol:` docstring
  mismatches on Reassembly and TraceFlow, and Reassembly's class docstring,
  which read "Base flow tracing class".

Unit tier 3.14: 1083 passed, 8 skipped, 2605 subtests, exit 0 read from a file;
3.10 foundation+dumpkit: 220 passed, 12 skipped, exit 0. pylint
25 messages to 20 with no new message ID; mypy unchanged at 2 pre-existing errors.

Python 3.10 follow-up: `mcls`/`name`/`bases`/`namespace` collide with
ABCMeta.__new__'s positional-or-keyword parameters before 3.11, so the two tests
that used `name=` as their unrecognised keyword asserted against a metaclass
TypeError. They now use a non-colliding keyword and run on every version, with
the colliding case pinned per version rather than skipped; the engine test using
`protocol=` loses its skip entirely. Engine's own keyword is `name`, so on 3.10
an engine must be registered via Extractor.register_engine -- documented in the
docstring and the changelog rather than worked around.
@JarryShaw
JarryShaw force-pushed the feat/514-optin-registration branch from 0cf8c4f to aea52cd Compare September 20, 2026 16:07
@JarryShaw

Copy link
Copy Markdown
Owner Author

✅ GOOD TO MERGE (re-review of head aea52cdee, was 0cf8c4f38) — installed real Python 3.10.21 and 3.11.15 interpreters on this host and ran the actual test file against them, not an inference from CI: on 3.10.21, class Engine(..., name=...) genuinely raises ABCMeta.__new__() got multiple values for argument 'name' exactly as documented, the Extractor.register_engine(...) workaround genuinely registers the class anyway, and tests/foundation/engines/test_engine_base.py runs 4 passed/1 skipped (the one skip being the single test that structurally cannot run on 3.10); on 3.11.15 all 5 pass. Ran the entire tests/foundation/ + tests/dumpkit/ suite under the real 3.10.21 interpreter too: 231 passed, 12 skipped, 0 failed. The removed skipIf (on test_engine_subclass_rejects_unrecognised_keyword, which uses the non-colliding protocol= keyword) was safe to remove and is not the one still needed by test_registration_is_not_inherited_by_a_subclass, which correctly keeps its guard since it is the one test that fundamentally cannot run on 3.10 for Engine.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Detailed re-review (independent verification, falsify-not-bless)

Head sha reviewed: aea52cdee8c1c896cdb7f1fbcf3dfb5bac2e1b83, superseding my earlier ✅ on 0cf8c4f38. This is a re-review, not an inherited verdict, per standing instruction.

Separating the rebase pickup from the actual fix

The full 0cf8c4f38..aea52cdee diff is 22 files / +1977/-56, but scoped to the four files that plausibly carry the real fix (pcapkit/foundation/engines/engine.py, tests/foundation/engines/test_engine_base.py, CHANGELOG.md, docs/source/changelog/1.5.0.rst), diffed against the shared rebase base d656b09f1 rather than the old head: 4 files, +276/-12. Everything else in the large diff is #539's and #542's test files picked up by the rebase, not part of this fix. The actual fix is small, matching the coordinator's expectation.

Within engine.py itself, the only change is a new Warning: block added to __init_subclass__'s docstring -- no executable-code change at all. The behavioural fix lives entirely in the test file: one @unittest.skipIf removed from test_engine_subclass_rejects_unrecognised_keyword, and expanded docstrings on two other tests explaining the real mechanism.

1. Is the documented workaround (Extractor.register_engine) actually true on 3.10?

Verified for real, not by reasoning about it. I found real Python interpreters already installed on this host (/home/jarryx/.local/share/mise/installs/python/3.10/bin/python3.10, version 3.10.21 -- the exact patch version the PR claims to have measured against -- and .../3.11/bin/python3.11, 3.11.15, also matching exactly). Built throwaway venvs for both (pip install aenum dictdumper chardet tbtrim mypy_extensions typing_extensions pytest) and ran the real tests/foundation/engines/test_engine_base.py against the real pcapkit source in this worktree:

  • 3.10.21: 4 passed, 1 skipped, exit 0. The skip is test_registration_is_not_inherited_by_a_subclass -- the one test that unconditionally needs name= at class-creation time for Engine and so cannot run at all on 3.10. test_engine_subclass_rejects_unrecognised_keyword (the test whose skipIf was removed) passes.
  • 3.11.15: 5 passed, 1 warning, exit 0 -- every test runs, including the one skipped on 3.10.

I also wrote a minimal standalone reproduction (copying EngineMeta/Engine's exact structure, swapping the one pcapkit-specific import for a local stub) and ran it directly under all three versions (3.10.21, 3.11.15, and this venv's 3.14.7) to isolate the mechanism from the rest of the test suite:

Python 3.10.21:
CLASS-KEYWORD CONSTRUCTION: raised TypeError: ABCMeta.__new__() got multiple values for argument 'name'
WORKAROUND: register_engine call succeeded: True
NON-COLLIDING KEYWORD: correctly reached __init_subclass__ and raised UnsupportedCall: ...

Python 3.11.15 / 3.14.7:
CLASS-KEYWORD CONSTRUCTION: succeeded, registered as True
WORKAROUND: register_engine call succeeded: True
NON-COLLIDING KEYWORD: correctly reached __init_subclass__ and raised UnsupportedCall: ...

This directly confirms every claim: the collision is real and version-specific exactly as described, the error message matches verbatim, the register_engine-style workaround genuinely works on every version tested including 3.10 (unsurprising once understood -- it is a plain classmethod call that never touches ABCMeta.__new__, since that only runs at class-creation time), and a non-colliding keyword (protocol=) reaches __init_subclass__ cleanly on 3.10 too.

Also ran the entire tests/foundation/ and tests/dumpkit/ directories under the real 3.10.21 interpreter, not just the one file: 231 passed, 12 skipped, 378 subtests passed, exit 0, no failures anywhere.

2. Is the collision list complete?

abc.ABCMeta.__new__'s own positional parameters before 3.11 are exactly mcls, name, bases, namespace -- confirmed by the fact that only name= collides in my empirical test while protocol= does not, which is the direct, checkable consequence of protocol not being one of those four names. fmt= (Dumper's keyword) was not separately tested against the collision, but structurally cannot collide for a different, independent reason: Dumper uses no pcapkit ABCMeta-subclass metaclass at all (confirmed by re-reading the class definition -- class Dumper(DumperBase):, no metaclass= argument), so the whole Generic[_T]-plus-ABCMeta shape that produces the collision for Engine/Reassembly/TraceFlow does not apply to it in the first place. So "protocol= and fmt= do not collide" holds for two different, independently-checkable reasons rather than one, and neither is coincidental.

3. Did removing the skipIf leave anything untested on 3.10?

No -- it increased coverage. Before this fix, test_engine_subclass_rejects_unrecognised_keyword (which passes protocol=, a keyword that never collides) was skipped on 3.10 under the mistaken "class keywords on a subscripted Engine[...] need 3.11+" theory that this fix corrects. Removing the guard makes that test run on 3.10 too, confirmed above (4 passed includes it). The one test still skipped, test_registration_is_not_inherited_by_a_subclass, is skipped because it fundamentally cannot execute on 3.10 -- it needs class Parent(Engine[str], name='ParentEngine') unconditionally, and that construction itself raises TypeError on 3.10 regardless of anything this library does. That is an accurate reflection of a real language-level constraint, not an avoidable gap.

CI status

Per the coordinator, CI has already gone green across the board (6/0/16) since the fix landed, so this re-review is about correctness of the fix rather than "did it pass," per the coordinator's framing. My own local evidence above (including genuine 3.10/3.11 runs, not just this venv's 3.14.7) independently corroborates that verdict.

What remains unverified

  • fmt='s safety was confirmed structurally (no ABCMeta-subclass metaclass on DumperBase) rather than by an equivalent empirical 3.10 reproduction of a Dumper collision attempt -- I judged the structural argument sufficient given it rules out the mechanism entirely rather than merely avoiding it by luck.
  • I did not independently re-run the full unit-tier suite under 3.10 or 3.11 (only tests/foundation/ and tests/dumpkit/), given the scope of this fix is confined to those areas and CI has already gone green on this branch.
  • Cleaned up both throwaway venvs (/tmp/py310venv, /tmp/py311venv) after use; nothing left behind in the worktree.

@JarryShaw
JarryShaw merged commit 956f8d8 into main Sep 20, 2026
24 checks passed
@JarryShaw
JarryShaw deleted the feat/514-optin-registration branch September 20, 2026 18:00
JarryShaw added a commit that referenced this pull request Sep 20, 2026
…works on Python 3.10 (#514)

* `Engine.__init_subclass__`'s keyword is `engine` rather than `name`. `name` is
  one of four class keyword names -- `mcls`, `name`, `bases`, `namespace` --
  that collide with `abc.ABCMeta.__new__`'s own parameters, which are
  positional-or-keyword before Python 3.11 and positional-only from 3.11. So on
  3.10 `class MyEngine(Engine, name='x')` raised `TypeError` from the metaclass
  before the hook ran, making the documented registration path unusable there.
  Those four are the whole collision surface, measured; `engine`, `protocol` and
  `fmt` are all outside it.
* no `name=` alias. A keyword that works on some interpreters and not others is
  the trap being removed, not a compatibility measure.
* the `skipIf(sys.version_info < (3, 11))` on the inheritance test and the inline
  version branch in the opt-in test both come out, since neither needed a guard
  for any reason other than requiring `name=` at class-creation time. The engine
  tests now run in full on every supported version.
* `name=` is an unrecognised keyword from here on, and its exception type is
  version-dependent -- `UnsupportedCall` from 3.11, `TypeError` from the
  metaclass on 3.10 -- so it is pinned per version rather than skipped.
* corrects the #547 changelog entry, which named the old keyword, and
  `docs/source/ext.rst`'s engine example, which showed the crashing form.

Unit tier 3.14: 1104 passed, 8 skipped, 2660 subtests, exit 0 read from a file.
3.10.21: engine + extraction + changelog tests 142 passed, 21 skipped, exit 0.
Reverting the rename fails 3 tests on 3.14 and 2 on 3.10, exit 1 on both. mypy
clean; pylint unchanged at its 3 pre-existing messages for this file.
JarryShaw added a commit that referenced this pull request Sep 20, 2026
…works on Python 3.10 (#514) (#557)

* `Engine.__init_subclass__`'s keyword is `engine` rather than `name`. `name` is
  one of four class keyword names -- `mcls`, `name`, `bases`, `namespace` --
  that collide with `abc.ABCMeta.__new__`'s own parameters, which are
  positional-or-keyword before Python 3.11 and positional-only from 3.11. So on
  3.10 `class MyEngine(Engine, name='x')` raised `TypeError` from the metaclass
  before the hook ran, making the documented registration path unusable there.
  Those four are the whole collision surface, measured; `engine`, `protocol` and
  `fmt` are all outside it.
* no `name=` alias. A keyword that works on some interpreters and not others is
  the trap being removed, not a compatibility measure.
* the `skipIf(sys.version_info < (3, 11))` on the inheritance test and the inline
  version branch in the opt-in test both come out, since neither needed a guard
  for any reason other than requiring `name=` at class-creation time. The engine
  tests now run in full on every supported version.
* `name=` is an unrecognised keyword from here on, and its exception type is
  version-dependent -- `UnsupportedCall` from 3.11, `TypeError` from the
  metaclass on 3.10 -- so it is pinned per version rather than skipped.
* corrects the #547 changelog entry, which named the old keyword, and
  `docs/source/ext.rst`'s engine example, which showed the crashing form.

Unit tier 3.14: 1104 passed, 8 skipped, 2660 subtests, exit 0 read from a file.
3.10.21: engine + extraction + changelog tests 142 passed, 21 skipped, exit 0.
Reverting the rename fails 3 tests on 3.14 and 2 on 3.10, exit 1 on both. mypy
clean; pylint unchanged at its 3 pre-existing messages for this file.
JarryShaw added a commit that referenced this pull request Sep 21, 2026
…or Protocol/ProtocolBase (#514) (#570)

* add a `code=` class keyword to `ProtocolBase`/`Protocol.__init_subclass__`
  that registers a subclass into the next-layer dispatch registry (or
  registries) it names -- `Link.__proto__`, `Internet.__proto__`,
  `TCP.__proto__`, `UDP.__proto__`, `SCTP.__proto__`, `Frame.__proto__` and
  `PCAPNG.__proto__`. Omitting `code` leaves the class unregistered, exactly
  as before this keyword existed: the built-in tables are still populated by
  literal assignment in each layer module, not by this hook, so nothing the
  library ships moves.
* add `register_protocol_code` in `foundation/registry/protocols.py`, backed
  by a small enum-type -> destination table: `EtherType` infers `Link`,
  `TransType` infers `Internet`, `PayloadProtocolIdentifier` infers `SCTP`,
  and `LinkType` infers *both* `Frame` and `PCAPNG` -- mirroring the fan-out
  `register_linktype` already does by hand. A raw `int` (e.g. a TCP/UDP port)
  has no inferable type and must name its destination explicitly via a
  `{destination: key}` mapping; either form may appear in an iterable to
  register one class into several registries from a single declaration. The
  explicit form is accepted even for a key whose type could be inferred.
  Inference refuses rather than guesses: an enum type absent from the table
  raises `RegistryError` instead of silently doing nothing or picking an
  arbitrary registry.
* reject unrecognised class keywords with `UnsupportedCall`, matching the
  guard #547 added to `Engine`/`Reassembly`/`TraceFlow`/`Dumper`.
* mirror the new name into the `__all__` lists of
  `pcapkit.foundation.registry`, `pcapkit.foundation` and `pcapkit.all`,
  which each re-declare every registry function by hand.
* document the keyword in `docs/source/ext.rst`'s "New Protocol" example and
  in `docs/source/changelog/1.5.0.rst` (regenerating `CHANGELOG.md`).

Verified backward compatible by measurement: `Protocol`/`ProtocolBase`
descendant counts (0/43) and the byte contents of all seven `__proto__`
tables are unchanged before and after, since no built-in class passes
`code=`. New unit tests in
`tests/protocols/test_protocol_code_registration_unit.py` (19 cases) proven
to fail without this change (18 of 19 fail on the pre-fix tree, the
remaining one being a before/after invariant that must hold both ways) and
pass with it. Full unit tier: 1279 passed, 17 skipped, 2850 subtests, exit 0.
mypy unchanged at its 4 pre-existing errors.
JarryShaw added a commit that referenced this pull request Sep 21, 2026
…nd register_extractor_engine's keyword (#577)

Two gaps the #514 keyword audit turned up while checking whether an
abandoned local change was still needed. Neither had a test before,
and both turned out to already be correct on main -- just unguarded.

- StreamEOFError's docstring did not say that @prepare always raises
  it with quiet=True (the same end-of-stream convention StructError
  follows via its own eof=True), and nothing pinned that silence.
  Document it, and add a test proving @prepare's StreamEOFError logs
  nothing, with a loud control proving the silence comes from
  quiet=True and not from StreamEOFError having stopped logging
  altogether.
- register_extractor_engine's real keyword is `name`, not `engine`;
  an earlier audit (noted on #557) found it documented the other way
  around. The docstring was already fixed, but the keyword itself was
  never under test, so a future rename could put docstring and
  signature back out of step exactly as quietly as before. Pin both
  directions: `name=` registers, `engine=` raises TypeError.

The rest of the inherited local change -- docstring edits to
Engine/Reassembly/TraceFlow's __init_subclass__ and two tests
asserting that an unrecognised `name=` class keyword is silently
swallowed -- is superseded by #547/#557, which made registration
opt-in and rejects an unrecognised class keyword with
UnsupportedCall instead of swallowing it. Confirmed by rebase
conflict (the Args: text they edited no longer exists) and by
running the swallowed-keyword tests against current main, where they
fail because the keyword is now rejected loudly rather than ignored.

Build: full unit tier (pytest -q --ignore=tests/integration
--ignore-glob='*_runtime.py' --ignore-glob='*_regression.py'):
1123 passed, 5 skipped, 2704 subtests passed, exit 0. Both new tests
proven to fail without their fix.
@JarryShaw JarryShaw added feat Pull requests that add a new capability (feat: subject prefix) breaking Breaks public-facing behaviour or API (apply alongside the type label) labels Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Breaks public-facing behaviour or API (apply alongside the type label) feat Pull requests that add a new capability (feat: subject prefix)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant