Repository navigation
refactor(corekit): move stream proxy classes such as _PeekableStream into corekit.io #1516
Description
The owner asked in chat for _PeekableStream to move into pcapkit.corekit.io, and for any similar proxy or wrapper classes to move to their proper homes in corekit. #1509 added _PeekableStream.
Known candidates on main 6af87f608:
_PeekableStream, atpcapkit/foundation/extraction.py:190. It addspeek()to a stream that lacks one (fix(extraction): extract(fin=...) with a file object raises AttributeError #1506, fix(extraction): accept a file object without a name as fin (#1506) #1509)._NamedStream, atpcapkit/foundation/engines/pypcapfile.py:34. It gives an unnamed stream a.namefor pypcapfile.pcapkit/corekit/io.pyalready holdsSeekableReader, so the two stream wrappers belong there too.
Scope:
- Inventory every proxy, wrapper or adapter class outside
corekit, and propose acorekithome for each one. Leave the lazy-attribute__getattr__data classes infoundation/reassembly/dataandfoundation/traceflow/datain place unless they are genuinely generic. - Move the classes, update the imports and the docs pages, and keep behaviour byte-identical.
- Add tests that pin each moved class from its new module.
Open question: whether the classes stay underscore-private once they live in corekit. The default is to make them public, like SeekableReader, because they would no longer be file-local.
Activity
JarryShaw commentedon Oct 9, 2026
A worker is on this: inventory first, then move the classes into
corekit.io, then tests. Default taken: the moved classes become public, likeSeekableReader.JarryShaw commentedon Oct 9, 2026
Scope widened by the owner in chat: every helper or proxy class outside
corekitis in scope, not just the stream wrappers. Shared helper functions are tracked in #1519. The worker is extending its inventory to both. This PR moves the stream wrappers plus any class that doesn't touch an in-flight PR's files; the rest follow in separate PRs.JarryShaw commentedon Oct 9, 2026
Inventory (classes). #1521 moves
PeekableStreamandNamedStreamintocorekit.io. Everything else either waits for an in-flight PR or needs a call:PortEnumField×3, differing only in the enumschema/transport/{sctp,tcp,udp}.pycorekit/fields/numbers.py, with aprotoparameter_EnumRegistryprotocols/schema/schema.py:1187corekit.multidictDeferredPacket×2, near-identicalfoundation/{reassembly,traceflow}/data/data.pycorekit.infoclassmixinPCAPNGReaderfoundation/engines/dpkt.py:46foundation/protocols, whichcorekitmust notStay where they are: protocol schema fields, the ESP and MH value types, the
utilities.compatbackports, metaclasses and dumpers.Question for the owner: should the two
DeferredPacketclasses merge into onecorekit.infoclassmixin? Each docstring names the other as its counterpart, and they differ only in what they resolve. My recommendation is to merge them after #1484.JarryShaw commentedon Oct 9, 2026
PortEnumField- good_EnumRegistry- rename asEnumRegistryand maybe put incorekit.enum? or a new module likeregistryor if other better choice (never touchcorekit.multidict, keep it a tailored clean copy from Werkzeug)DeferredPacket- merge, and maybe create a new module for it, likepacketor if other better choicePCAPNGReader- staysJarryShaw commentedon Oct 9, 2026
Recorded owner rulings:
PortEnumField: the three copies become one field incorekit/fields/numbers.py.DeferredPacket: the two copies merge into a new module. I'm takingpcapkit.corekit.packet.PCAPNGReader: stays._EnumRegistry: made public and moved intocorekit.corekit.multidictmust stay a clean Werkzeug copy and is not touched.Workers are starting on
PortEnumFieldandDeferredPacketnow. #1497 has merged, and no open PR touches their files.One question on
_EnumRegistry. It can't be calledEnumRegistryincorekit.enum, because that name is taken.pcapkit.corekit.enum.EnumRegistry(enum.py:436) is the public base of every openpcapkit.constenumeration, andpcapkit.allexports it. What_EnumRegistryactually is: adefaultdictholdingEnumSchemasubclasses by code, which never inserts on a lookup miss.My recommendation is
SchemaRegistryin a newpcapkit.corekit.registrymodule.pcapkit.foundation.registryis a different module, the publicregister_*API, socorekit.registrysits near that name but doesn't clash with it. Okay, or would you prefer another name?JarryShaw commentedon Oct 9, 2026
what exactly does this
_EnumRegistrydo?JarryShaw commentedon Oct 9, 2026
What
_EnumRegistrydoes (protocols/schema/schema.py:1187):It is a
collections.defaultdictsubclass that overrides only__missing__: on a miss it returnsdefault_factory()without storing it.EnumSchemafamily keeps a class-level registry,__enum__, mapping each code to its schema class.EnumSchema.__init_subclass__creates it as_EnumRegistry(cls.__default__)(:1366), and swaps a plaindefaultdictdeclared in a class body for one (:1376).registry[code], e.g.Option.registry[type]. An unknown code then falls back to theUnknown*orUnassigned*schema, by design.defaultdictwould insert that fallback on the first miss. Because the registry lives on the class, one packet with an unrecognised code would then permanently grow a process-wide registry. A later, legitimateregister(code, ...)would also report an overwrite that never happened.So the class is not tied to enums at all. It is a generic "default-on-miss, never insert" dict, used only by the schema registries. The two names that fit are:
SchemaRegistry, named for its role, in a newcorekit.registry. This is my recommendation, since it is only used there.FallbackDict, named for its behaviour, if you would rather keep it generic.JarryShaw commentedon Oct 9, 2026
im thinkin put still in schema since it's only used there, especially it's a subclass of
defaultdict. if we wanna move it tocorekit(maybecollections? to matchdefaultdict), i would consider moving it to one of thecollections.abcsubclasses and implement thedefault_factory()by hand.3 remaining items
refactor(corekit): move the stream proxy classes into corekit.io (#1516)
refactor(corekit): merge the two DeferredPacket classes into corekit.…
refactor(corekit): move the stream proxy classes into corekit.io (#1516)
refactor(corekit): share one PortEnumField across TCP, UDP and SCTP (#…
refactor(corekit): move the stream proxy classes into corekit.io (#1516…
JarryShaw commentedon Oct 10, 2026
Reopened: #1521's
Closeskeyword closed this, but theDeferredPacketslice is still open in #1532. This closes when #1532 merges.refactor(corekit): merge the two DeferredPacket classes into corekit.…
refactor(corekit): merge the two DeferredPacket classes into corekit.…
JarryShaw commentedon Oct 10, 2026
Closing: #1532 has merged (
de72d0154). BothDeferredPacketclasses are nowpcapkit.corekit.packet.DeferredPacket, and #1521 already moved the stream proxies intocorekit.io, so every slice of this issue is onmain.