Repository navigation
corekit: a negative field length only warns in Schema.unpack, instead of raising ProtocolError #805
Description
Activity
- addedbugIssues reporting a defect (set by the bug report template; a default, not an assessment)Issues reporting a defect (set by the bug report template; a default, not an assessment)fixPull requests that fix a defect (fix: subject prefix)Pull requests that fix a defect (fix: subject prefix)breakingBreaks public-facing behaviour or API (apply alongside the type label)Breaks public-facing behaviour or API (apply alongside the type label)
on Sep 25, 2026 Labelled
bug,fix,breaking.breakingis not precautionary — I measured it, and the measurement also shows one of the two fixes proposed here would be wrong.The body offers the raise "in
Schema.unpackand/orFieldBase.length/Field.unpack". Those are different conditions, and only the second is safe.schema.py:898-900warns when the running counterpacket['__length__']goes negative — which happens on inputs that parse perfectly well today:PROVENANCE: /tmp/v801/pcapkit/__init__.py SETTINGS declared 15, buffer 9 (exact header) : PARSED, no warnings SETTINGS declared 15, buffer 13 (4 payload) : PARSED, warnings=['SchemaWarning', 'SchemaWarning']That second case succeeds while warning twice. Convert that site to a raise and it starts rejecting parses that work now — so the naive reading of this issue is a regression, not a fix.
The condition that is actually broken is a resolved field length being negative, which is what reaches
struct.calcsize('-5s').field.py:273is where the template is built;schema.py:857'slength = field.lengthis where the negative value is consumed. Raising there rejects exactly the inputs that crash today and nothing else.So the scope should read: raise on a negative resolved field length; leave the running-counter warning alone, or tighten the counter warning separately with its own evidence. Worth stating in the body before anyone picks this up, because the two sites are four lines apart and the wrong one is the more obvious.
Two corrections to the citations while I was in there — both off by a small amount against
main(477ed00c4), the same class of drift that has bitten twice today:schema.py:894-899→ the warn block is at:898-900, and thepacket['__length__'] -= lengthit follows is at:896.- The blast-radius list is right in substance. Worth adding that
pcapng.pyis the one file on it that another open PR (feat(corekit,schema,utilities)!: enforce @final at runtime on Info and Schema (#778) #788) is also editing, so whoever takes this should check the ordering against that.
Not
blocked— it is actionable now, and it is what would let #802's_guess_versionnarrowing be safe. No agent is on it yet.Correcting myself: this does need
blocked, and my "notblocked— it is actionable now" two comments up was wrong. I reasoned from the issue text instead of checking which files the open PRs hold.Checkable blocker: #788 merged.
gh pr view 788 -R JarryShaw/PyPCAPKit --json state,mergedAt#788 edits both files this fix has to change:
#788 (94416199e): pcapkit/protocols/schema/schema.py +121/-8 pcapkit/protocols/schema/misc/pcapng.py +10/-1 #802, #803, #657: neitherThe raise belongs at
schema.py:857(length = field.length) and/orpcapkit/corekit/fields/field.py:273(where the'-5s'template is built), andpcapng.pyis on this issue's own blast-radius list. #788 rewritesSchema.__new__/__init_subclass__in the same file and hoistssuper().__init_subclass__()in the other, so landing this first means one of the two rewrites silently discards the other's edits — no conflict marker, the later write just wins.#788 is green at
94416199e, awaiting a cross-review verdict, and I have already verified it merges cleanly into currentmainwith its tests passing, so this hold should be short.Not
needs: decision: the scope question I raised above — raise on a resolved field length, leave the running-counter warning alone — I answered with measurement rather than leaving it open, and the evidence is in that comment. Flagging one thing for your eye rather than blocking on it: this isbreakingin shared schema machinery with ten schema modules on the blast-radius list, so if you would rather it stay a warning and have_guess_versionkeep itsstruct.errorsuppression instead, say so and I will close this aswontfixrather than spend a worker on it.- addedblockedDeferred pending another issue or decision; see the last comment for what unblocks itDeferred pending another issue or decision; see the last comment for what unblocks it
on Sep 25, 2026 Unblocked — #788 merged at 21:40:55Z as
ccf5f623b, sopcapkit/protocols/schema/schema.pyandschema/misc/pcapng.pyare free.mainis nowf046b38f8.Dispatching a worker. Re-stating the scope correction from above, because the obvious reading of this issue is a regression rather than a fix:
- Raise on a negative resolved field length — consumed at
schema.py:857(length = field.length), turned into a'-5s'template atpcapkit/corekit/fields/field.py:273. - Do not touch the running-counter warning at
schema.py:898-900. Measured:SETTINGS declared 15, buffer 13parses successfully today while warning twice, so converting that site would reject working input.
One addition from #802's cross-review, which matters for scoping:
read()'sschema.length > lengthguard cannot prevent these escapes, because the crash happens insideSchema.unpackduringunpack()andread()never runs. So this is not aread()fix and must not be written as one.Residual to close, measured at
3d85e56f6: 1136 barestruct.errorescapes via directly-constructedHTTPv2— GOAWAY at buflen 9–16, PUSH_PROMISE 9–12, any over-padded DATA/HEADERS/PUSH_PROMISE. Unchanged from base; theHTTP()route is already clean.- Raise on a negative resolved field length — consumed at
- addedwipWork in flight - a covering PR is open or an agent is actively on itWork in flight - a covering PR is open or an agent is actively on itand removedblockedDeferred pending another issue or decision; see the last comment for what unblocks itDeferred pending another issue or decision; see the last comment for what unblocks it
on Sep 25, 2026 - removedwipWork in flight - a covering PR is open or an agent is actively on itWork in flight - a covering PR is open or an agent is actively on it
on Sep 25, 2026 - added 13 commits that reference this issue
on Sep 26, 2026
Metadata
Metadata
Assignees
Labels
Projects
- StatusShow more project fieldsDone
Split off #799's review. httpv2.py:223's frame-length guard has been fixed
there so a buffer under nine octets, or a declared length exceeding the
buffer, is rejected uniformly before the schema layer runs. That closes the
outer-header truncation class, but not a related one one level deeper:
pkt['__length__']inSchema.unpack(schema.py:894-899) is decremented byeach field's nominal width regardless of how many octets the buffer
actually had, and going negative is only ever a
SchemaWarning— never araise. A field whose own
length=lambda pkt: pkt['__length__']-stylecallback resolves to that negative number builds a struct template like
'-5s', andstruct.calcsizeraises a barestruct.errorfor it: not aProtocolError, not aValueError, uncatchable by ordinary caller code.Reproduction, httpv2 specifically (buffer clears the fixed 9-octet header,
but the frame type's own fixed-width payload fields don't fit in what's
left):
Same shape at other sizes/types: GOAWAY at 9-16 octets (
stream+errorareeight fixed octets alone), PUSH_PROMISE at 9-12, and any PADDED
DATA/HEADERS/PUSH_PROMISE whose
pad_lenexceeds what remains.pkt['__length__']is generic machinery, not an httpv2 particular — atleast these schema modules also key a field's length off it and would need
checking for the same latent crash before any fix lands:
Proposed fix: make a negative resolved field length raise
ProtocolError(or a dedicated subclass) in
Schema.unpackand/orFieldBase.length/Field.unpack, in place of the currentwarn(f'packet length < 0: ...', SchemaWarning, ...). That closes theinner-payload class the same way #799 closed the outer-header one, and lets
HTTP._guess_version's last arm inpcapkit/protocols/application/http.pydrop its
struct.errorsuppression for good. Scoped as its own issuebecause the change is in shared schema/field machinery, not a single
protocol, and needs the blast-radius check above before it lands.