Skip to content

fix(ipv4): three defects in the IPv4 option schemas (#552) - #559

Merged
JarryShaw merged 1 commit into
mainfrom
fix/552-ipv4-option-schemas
Sep 20, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/552-ipv4-option-schemas

Conversation

@JarryShaw

@JarryShaw JarryShaw commented Sep 20, 2026

Copy link
Copy Markdown
Owner

Closes #552.

Three defects in the IPv4 option schemas, grouped because they share
pcapkit/protocols/schema/internet/ipv4.py. Every measurement below was taken on
this machine against main at e7e9ba98f, with pcapkit.__file__ asserted to be
this tree.

1. TSOption.post_process laundered a bool into an address

post_process converted the ts_data entries that the timestamp flag makes
addresses with a bare ipaddress.ip_address, which accepts any int below
2**32 — and bool is an int subclass. Measured before the fix:

TSOption(type=TS, length=12, pointer=5, flags={'oflw': 0, 'flag': 1},
         ts_data=[True, 5]).pack()
-> 440c050100000001000000050000000000000000
   data = OrderedMultiDict([(IPv4Address('0.0.0.1'), 5)])

No exception and no warning. Nothing downstream could have caught it: ts_data's
item type is a UInt32Field, which packs True as the 1 it is, so #500's guard
in _IPAddressField.pre_process is never reached. And post_process runs on the
packing path as well as the unpacking one, so the entries it converts are
whatever the caller passed — reachable through public IPv4.make, which takes a
caller-built option schema and packs it.

This is the fifth site of the defect #481, #500, #539 and #540 fixed before it, so
the fix is #539's: route all three conversions in the method through
parse_ip_address. After:

FieldValueError: IPv4: [OptNo 68] invalid timestamp address: must not be a bool,
not True -- pass int(True) if the numeric value is what is wanted

The third conversion — the one that reads the prespecified addresses out of the
option's padding, ipv4.py:308 — is included even though #540 and #552 both judged
it unable to launder a bool, which it cannot: remainder is a PaddingField, so
it holds octets. It is routed through anyway because a bare ipaddress.ip_address
there still raises a plain ValueError for a tail that is not a whole number of
8-octet pairs, which no except BaseError can catch — and because leaving one of
three conversions in one method unguarded is exactly how this became a fifth site.

2. Quick-Start was sized with a hardcoded SchemaField(length=5)

Five is the width of a Quick-Start Request's ttl and nonce alone; the type,
length and flags octets the suboption schema re-declares were unaccounted for.
RFC 4782 §19.1 and §19.2 both give the option as eight octets, and _read_opt_qs
rejects any other length outright. Measured on a well-formed 8-octet option:

1908002adeadbee0 -> SchemaWarning: packet length < 0: -3
                    nonce = 55        (not 933982136)
in a full datagram -> ProtocolError: IPv4: invalid format

The three unconsumed octets were read as a further, fabricated option, which is
what made the enclosing datagram unparseable. Silent corruption on the way to a
misleading failure.

The length now comes from quick_start_option_length(schema), summed from the
fields of the suboption the selector just resolved rather than written as a
literal: the registry is open — QSOption is an EnumSchema, so a caller may
register a further function code — and a literal has to be kept in step by hand
with every field the suboptions declare, which is how the 5 came to be wrong in
the first place. The helper refuses rather than guesses for a field whose width is
not fixed, and both shapes of that are covered by tests: a length callback
(LSROption.route) and a SwitchField, which has none and reports zero until its
own selector resolves it (_QSOption.data).

QuickStartReportOption was also one octet short, missing the Not Used octet
RFC 4782 §19.2 puts where a Request has QS TTL — the diagram in this repo's own
_read_opt_qs docstring shows it, Length=8. It therefore packed seven octets
against the length=8 that _make_opt_qs writes and _read_opt_qs demands, so a
spec-correct Report read off the wire decoded its nonce one octet early. Fixing
only the selector would have baked that seven into the reader, so the missing octet
is declared here as a PaddingField. Both functions now measure 8 and both
round-trip:

1908012adeadbee0  (request)  nonce 933982136, rate 80, no warning, rebuilds identically
19088100deadbee0  (report)   nonce 933982136, rate 80, no warning, rebuilds identically

3. _make_opt_ts passed a keyword the schema does not have

data=ts_list where the field is ts_data; data is the attribute
post_process derives. Schema.__update__ answers an unknown field name with an
UnknownFieldWarning and carries on, so every timestamp was dropped in silence and
ts_data stayed bound to its class-level ListField, which post_process then
tried to iterate — TypeError: 'ListField' object is not iterable. The IPv4
Timestamp option was unbuildable through make.

The TYPE_CHECKING __init__ stub on TSOption is corrected too: it advertised
data: 'list[int]' as a constructor argument, which is what the maker was written
against.

Coverage

Three new tests in tests/protocols/internet/test_ipv4_unit.py, each shown to fail
without its fix by reverting exactly that hunk, reading pytest's exit code from a
file:

fix test exit without it how it failed
1 test_ipv4_timestamp_option_refuses_a_bool_as_an_address 1 AssertionError: FieldValueError not raised on the public IPv4.make path, plus four failing subtests
2, selector length only test_ipv4_quick_start_option_is_eight_octets_wide_on_the_wire 1 both wire subtests fail on the under-read
2, report octet only same 1 AssertionError: 7 != 8
3 test_ipv4_timestamp_option_is_buildable_through_make 1 TypeError: 'ListField' object is not iterable

ipv4-option/TS is deleted from EXPECTED_FAILURES — it round-trips now, and the
table's own rule is that a fixed defect loses its entry rather than keeping a stale
one. The table goes from 55 entries to 54, ipv4-option/TS removed and nothing
added; measured by importing the dict out of each ref's own copy of the module and
taking len(), since the ** comprehensions contribute 30 of the 55 keys and no
grep can see them. ipv4-option/QS stays, because the unrelated func defect at
ipv4.py:1178 fails first; its defect string drops the half this PR fixed. Four
stale file:line references in the neighbouring Quick-Start entries are corrected
against the current tree (hopopt.py:869918, ipv6_opts.py:881921, and both
schema modules' 224255), since a wrong line in that table "reads as
documentation of a defect nobody can find".

Full unit tier: 1107 passed, 8 skipped, 2666 subtests passed, exit 0. Both
changed modules under coverage run -m pytest:
schema/internet/ipv4.py 182 statements / 30 branches, 100% (was 172/24 at
100%, so every added statement and branch is exercised);
internet/ipv4.py 471 statements, 0 missed, one pre-existing partial branch.
mypy clean, isort --check clean, pylint reports nothing new.

Not fixed here, and worth its own issues

  • The identical SchemaField(length=5) is still at
    pcapkit/protocols/schema/internet/hopopt.py:255 and
    .../ipv6_opts.py:255
    , with the same measured nonce of 55. IPv4 option schemas: a fifth bool-laundering site, a hardcoded Quick-Start length, and a dropped ts_data keyword #552 was scoped to
    IPv4's copy; quick_start_option_length is the shape a fix for those wants.
  • _make_opt_qs returns a bare nested schema whose func is set only by
    _QSOption.post_process
    , which runs on the parse path — so IPv4.make with a
    Quick-Start option still raises AttributeError: 'QuickStartRequestOption' object has no attribute 'func', in all three protocols. This is why the QS
    EXPECTED_FAILURES entries remain.
  • Whether UnknownFieldWarning should be fatal during make. Items 1 and 3
    here were silent for exactly that reason, and so is hip.py's cipher= (Two dropped-keyword/wrong-cast defects flagged in review and never filed (hip.py:3533, ipv6_route.py:207) #556).
    Measured by instrumenting Schema.__update__ over all 322 cases of the option
    sweep: 16 distinct (schema, keyword) pairs reach it. Two are defects —
    TSOption.data (this PR) and EncryptedParameter.cipher. The other 14 are seven
    MPTCP nested schemas passing kind and length (seven schemas × two keywords),
    which
    pcapkit/protocols/schema/schema.py:95-104 documents as deliberate: __init__
    forwards **kwargs specifically so those keep reaching __update__ rather than
    raising TypeError. Statically there are 11 such _make_mptcp_* constructors in
    pcapkit/protocols/transport/tcp.py, i.e. 22 keywords. So making it fatal would
    catch all three defects and break 11 call sites that are working as designed —
    worth its own issue, and the issue has to convert those sites first.

- `TSOption.post_process` converted `ts_data` entries to addresses with a bare
  `ipaddress.ip_address`, which takes a `bool` as the `int` it subclasses, so
  `ts_data=[True, 5]` packed and reported `IPv4Address('0.0.0.1')` with no
  exception. It runs on the packing path too, and `IPv4.make` accepts a
  caller-built option schema, so this was reachable from the public API. All
  three conversions now go through `parse_ip_address`, the fifth site of the
  defect #481, #500, #539 and #540 fixed before it.
- `quick_start_data_selector` sized the nested Quick-Start suboption with a
  hardcoded `SchemaField(length=5)` -- the width of a Request's `ttl` and
  `nonce` alone. A well-formed 8-octet option decoded its nonce as 55 rather
  than 933982136 and left three octets to be read as a fabricated option, so
  the datagram failed with `ProtocolError`. The length now comes from
  `quick_start_option_length`, computed from the resolved suboption, and
  `QuickStartReportOption` gains the RFC 4782 section 3.1 `Not Used` octet it
  was missing, which had made it seven octets wide against the `length=8` both
  `_make_opt_qs` and `_read_opt_qs` use.
- `_make_opt_ts` passed `data=` where the schema field is `ts_data`, so every
  timestamp was dropped with an `UnknownFieldWarning` and the Timestamp option
  was unbuildable through `make`. The `TYPE_CHECKING` `__init__` stub that
  advertised `data` is corrected too.

Three new tests, each shown to fail without its fix; `ipv4-option/TS` deleted
from `EXPECTED_FAILURES` now that it round-trips. Full unit tier green, 1107
passed with 2666 subtests; both changed modules at 100% statement and branch
coverage.
@JarryShaw
JarryShaw force-pushed the fix/552-ipv4-option-schemas branch from ecac29c to 1a92f00 Compare September 20, 2026 21:08
@JarryShaw
JarryShaw merged commit 43318f6 into main Sep 20, 2026
25 checks passed
@JarryShaw
JarryShaw deleted the fix/552-ipv4-option-schemas branch September 20, 2026 21:37
@JarryShaw JarryShaw added the fix Pull requests that fix a defect (fix: subject prefix) label Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix Pull requests that fix a defect (fix: subject prefix)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

IPv4 option schemas: a fifth bool-laundering site, a hardcoded Quick-Start length, and a dropped ts_data keyword

1 participant