fix(tcp): give MPTCP real kind/length fields, repairing pack and parse - #565
Conversation
9e93c77 to
45d37ad
Compare
|
❌ NEEDS CHANGES Cross-model review (Opus 5; the PR was authored on Sonnet), head The code change is correct, and it fixes considerably more than this PR claims. That is the headline finding and I proved it independently: on Two changes needed, both about what this PR documents and tests rather than what it does:
The amend from I did not treat #566's and #567's absence as a finding — deferring them was right. But one finding on #567 itself, worth acting on while it is fresh rather than a change to this PR: its description says the maker's Everything I ran is green: transport tier 82 passed / 73 subtests (exit 0, matching your numbers exactly), roundtrip 6 passed / 358 subtests (exit 0), changelog drift exit 0, and all five new tests fail with the documented |
Detailed cross-review — #565 @
|
| option | bytes | main @ 8cfd6ab01 |
this branch |
|---|---|---|---|
| IPv4, no port | 1e08340101020304… (1e083401c0000201) |
FieldError: TCP: [OptNo 30] 3 invalid IP version |
parses: subtype 3, length 8, addr_id 1, port None |
| IPv4, with port | 1e0a3401c00002011f90 |
FieldError: … invalid IP version |
parses: length 10, port 8080 |
| IPv6, no port | 1e14360120010db8…01 |
FieldError: … invalid IP version |
parses: length 20, port None |
| IPv6, with port | 1e16360120010db8…011f90 |
FieldError: … invalid IP version |
parses: length 22, port 8080 |
The main failure is exactly the predicted misalignment: MPTCPAddAddress.test read the kind octet 0x1e, so version came out as 0x1e & 0x0f == 14, and mptcp_add_address_selector rejects anything that is not 4 or 6.
And it is not only ADD_ADDR. A hand-built MP_PRIO (1e045109 — subtype 5, backup set, addr_id 9, length 4):
main: RAISED builtins.KeyError: 'length'
this branch: subtype = <MPTCPOption.MP_PRIO: 5>, length = 4, backup = True, addr_id = 9
All four values exactly as constructed. So this PR repairs MPTCP parsing of real wire bytes, not just construction — the change is strictly a correction of an existing off-by-two, not a wire-format shift. That is a considerably stronger result than the PR title and summary claim, and it is the basis of my requests 1 and 2.
One accuracy correction. The new schema comment says unpacking "silently misread the kind octet as the subtype/flags octet". In both subtypes I measured it does not fail silently — it raises (FieldError for ADD_ADDR, KeyError for MP_PRIO). Silent misreading may well be what happens for subtypes with no conditional field and no validation (MP_FAIL, MP_FASTCLOSE, REMOVE_ADDR), which I did not test; but as written the comment understates ADD_ADDR and MP_PRIO, where the old behaviour was a hard error. Worth a word change, since "silently" is doing load-bearing work in the reader's sense of severity.
2. It did not make UnknownFieldWarning fatal — confirmed
I checked this specifically, since making it fatal would have broken the ~11 _make_mptcp_* call sites that rely on **kwargs forwarding, which pcapkit/protocols/schema/schema.py:95-105 documents as deliberate. schema.py is not in the diff at all. The fix gives the keywords somewhere to land rather than making their absence an error, which is the right shape.
3. Fails-without-the-fix — measured
Reverted pcapkit/protocols/schema/transport/tcp.py to its main content, keeping the PR's tests, then restored from a byte-identical copy and confirmed git status clean.
5 failed, 3 warnings in 0.72s
exit code (from file): 1
All five new tests fail, on exactly the documented cause:
E AssertionError: unexpected UnknownFieldWarning(s):
["'kind' is not a valid field name", "'length' is not a valid field name"]
Your "verified each fails … on the pre-fix code" claim is corroborated.
4. The tests assert packed bytes, as they must
Checked because construction succeeds and only pack() fails, so a construction-only test would pass on broken code. These do not: every case calls schema.pack() and asserts exact octets, for both the port-present and port-absent branches on both address families.
IPv4 no port 1e08340101020304 (8 octets, length octet 8)
IPv4 with port 1e0a34010102030401bb (10 octets, port 0x01bb)
IPv6 no port … (20 octets, length octet 20)
IPv6 with port … (22 octets, port 0x01bb)
I checked those byte strings against RFC 8684 §3.4 by hand: 1e kind 30, 08/0a length, 34 = subtype 3 << 4 | version 4, 01 addr_id, then the address, then the port. Correct.
The gap is the other direction (request 1): nothing here constructs a TCP from bytes. test_opt_argument_round_trips_through_the_maker is maker→maker — it passes a previously-parsed opt object back into the maker — not bytes→parse, so despite the name it does not cover the unpack repair.
5. The EXPECTED_FAILURES re-points are right, and the citations are exact
I checked each line number rather than trusting them:
pcapkit/protocols/schema/transport/tcp.py:650is exactlysubtype: 'Enum_MPTCPOption', inside theif TYPE_CHECKING:block. Correct — and it names the cause rather than the raise site, matching the convention in neighbouring entries.pcapkit/protocols/transport/tcp.py:2669is exactlylength=20 if rkey is None else 32,. Correct.- Fragments are per-class —
"'MPTCPCapable' object has no attribute 'subtype'","'MPTCPAddAddress' object has no attribute 'subtype'", and so on. That is the right call: a shared"no attribute 'subtype'"would have matched all seven cases and pinned none of them, which is precisely whatGap.fragment's docstring warns against. Each entry now pins its own case. - Re-pointing rather than deleting is correct, since none of the seven round-trips yet.
tcp-mptcp/MP_JOINcorrectly left alone onno attribute '_flags'.- Issues MPTCP.subtype is declared only under TYPE_CHECKING, so every subtype built via TCP's convenience constructor raises AttributeError #566 and TCP._make_mptcp_capable writes length 20/32 where RFC 8684 gives 12/20, and now packs wrong bytes rather than crashing #567 both exist and are open, with accurate titles.
pytest tests/protocols/test_option_roundtrip_unit.py → exit 0, 6 passed, 358 subtests.
6. #567 is scoped too narrowly — the schema's own predicate is also wrong
This is request 3, and it is the finding I would most like acted on while the context is fresh, because #567 as written will produce a fix that is still broken.
MPTCPCapable.rkey is gated on
rkey: 'int' = ConditionalField(
UInt64Field(),
lambda pkt: pkt['length'] != 32,
)— include the receiver key for any length except 32. Meanwhile the maker writes length = 20 if rkey is None else 32, i.e. it signals "receiver key present" with 32. Those two are in direct contradiction: the value the maker uses to mean "rkey present" is the one value the schema reads as "omit rkey".
Measured on this branch, packing MPTCPCapable directly with skey=0x0102030405060708:
| rkey supplied? | declared length |
actual packed | result |
|---|---|---|---|
| yes | 32 (what the maker writes) | 12 octets | 1e2001010102030405060708 — receiver key silently dropped, length octet claims 32 |
| yes | 20 (RFC 8684 §3.1) | 20 octets | 1e14010101020304 05060708 deadbeefcafef00d — correct |
| no | 20 (what the maker writes) | 20 octets | 1e14…0000000000000000 — spurious all-zero receiver key |
| no | 12 (RFC 8684 §3.1) | 20 octets | still emits the zero receiver key; the schema cannot express a 12-octet MP_CAPABLE at all |
So the last row is the important one: correcting only the maker to 12/20, as #567 describes, leaves the no-rkey case emitting a 20-octet option with a phantom zero key. The != 32 predicate has to change too — presumably to something that distinguishes 12 from 20. RFC 8684 §3.1 gives MP_CAPABLE as kind(1) + length(1) + subtype/version(1) + flags(1) + sender key(8) = 12 octets, plus receiver key(8) = 20.
None of this is #565's business and deferring it was right. I am only asking that #567's description be corrected so the follow-up is complete.
7. Other checks
- Changelog generated, not hand-matched:
python util/changelog_md.py --check→ exit 0, "in step withdocs/source/changelog/1.5.0.rst". (Content gap is request 2; the mechanics are fine.) pytest tests/protocols/transport→ exit 0, 82 passed, 73 subtests — matching your claimed numbers exactly.- Full tier
pytest tests(no coverage, per the host memory constraint) → exit 0 read from a file,1265 passed, 17 skipped, 2850 subtests passed in 1133.58s. Your "0 failed" is corroborated, and the count is self-consistent: I measured 1261 on fix(corekit): stop a malformed TCP SACK's exception type depending on sys.modules state (#525) #562's branch, which ismain's 1260 plus its own one new test, somain+ this PR's five new tests = 1265. Fixtures generated first viaexamples/generators/make_samples.py(exit 0), since a fresh worktree has none. - The amend
9e93c7772→45d37ad81touches onlytests/protocols/test_option_roundtrip_unit.py(+40/−21); the source diff is unchanged, as intended. - The seven re-pointed fragments are verified against real behaviour, not just eyeballed. The harness asserts
fragment in outcome.detailfor every recorded gap, and the roundtrip run passes with 358 subtests — so each of the seven per-class fragments was matched against the detail the current code actually produces. - Scope justification — I think this is genuinely well done. Naming all ten other
_make_mptcp_*helpers the one change also fixes, and separating the newly-exposed defects into MPTCP.subtype is declared only under TYPE_CHECKING, so every subtype built via TCP's convenience constructor raises AttributeError #566 and TCP._make_mptcp_capable writes length 20/32 where RFC 8684 gives 12/20, and now packs wrong bytes rather than crashing #567 rather than folding speculative fixes in here, is the right division. The_make_mptcp_join/_flagsexclusion is also correctly characterised as unaffected either way.
Could not verify
- CI. Reached deliberately on local evidence only; I did not wait on GitHub Actions and make no claim about its tally.
- Any interpreter other than CPython 3.14.7.
- Whether unpacking fails silently for
MP_FAIL,MP_FASTCLOSEorREMOVE_ADDRonmain(§1). I measured ADD_ADDR and MP_PRIO only; both raise. I am not claiming the "silently" wording is wrong for every subtype, only that it is wrong for the two I tested. - Real MPTCP capture behaviour. All my parse evidence is from hand-built options I checked against RFC 8684 by hand, not from a capture file.
- That the ten other
_make_mptcp_*helpers all now construct and pack, which the description says was confirmed by hand for each. I verified ADD_ADDR thoroughly and MP_CAPABLE and MP_PRIO incidentally; I did not walk all ten.
45d37ad to
1388787
Compare
|
❌ NEEDS CHANGES Re-review after the force-push, head One of my two items is fully addressed. The other is untouched, and it is a one-sentence fix. Item 1 — parse-direction test: done, and done well. The new
Both docstrings name the pre-fix failure precisely, including the Item 2 — the changelog still covers only the packing direction, unchanged. That is the half a user will actually notice. On Everything else remains green at this head: 7 passed / exit 0 on the new file, changelog drift exit 0, and the shape is still one commit on current The #567 scoping note from my previous comment also still stands as a finding on #567 rather than on this PR: |
Addendum on the parse tests — bytes verified, and one item still openChecked at head The committed test uses the correct form, and it is fully self-consistent. The literals are module-level constants: ADD_ADDR_SPEC_OCTETS = bytes([0x1E, 0x08, 0x34, 0x01, 0xC0, 0x00, 0x02, 0x01])
MP_PRIO_SPEC_OCTETS = bytes([0x1E, 0x04, 0x51, 0x05])
And the technique really is maker-free, which was the defect in the old That closes my item 1 with no reservations. Item 2 is still open. The changelog entry has not changed.
Everything in it is about Verdict unchanged from my previous comment: ❌ NEEDS CHANGES, for that alone. The code, the tests and the |
#541) TCP._make_mptcp_addaddr could not build an ADD_ADDR option end to end: its kind=/length= arguments were rejected with UnknownFieldWarning and dropped, and .pack() then raised KeyError: 'length' from port's own condition, pkt['length'] in (10, 22). - Root cause was one layer up: MPTCP, the base class every Multipath TCP subtype schema inherits, declared kind and length only under typing.TYPE_CHECKING -- annotations for a type checker, not real fields -- unlike Option, which every non-Multipath TCP option schema inherits instead. MPTCP now declares both for real, the same way Option already did, so kind=/length= land and any sibling field reading pkt['length'] sees it. - The port predicate's apparent circularity (length derived from whether port is present, port gated on length) is not independent: length is supplied by the caller before port is packed, so once it is a real field landing earlier in field order, there is nothing circular left to resolve. - That one change also fixes construction for the other ten _make_mptcp_* helpers that pass kind=/length= the same way, not only _make_mptcp_addaddr's -- confirmed by hand for each. - The identical missing fields broke unpacking too, and for the same reason: with no kind/length fields ahead of it, MPTCPAddAddress's own leading field read the kind octet itself as its subtype/version octet, an off-by-two in field alignment rather than a wire-format change -- a correct sender's octets were always right, only this library's reading of them was shifted. Hand-built, spec-correct ADD_ADDR and MP_PRIO octets (RFC 8684 figures, built through no maker) now parse into a TCP with every field exactly as built; before this fix they raised FieldError: TCP: [OptNo 30] 3 invalid IP version and KeyError: 'length' respectively -- the latter the same exception #541 was filed against, reached by parsing instead of packing. - Added tests/protocols/transport/test_tcp_mptcp_addaddr_unit.py: packed-byte assertions for both IP versions and both port cases plus the opt= re-entry path (pack direction), and the two spec-octet parses above (parse direction). Each fails on the pre-fix code with the exception named for it and passes after. This fix exposes two further, independent defects rather than fixing them, filed separately and not touched here: #566, MPTCP.subtype is also TYPE_CHECKING-only and is set only by _MPTCP.post_process, which never runs on a schema built via TCP's convenience constructor; and #567, _make_mptcp_capable's length=20 if rkey is None else 32 is RFC 8684 section 3.1's 12/20 swapped, previously masked by the KeyError this fix removes and now packing successfully with wrong bytes. Re-pointed the seven tcp-mptcp/* EXPECTED_FAILURES entries in tests/protocols/test_option_roundtrip_unit.py (MP_CAPABLE, ADD_ADDR, REMOVE_ADDR, MP_PRIO, DSS, MP_FAIL, MP_FASTCLOSE) that this fix moves from KeyError: 'length' / no attribute 'kind' to a uniform no attribute 'subtype', quoting the class name each now raises against and pointing their defect prose at #566 -- MP_CAPABLE's entry also names #567, since it packs wrong bytes even once #566 is fixed. MP_JOIN (no attribute '_flags') is unaffected and left alone. That file is owned by nobody; not editing it unasked was the right call, and the decision was made and provided by the maintainer. Build/test: tests/protocols/transport passes in full (90 passed, 431 subtests); the full tier passes in full as well.
1388787 to
9001e4c
Compare
|
✅ GOOD TO MERGE Final re-review, head Both of my items are now closed. The delta from On the changelog wording, which is all you asked me to re-read: it does the job, and better than I asked for. The two things a user needs are both there. The searchable strings are verbatim and exact. And the distinction reads clearly to an outsider — this sentence is the one that earns its place:
That answers the question a user actually has after a capture fails to parse, which is "is my capture corrupt?" — no, and the entry says so in plain terms without requiring the reader to know what One judgement call I think it got right rather than wrong: it says "Spec-correct
Nothing further from me. For the record, the full set of things I verified across the three rounds and am not re-raising: the wire-format risk (this is a correction of an existing off-by-two, not a shift — spec-correct The |
…rkey (#566, #567) TCP(options=[(Enum_Option.Multipath_TCP, {...})]) raised AttributeError: '<schema class>' object has no attribute 'subtype' for every Multipath TCP subtype but MP_JOIN, and _make_mptcp_capable packed MP_CAPABLE with the wrong length either way. - MPTCP.subtype was declared only under typing.TYPE_CHECKING, so it was an annotation, never a field; the only code that ever set it was _MPTCP.post_process, which runs on a real byte-level unpack, not on the in-memory schema TCP's convenience constructor builds and reads straight back through _read_mptcp_*. Fixed on the construction path (TCP._make_mode_mp, the single dispatcher every _make_mptcp_* maker returns through) rather than by adding a third real field alongside kind/length: subtype is already packed as 4 bits of each subtype's own test bitfield, and a second field for the same bits would either double-pack them or need a "derive, don't pack" field kind this library's corekit.fields does not have. Recorded as a comment on MPTCP itself, since this is the third and last TYPE_CHECKING-only attribute that class had (#566). - _make_mptcp_capable wrote length=20 if rkey is None else 32, where RFC 8684 section 3.1 gives 12 and 20 -- both branches wrong, and the no-key branch writing the other case's value. MPTCPCapable.rkey's own condition (pkt['length'] != 32) independently dropped the receiver's key for exactly the length the maker used to mean "key present", so a 12-octet, key-absent MP_CAPABLE could not be built at all. Fixed together, to 12/20 and pkt['length'] == 20. A third site sharing the same wrong constants, _read_mptcp_capable's length guard and its rkey=... if length == 32 else None, is fixed alongside them -- only reachable once #566 let construction get that far (#567). - Added tests/protocols/transport/test_tcp_mptcp_subtype_unit.py and test_tcp_mptcp_capable_length_unit.py: subtype round-trips through the public TCP() constructor for every buildable subtype, and byte-exact packed assertions for both RFC 8684 MP_CAPABLE forms via the maker, via hand-built octets parsed independently of any maker, and via TCP() end to end. Each confirmed to fail on the pre-fix code. - Re-pointed tests/protocols/test_option_roundtrip_unit.py: six of the seven tcp-mptcp/* EXPECTED_FAILURES entries #541 left pointing at "no attribute 'subtype'" now read 'OK' and are deleted. MP_FASTCLOSE does not: fixing subtype gets it past that AttributeError and into a second, independent defect (its maker, schema and parser disagree on its own length) that #566/#567 do not touch, filed as #576. - test_tcp_udp_unit.py's test_tcp_mptcp_readers_cover_subtype_and_error_branches hand-marked MP_CAPABLE schemas at the pre-#567 lengths (20/32/12) as if that were correct behaviour, which #567 makes wrong; updated to 12/20, with the invalid-length case moved to 32. - Also found, filed separately, not fixed here (#576): the same shape of length-arithmetic defect in MP_FASTCLOSE, MP_JOIN SYN/ACK, MP_JOIN ACK, REMOVE_ADDR, MP_PRIO and DSS. Stacked on #565 (branched from its head, 9001e4c); merge after it. Build/test: tests/protocols/transport and tests/protocols pass in full (528 passed, 1412 subtests); the full tier passes in full as well (1281 passed, 17 skipped, 2850 subtests).
…rkey (#566, #567) TCP(options=[(Enum_Option.Multipath_TCP, {...})]) raised AttributeError: '<schema class>' object has no attribute 'subtype' for every Multipath TCP subtype but MP_JOIN, and _make_mptcp_capable packed MP_CAPABLE with the wrong length either way. - MPTCP.subtype was declared only under typing.TYPE_CHECKING, so it was an annotation, never a field; the only code that ever set it was _MPTCP.post_process, which runs on a real byte-level unpack, not on the in-memory schema TCP's convenience constructor builds and reads straight back through _read_mptcp_*. Fixed on the construction path (TCP._make_mode_mp, the single dispatcher every _make_mptcp_* maker returns through) rather than by adding a third real field alongside kind/length: subtype is already packed as 4 bits of each subtype's own test bitfield, and a second field for the same bits would either double-pack them or need a "derive, don't pack" field kind this library's corekit.fields does not have. Recorded as a comment on MPTCP itself, since this is the third and last TYPE_CHECKING-only attribute that class had (#566). - _make_mptcp_capable wrote length=20 if rkey is None else 32, where RFC 8684 section 3.1 gives 12 and 20 -- both branches wrong, and the no-key branch writing the other case's value. MPTCPCapable.rkey's own condition (pkt['length'] != 32) independently dropped the receiver's key for exactly the length the maker used to mean "key present", so a 12-octet, key-absent MP_CAPABLE could not be built at all. Fixed together, to 12/20 and pkt['length'] == 20. A third site sharing the same wrong constants, _read_mptcp_capable's length guard and its rkey=... if length == 32 else None, is fixed alongside them -- only reachable once #566 let construction get that far (#567). - Added tests/protocols/transport/test_tcp_mptcp_subtype_unit.py and test_tcp_mptcp_capable_length_unit.py: subtype round-trips through the public TCP() constructor for every buildable subtype, and byte-exact packed assertions for both RFC 8684 MP_CAPABLE forms via the maker, via hand-built octets parsed independently of any maker, and via TCP() end to end. Each confirmed to fail on the pre-fix code. - Re-pointed tests/protocols/test_option_roundtrip_unit.py: six of the seven tcp-mptcp/* EXPECTED_FAILURES entries #541 left pointing at "no attribute 'subtype'" now read 'OK' and are deleted. MP_FASTCLOSE does not: fixing subtype gets it past that AttributeError and into a second, independent defect (its maker, schema and parser disagree on its own length) that #566/#567 do not touch, filed as #576. - test_tcp_udp_unit.py's test_tcp_mptcp_readers_cover_subtype_and_error_branches hand-marked MP_CAPABLE schemas at the pre-#567 lengths (20/32/12) as if that were correct behaviour, which #567 makes wrong; updated to 12/20, with the invalid-length case moved to 32. - Also found, filed separately, not fixed here (#576): the same shape of length-arithmetic defect in MP_FASTCLOSE, MP_JOIN SYN/ACK, MP_JOIN ACK, REMOVE_ADDR, MP_PRIO and DSS. Stacked on #565 (branched from its head, 9001e4c); merge after it. Build/test: tests/protocols/transport and tests/protocols pass in full (528 passed, 1412 subtests); the full tier passes in full as well (1281 passed, 17 skipped, 2850 subtests).
…rkey (#566, #567) TCP(options=[(Enum_Option.Multipath_TCP, {...})]) raised AttributeError: '<schema class>' object has no attribute 'subtype' for every Multipath TCP subtype but MP_JOIN, and _make_mptcp_capable packed MP_CAPABLE with the wrong length either way. - MPTCP.subtype was declared only under typing.TYPE_CHECKING, so it was an annotation, never a field; the only code that ever set it was _MPTCP.post_process, which runs on a real byte-level unpack, not on the in-memory schema TCP's convenience constructor builds and reads straight back through _read_mptcp_*. Fixed on the construction path (TCP._make_mode_mp, the single dispatcher every _make_mptcp_* maker returns through) rather than by adding a third real field alongside kind/length: subtype is already packed as 4 bits of each subtype's own test bitfield, and a second field for the same bits would either double-pack them or need a "derive, don't pack" field kind this library's corekit.fields does not have. Recorded as a comment on MPTCP itself, since this is the third and last TYPE_CHECKING-only attribute that class had (#566). - _make_mptcp_capable wrote length=20 if rkey is None else 32, where RFC 8684 section 3.1 gives 12 and 20 -- both branches wrong, and the no-key branch writing the other case's value. MPTCPCapable.rkey's own condition (pkt['length'] != 32) independently dropped the receiver's key for exactly the length the maker used to mean "key present", so a 12-octet, key-absent MP_CAPABLE could not be built at all. Fixed together, to 12/20 and pkt['length'] == 20. A third site sharing the same wrong constants, _read_mptcp_capable's length guard and its rkey=... if length == 32 else None, is fixed alongside them -- only reachable once #566 let construction get that far (#567). - Added tests/protocols/transport/test_tcp_mptcp_subtype_unit.py and test_tcp_mptcp_capable_length_unit.py: subtype round-trips through the public TCP() constructor for every buildable subtype, and byte-exact packed assertions for both RFC 8684 MP_CAPABLE forms via the maker, via hand-built octets parsed independently of any maker, and via TCP() end to end. Each confirmed to fail on the pre-fix code. - Re-pointed tests/protocols/test_option_roundtrip_unit.py: six of the seven tcp-mptcp/* EXPECTED_FAILURES entries #541 left pointing at "no attribute 'subtype'" now read 'OK' and are deleted. MP_FASTCLOSE does not: fixing subtype gets it past that AttributeError and into a second, independent defect (its maker, schema and parser disagree on its own length) that #566/#567 do not touch, filed as #576. - test_tcp_udp_unit.py's test_tcp_mptcp_readers_cover_subtype_and_error_branches hand-marked MP_CAPABLE schemas at the pre-#567 lengths (20/32/12) as if that were correct behaviour, which #567 makes wrong; updated to 12/20, with the invalid-length case moved to 32. - Also found, filed separately, not fixed here (#576): the same shape of length-arithmetic defect in MP_FASTCLOSE, MP_JOIN SYN/ACK, MP_JOIN ACK, REMOVE_ADDR, MP_PRIO and DSS. Stacked on #565 (branched from its head, 9001e4c); merge after it. Build/test: tests/protocols/transport and tests/protocols pass in full (528 passed, 1412 subtests); the full tier passes in full as well (1281 passed, 17 skipped, 2850 subtests).
…rkey (#566, #567) (#579) TCP(options=[(Enum_Option.Multipath_TCP, {...})]) raised AttributeError: '<schema class>' object has no attribute 'subtype' for every Multipath TCP subtype but MP_JOIN, and _make_mptcp_capable packed MP_CAPABLE with the wrong length either way. - MPTCP.subtype was declared only under typing.TYPE_CHECKING, so it was an annotation, never a field; the only code that ever set it was _MPTCP.post_process, which runs on a real byte-level unpack, not on the in-memory schema TCP's convenience constructor builds and reads straight back through _read_mptcp_*. Fixed on the construction path (TCP._make_mode_mp, the single dispatcher every _make_mptcp_* maker returns through) rather than by adding a third real field alongside kind/length: subtype is already packed as 4 bits of each subtype's own test bitfield, and a second field for the same bits would either double-pack them or need a "derive, don't pack" field kind this library's corekit.fields does not have. Recorded as a comment on MPTCP itself, since this is the third and last TYPE_CHECKING-only attribute that class had (#566). - _make_mptcp_capable wrote length=20 if rkey is None else 32, where RFC 8684 section 3.1 gives 12 and 20 -- both branches wrong, and the no-key branch writing the other case's value. MPTCPCapable.rkey's own condition (pkt['length'] != 32) independently dropped the receiver's key for exactly the length the maker used to mean "key present", so a 12-octet, key-absent MP_CAPABLE could not be built at all. Fixed together, to 12/20 and pkt['length'] == 20. A third site sharing the same wrong constants, _read_mptcp_capable's length guard and its rkey=... if length == 32 else None, is fixed alongside them -- only reachable once #566 let construction get that far (#567). - Added tests/protocols/transport/test_tcp_mptcp_subtype_unit.py and test_tcp_mptcp_capable_length_unit.py: subtype round-trips through the public TCP() constructor for every buildable subtype, and byte-exact packed assertions for both RFC 8684 MP_CAPABLE forms via the maker, via hand-built octets parsed independently of any maker, and via TCP() end to end. Each confirmed to fail on the pre-fix code. - Re-pointed tests/protocols/test_option_roundtrip_unit.py: six of the seven tcp-mptcp/* EXPECTED_FAILURES entries #541 left pointing at "no attribute 'subtype'" now read 'OK' and are deleted. MP_FASTCLOSE does not: fixing subtype gets it past that AttributeError and into a second, independent defect (its maker, schema and parser disagree on its own length) that #566/#567 do not touch, filed as #576. - test_tcp_udp_unit.py's test_tcp_mptcp_readers_cover_subtype_and_error_branches hand-marked MP_CAPABLE schemas at the pre-#567 lengths (20/32/12) as if that were correct behaviour, which #567 makes wrong; updated to 12/20, with the invalid-length case moved to 32. - Also found, filed separately, not fixed here (#576): the same shape of length-arithmetic defect in MP_FASTCLOSE, MP_JOIN SYN/ACK, MP_JOIN ACK, REMOVE_ADDR, MP_PRIO and DSS. Stacked on #565 (branched from its head, 9001e4c); merge after it. Build/test: tests/protocols/transport and tests/protocols pass in full (528 passed, 1412 subtests); the full tier passes in full as well (1281 passed, 17 skipped, 2850 subtests).
Summary
Fixes #541.
TCP._make_mptcp_addaddrcould not build anADD_ADDRoption end to end: itskind=/length=arguments were rejected withUnknownFieldWarningand dropped, and.pack()then raisedKeyError: 'length'from theportfield's own condition,pkt['length'] in (10, 22).Root cause, deciding between the two the issue names:
MPTCP, the base class every Multipath TCP subtype schema inherits, declaredkindandlengthonly undertyping.TYPE_CHECKING-- annotations for a type checker, not real fields -- unlikeOption, which every non-Multipath TCP option schema inherits instead.Schema.__update__therefore rejected both keyword arguments as unknown. Theportpredicate's apparent circularity (lengthlooks like it depends on whetherportis present, andportis gated onlength) is not an independent problem:lengthis computed by the caller before packing starts, so once it is a real field that lands earlier in field order thanport, the predicate simply reads a value that is already there. Fixed by declaring realkind/lengthfields onMPTCPitself, mirroringOption.This repairs unpacking as well as packing, and for the identical reason -- confirmed independently by cross-review. With no
kind/lengthfields ahead of it,MPTCPAddAddress's own leading field (test, the subtype/IP-version octet) read thekindoctet itself rather than the third octet of the option -- an off-by-two in field alignment, not a wire-format change: a correct sender's octets were always right, only this library's reading of them was shifted. Hand-built, spec-correctADD_ADDRandMP_PRIOoctets (from the RFC 8684 figures, built through no_make_mptcp_*maker) failed to parse onmain--FieldError: TCP: [OptNo 30] 3 invalid IP versionforADD_ADDR(0x1e's low nibble decoded as IP version14) andKeyError: 'length'forMP_PRIO(the same exception #541 was filed against, reached by parsing instead of packing) -- and now parse correctly, every field exactly as built.Scope: that one change also fixes construction for the other ten
_make_mptcp_*helpers that passkind=/length=the same way --_make_mptcp_unknown,_make_mptcp_capable,_make_join_syn,_make_join_synack,_make_join_ack,_make_mptcp_dss,_make_mptcp_remove,_make_mptcp_prio,_make_mptcp_fail,_make_mptcp_fastclose-- confirmed by hand for each (all now construct and pack without error). Left alone, as distinct pre-existing defects this fix exposes rather than causes, and filed rather than fixed here:_make_mptcp_joinstill raisesAttributeError: 'TCP' object has no attribute '_flags'-- it reads parser-only state, so it cannot be called outside a real parse. Unaffected by this change either way.TCP(options=[(Enum_Option.Multipath_TCP, {...})], ...)convenience constructor (as opposed to calling_make_mptcp_*directly, or parsing real wire octets, both of which this PR fixes) still fails, withAttributeError: '<Class>' object has no attribute 'subtype'. That constructor immediately reads options back through_read_mptcp_*off the in-memory schema object without ever going through a byte-level parse, and.subtypeis only ever populated by_MPTCP.post_process(), which runs on a real unpack. Same category of defect askind/length(an attribute_read_mptcp_*expects that direct construction never populates), but a distinct fix._make_mptcp_capable's length arithmetic is swapped (length=20 if rkey is None else 32, where RFC 8684 section 3.1 wants 12/20) -- previously masked entirely by theKeyErrorthis PR fixes, now exposed as wrong-but-non-crashing packed bytes. The more dangerous of the two, since it packs successfully rather than raising. (Cross-review additionally foundMPTCPCapable.rkey's own predicate,pkt['length'] != 32, contradicts that arithmetic in the opposite direction -- filed as an amendment to TCP._make_mptcp_capable writes length 20/32 where RFC 8684 gives 12/20, and now packs wrong bytes rather than crashing #567, not fixed here either.)tests/protocols/test_option_roundtrip_unit.py'sEXPECTED_FAILUREStable (owned by nobody, decision made by the maintainer rather than by me): this fix moves 7 of the 8tcp-mptcp/*entries (all butMP_JOIN) fromKeyError: 'length'/no attribute 'kind'to a uniformno attribute 'subtype', still at theCONSTRUCTstage. Re-pointed rather than deleted: each entry'sfragmentnow quotes the exact class name the current run raises against (e.g."'MPTCPCapable' object has no attribute 'subtype'"), and eachdefectpoints at #566 --MP_CAPABLE's also names #567, since that case would still pack the wrong length even once #566 is fixed.MP_JOIN(no attribute '_flags') is untouched.tests/protocols/test_option_roundtrip_unit.py::OptionRoundTripTests::test_round_trip_is_identity_or_a_recorded_gappasses again as a result.Test plan
tests/protocols/transport/test_tcp_mptcp_addaddr_unit.py:ADD_ADDRon IPv4/IPv6 x with/without port, plus theopt=re-entry path. Verified each fails with the filedUnknownFieldWarnings andKeyError: 'length'on the pre-fix code, and passes after it.ADD_ADDRandMP_PRIOoctets (built through no maker) parsed into aTCPand asserted field by field. Verified each fails on the pre-fix code withFieldError: TCP: [OptNo 30] 3 invalid IP versionandKeyError: 'length'respectively, and passes after it.pytest tests/protocols/transport-- 90 passed, 431 subtests passed (samples regenerated viaexamples/generators/make_samples.py).pytest tests/protocols/test_option_roundtrip_unit.py-- 6 passed, 358 subtests passed, after re-pointing the sevenEXPECTED_FAILURESentries above.pytest tests(full tier) -- 0 failed.python util/changelog_md.py --check-- clean after regeneratingCHANGELOG.mdfromdocs/source/changelog/1.5.0.rst.