Skip to content

fix(pcapng): cap a Decryption Secrets Block's secrets at its block (#1980) - #1983

Merged
JarryShaw merged 3 commits into
mainfrom
fix/1980-dsb-secrets-length
Oct 20, 2026
Merged

JarryShaw merged 3 commits into
mainfrom
fix/1980-dsb-secrets-length

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • fix — a bug fix

Description of your pull request and other information

A Decryption Secrets Block whose secrets_length runs past its block no longer aborts extraction. A new secrets_area caps the secrets at what the block holds; it shares _held_octets with captured_area. The DSB's padding and option area are sized from the capped length. DecryptionSecretsBlock.post_process keeps the declared secrets_length and warns, the way the EPB does ("secrets length N runs past the block … kept as captured"). If a key log is cut off by the block, its partial last line is dropped, as a short read already does, so the TLS and WireGuard parsers keep the complete entries. The frames before and after the bad block survive.

An audit of the module's other length-driven fields found them already capped at the block, or computed from the block's own length.

The new test_pcapng_dsb_secrets_overrun_1980_unit.py fails on main (8 failed) and passes here. Well-formed DSBs still rebuild byte-exact.

Closes #1980

…1980)

- dsb_secrets_selector sized the secrets from secrets_length alone, so a
  length past the block read the blocks after it, and past the zero-pad
  ceiling raised FieldValueError inside the schema, aborting the extract.
  The new secrets_area bounds it at the Block Total Length less the
  framing, sharing _held_octets with captured_area; padding_data and the
  option area are sized from the bounded length.
- DecryptionSecretsBlock.post_process keeps the declared secrets_length
  and warns with a ProtocolWarning worded like the EPB one, on the
  unpacking path only.
- A key log the block ends inside drops its cut last line, as a short
  read already does, so the TLS and WireGuard parsers keep the entries
  before it instead of raising.
- Document secrets_area; the selector unit test now passes the block
  length.

New test module fails on origin/main and passes here; tests/protocols/misc,
tests/protocols/schema, the PCAP-NG engine and tier-guard tests pass.

Closes #1980
@JarryShaw JarryShaw added fix Pull requests that fix a defect (fix: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 20, 2026
@JarryShaw JarryShaw moved this to In review in PyPCAPKit Oct 20, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at 11f3f2783. A Sonnet review confirmed that well-formed DSBs are unchanged and that the cap works. One gap remains: an oversized DSB that now parses does not rebuild byte-exact. _make_block_dsb recomputes secrets_length, and it wrote 04000000 where the original had f0ffffff. An oversized EPB keeps captured_len and round-trips exactly, so the DSB build must keep the declared length the same way. The PR's test should check from_data(...).data against the original bytes, not __header__.pack().

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 20, 2026
…#1980)

- _make_block_dsb recomputed secrets_length from the secrets it wrote, so
  a block whose declared length ran past it rebuilt with the held length.
  It now keeps the data model's declared secrets_length when that exceeds
  the secrets held, as the EPB build keeps captured_len; a block built
  fresh from user data still computes it.
- The secrets are measured in the context of the block they were parsed
  from, so a key log the block cut is still read as cut instead of being
  refused while the block is rebuilt.

The new rebuild assertions fail on 11f3f27 and pass here.

Closes #1980
@JarryShaw

Copy link
Copy Markdown
Owner Author

Pushed aed6dfe7a. _make_block_dsb now keeps the data model's secrets_length when it is larger than the secrets held, as the EPB build keeps captured_len, so an oversized DSB rebuilds byte-exact through from_data(...).data. A DSB built fresh still computes its length. The new rebuild test fails on 11f3f2783 (5 failed) and passes here. The PR merges cleanly with #1979 and with main. A delta review is running.

@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Oct 20, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at aed6dfe7a. The oversized-DSB rebuild is now byte-exact. But keeping the declared secrets_length whenever it is larger than the secrets held also catches lengths that fit the block, and the pad then vanishes:

  • A ZigBee key declared 20 over 18 held octets writes 38 octets against a Block Total Length of 40.
  • An unknown DSB whose secrets are edited down keeps its stale length.

The declared length should be kept only on a true overrun, where it runs past the block body; otherwise the length of the secrets held is used. The 19 failures the review saw in tests/protocols/misc and schema come from its environment: main passes all 297 here.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 20, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

CodeQL fails on this PR with 9 "clear-text logging/storage of sensitive data" alerts, all false positives. The SARIF traces every alert back to one source: the new warning's {self.secrets_length} in DecryptionSecretsBlock.post_process, plus two test literals. CodeQL's sensitive-data heuristic matches the substring secret in the name secrets_length. The value is an integer octet count read off the wire, not secret material. The sinks are the warning and logging paths in warnings.py, extraction.py and corekit/io.py.

There are three ways to clear it:

  1. Dismiss the alerts as false positives in Code Scanning. The code stays as it is.
  2. Bind the count to a neutral name before formatting. That fixes nothing real and only dodges the heuristic.
  3. Add a CodeQL config that excludes py/clear-text-* for pcapkit/protocols/. That is broad.

I recommend option 1. Dismissing alerts is a security-tab action, though, so I'll leave it to you unless you say otherwise.

@JarryShaw JarryShaw added the needs: decision Waiting on the maintainer to decide — not blocked by other work label Oct 20, 2026
…lock (#1980)

- _make_block_dsb kept a data model's secrets_length whenever it exceeded
  the secrets written, so a length that fits the block's padding, or one
  an edit to secrets_data had outdated, was written over the new secrets
  and the block came out shorter than its Block Total Length. It is now
  kept only on a true overrun, past what the block body holds as
  secrets_area measures it, the condition the parse side warns on.
- Test a ZigBee key declared within its padding and an edited unknown
  DSB for a block as long as its Block Total Length says.

The new cases fail on aed6dfe and pass here; tests/protocols/misc and
tests/protocols/schema pass.

Closes #1980
@JarryShaw

Copy link
Copy Markdown
Owner Author

Pushed 22355b653. _make_block_dsb now keeps the declared secrets_length only when it runs past what secrets_area says the block body holds, which is the same condition the parse side warns on. Otherwise it uses the length of the secrets written. The ZigBee key declared as 20 now rebuilds as a valid 40-octet block, and an edited DSB takes its new length. Both new cases fail on aed6dfe7a and pass here, and the oversized round trip is still byte-exact. A delta review is running. The CodeQL question above is still open.

@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Oct 20, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO at 22355b653. A Sonnet delta review found that every harness case writes octets equal to its Block Total Length. Oversized TLS, WireGuard and unknown DSBs, cut and uncut, rebuild byte-exact, and well-formed DSBs match main. The overrun test shares secrets_area/_held_octets with the parse side. The two new tests fail on aed6dfe7a. The CodeQL false positives above still need your call before CI goes green.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 20, 2026
@JarryShaw JarryShaw moved this from In review to Needs decision in PyPCAPKit Oct 20, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Option 1

@JarryShaw

Copy link
Copy Markdown
Owner Author

Done. I dismissed the CodeQL alerts on this PR as false positives, with the reason recorded on each. CodeQL now reports success.

@JarryShaw JarryShaw removed the needs: decision Waiting on the maintainer to decide — not blocked by other work label Oct 20, 2026
@JarryShaw JarryShaw moved this from Needs decision to In review in PyPCAPKit Oct 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage: 91.52% (unit tier, Python 3.15, 22355b653, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 2 95.16%
pcapkit/const 19597 881 2618 867 92.11%
pcapkit/corekit 2524 70 800 36 96.57%
pcapkit/dumpkit 276 3 100 3 98.40%
pcapkit/foundation 2916 23 1016 25 98.78%
pcapkit/interface 138 4 52 3 96.32%
pcapkit/protocols 17438 226 4592 191 98.09%
pcapkit/toolkit 872 36 298 10 95.21%
pcapkit/utilities 522 2 158 4 99.12%
pcapkit/vendor 4431 1995 1018 167 51.73%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw
JarryShaw merged commit d811bf2 into main Oct 20, 2026
60 checks passed
@JarryShaw
JarryShaw deleted the fix/1980-dsb-secrets-length branch October 20, 2026 17:07
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 20, 2026
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 20, 2026
JarryShaw added a commit that referenced this pull request Oct 20, 2026
…#1985) (#1989)

* fix(pcapng): warn on a malformed TLS key-log line instead of aborting (#1985)

A TLS or WireGuard key log line that did not parse raised FieldValueError
from post_process, which aborted the whole extraction, wherever the line
sat in the log. It now raises a ProtocolWarning ("...; line kept as
captured"), is left out of the entries, and the well-formed entries around
it are kept. The log text is already kept verbatim on the data model, so
the block still rebuilds octet for octet. A last line the capture or the
block cut is still left out silently, as #1983 does.

Tests that pinned the old refusal now pin the warning.

Closes #1985

* fix(pcapng): keep a malformed key-log line out of entries entirely (#1985)

TLSKeyLog.post_process indexed entries[label] before bytes.fromhex ran, so a line with a known label but fields that do not decode left an empty bucket for that label. Both values are decoded first and entries is touched only once they are. WireGuardKeyLog decodes before it adds, so it needs no change.

Also drops timed wording from the test docstrings and rewraps the post_process docstring.

Refs #1985

* fix(pcapng): elide key-log values from the malformed-line warning (#1985)

The malformed-line warning quoted the whole key-log line, which carries key material. _malformed_line_message names only the label, when the key log knows it, and elides the values. An unrecognised label is not quoted either, since it may be key material too. The WireGuard parser's internal ValueError no longer carries the line.

Tests assert the new text and that no warning quotes a hex secret or base64 key from their input.

Refs #1985
@JarryShaw JarryShaw added this to the PyPCAPKit v1.5 milestone Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix Pull requests that fix a defect (fix: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

fix(pcapng): a Decryption Secrets Block with an oversized secrets_length aborts extraction

1 participant