Repository navigation
fix(protocols): handle declared lengths above sys.maxsize as overruns (#1975) - #1979
Conversation
…#1975) Where Py_ssize_t is 32 bits wide (GraalPy, 32-bit CPython) a read or seek past sys.maxsize raises OverflowError instead of coming back short, so a PCAP incl_len or PCAP-NG length reaching that far crashed the reader before the overrun handling could keep the record as captured. - Schema.unpack bounds its payload and field reads at sys.maxsize. - Frame.read bounds the record read, and a next-record offset past sys.maxsize becomes the end of the octets read. - PCAPNG.read and _read_block_cut bound the block read. - SeekableReader._fill bounds a READ_CHUNK of -1 at sys.maxsize. - Lift #1974's sys.maxsize skips from the #1541 tests, and add a test that runs the narrow build's limit on any interpreter. Claims that fit read exactly as before. Closes #1975
|
When the desired read is beyond |
|
Partly yes. I measured on GraalPy 25.4.4, where
The PR gets the second part wrong. When a record's next offset lies past
The PCAP-NG block reads get the same treatment. Later records then parse exactly as they do on 64-bit CPython. |
… of truncating (#1975) A record or block the file really holds past sys.maxsize was read only to the limit, and the next one was then parsed from inside it. - corekit.io.advance moves a stream forward any distance: relative seeks of at most sys.maxsize, or chunked read-and-discard on a SeekableReader or a non-seekable stream. - Frame.read and PCAPNG.read call it when the bounded read came back whole, so the next record starts at the declared end; a short read is unchanged. - The PCAP and PCAP-NG engines cap the read-ahead at sys.maxsize, and the PCAP frame schema reads no more than the header and payload can hold in one read, so a SeekableReader keeps the record start in its window. - SeekableReader._fill bounds any READ_CHUNK at sys.maxsize. - Tests cover a sparse file holding a record past a lowered limit, seekable and through a pipe, for PCAP and PCAP-NG. Closes #1975
|
Pushed
The new sparse-stream tests fail on |
|
GOOD TO GO at |
|
Coverage: 91.52% (unit tier, Python 3.15,
Per-file detail: the |
|
use |
|
Keeping |
make pylint,make mypy,make isort)make testpasses, and a test case covers the changeWhat is the purpose of your pull request?
fix— a bug fixDescription of your pull request and other information
A declared length above
sys.maxsize, such as a PCAPincl_lenof2**31 - 16on GraalPy or 32-bit CPython, used to crash a read or seek withOverflowErrorbefore the overrun handling could see it. Every length-driven read is now bounded atsys.maxsize, so a hostile claim comes back short and gets the same "captured length runs past the data" or "block truncated" handling as on 64-bit CPython. Claims that fit are read exactly as before.Schema.unpack: the payload and field reads. These cover PCAP payloads, PCAP-NG nested fields and the next layer'scap_len.Frame.read: the record read, and the next-record offset when it lies pastsys.maxsize.PCAPNG.readand_read_block_cut: the block reads.SeekableReader._fill: aREAD_CHUNKof-1.sys.maxsizeskips in the two fix(dumpkit): a PCAP trace copies a truncated record's claimed incl_len, writing a malformed file #1541 modules are removed.The new
test_declared_length_past_maxsize_1975_unit.py(ALL) simulates a 32-bitPy_ssize_ton CPython. It fails onmain(12 failed) and passes here. On GraalPy 25.4.4, the two #1541 modules and the new one pass with no skips, whilemainfails 14 withOverflowError. The third-party engines are left as they are, as best effort under #1687.Closes #1975