Skip to content

Fix: readUInt8Array must copy, not view the reused scan buffer#5491

Open
DexterKoelson wants to merge 1 commit into
clockworklabs:masterfrom
DexterKoelson:fix/u8-array-buffer-aliasing
Open

Fix: readUInt8Array must copy, not view the reused scan buffer#5491
DexterKoelson wants to merge 1 commit into
clockworklabs:masterfrom
DexterKoelson:fix/u8-array-buffer-aliasing

Conversation

@DexterKoelson

Copy link
Copy Markdown
Contributor

Description of Changes

Fixes #5490.

Reading an array<u8> column returned a Uint8Array that was a view over the row iterator's scan buffer instead of an owned copy. That buffer comes from a shared pool and gets reused by the next table scan, so a u8-array value that outlived the scan it was read from would silently start reflecting some later scan's bytes. No error, just wrong data. Every other column type decodes to an owned value, so only the array<u8> fast path had this.

Changes in BinaryReader:

  • readUInt8Array now returns an owned copy (readBytes(length).slice()), so decoded column values have a lifetime independent of the reader's buffer reuse.
  • readString reads its bytes via readBytes directly. TextDecoder copies synchronously and keeps no reference to the buffer, so strings stay zero-copy and don't pay the new copy.
  • readBytes is unchanged and still returns a view, with a comment noting the view is only valid until the buffer is reused.

API and ABI breaking changes

None. The return type of readUInt8Array is still Uint8Array; callers just get a copy they own instead of a view that could be invalidated under them.

Expected complexity level and risk

  1. The change is three small methods in one file. The only thing to watch is the extra allocation per u8-array column read, which is why readString was routed around it to avoid a double copy on the common string path.

Testing

  • Added unit tests in binary_read_write.test.ts: a read value survives mutation of the reader buffer, does not share the backing ArrayBuffer, and readString still decodes UTF-8 correctly.
  • Verified against the original repro (two tables, read a u8-array column after another table scan): correct before the second scan, corrupted after on master, correct in both cases with this change.

- `readUInt8Array` now returns an owned copy (`readBytes(length).slice()`), so decoded column values have a lifetime independent of the reader's buffer reuse.
- `readString` reads its bytes via `readBytes` directly. `TextDecoder` copies synchronously and keeps no reference to the buffer, so strings stay zero-copy and don't pay the new copy.
- `readBytes` is unchanged and still returns a view, with a comment noting the view is only valid until the buffer is reused.
@JasonAtClockwork JasonAtClockwork self-requested a review July 7, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reading a u8 array column returns a view into a reused scan buffer (huge footgun)

2 participants