stream: reduce per-chunk overhead in streams and webstreams by anonrig · Pull Request #64312 · nodejs/node

anonrig · 2026-07-05T20:06:08Z

Two independent, behavior-preserving performance changes to the streams
implementations, one per commit (intended to land separately via
commit-queue-rebase):

1. `stream: hoist repeated loads in readable paths`

fromList() re-read state.length and each chunk's length several times per
call and re-loaded buffer[idx] around every copy, and read() loaded
state.length three times in its read(0) check. The engine cannot fold these
loads across the intervening copy/slice calls, so they are cached in locals.
This mirrors what Bun's fork of the same functions carries on top of the shared
readable-stream lineage (bun/src/js/internal/streams/readable.ts).

2. `stream: use ring buffer for WHATWG stream queues`

The [[queue]] backing every default readable/writable controller was a plain
array of { value, size } wrappers consumed with ArrayPrototypeShift: one
wrapper allocation per buffered chunk, and element movement (or engine
re-linearization) per dequeue. The byte controller queue paid the same shift
cost for its chunk-descriptor records.

This replaces the array with a power-of-two ring buffer:

default controller queues store each entry as (value, size) in two
consecutive slots — the per-chunk wrapper allocation disappears;
the byte controller keeps its descriptor records (mutated in place at the
head) in single slots;
controllers start from (and are reset to) a shared immutable empty queue, so
constructing a stream allocates no queue storage until a chunk is actually
buffered;
enqueues measured by the internal default size algorithm (never observable
by user code, always returns 1, cannot throw) skip the algorithm call and
its try/catch.

The layout mirrors what Bun/WebKit use for the same spec structure
(WTF::Deque in WebCore's StreamQueue.h, the pure-JS ring buffer in Bun's
src/js/internal/fifo.ts).

Benchmark results

benchmark/compare.js against an unmodified-HEAD baseline binary on an
otherwise idle machine, Welch t-test (Rscript unavailable, so computed from
the CSV; happy to share the raw captures). Full webstreams suite at 30 runs
plus an independent 15-run repeat; streams set at 60 runs plus a 30-run
repeat. Improvements below are stable-sign across both captures.

Ring buffer (webstreams), all p < 1e-5:

benchmark	improvement
`webstreams/pipe-to.js` (all 16 HWM configs)	+12% … +17%
`webstreams/readable-read-buffered.js bufferSize=1`	+20%
`webstreams/readable-read-buffered.js bufferSize=10/100`	+36-38%
`webstreams/readable-read-buffered.js bufferSize=1000`	+49%
`webstreams/readable-async-iterator.js`	+21%

Readable hoists (node:stream):

benchmark	improvement
`streams/readable-unevenread.js`	+0.65% (p=4.3e-4) / +0.59% (p=5.9e-3)
`streams/pipe.js`	+0.74% (p=2.0e-4) / +0.52% (p=8.8e-3)

No statistically significant regression across the rest of the captured
streams/webstreams benchmarks (creation, js_transfer, readable-read,
boundaryread, bigread, uint8array, pipe-object-mode); two apparent full-suite
deltas (creation kind='ReadableStream.tee', readable-read type='normal')
disappear in isolated 60-run re-checks (p=0.50 / p=0.95) and are attributable
to thermal drift in the suite-ordered run.

Correctness

No public API, ordering, or error-behavior change; the queue is internal
state behind kState.
test/parallel/test-stream*, test-whatwg-*stream*, test-webstream*, and
test/wpt/test-streams.js all pass; full test/sequential passes.
New regression test test-webstreams-queue-wraparound.js drives ring
growth/wrap-around, drain-rewind/shrink, fractional/zero size()
accounting, BYOB partial in-place head consumption, and writable
backpressure drain through the public API only (it also passes on the
unpatched binary).

AI assistance disclosure

This is an AI-assisted pull request. The survey of Bun's implementation,
the ports/implementation, the benchmark methodology (baseline builds,
benchmark/compare.js captures, Welch t-test analysis, isolated re-runs for
suspected noise), the regression test, and drafts of the review-thread
responses were produced by an AI agent (Grok, xAI) operating under my
direction. I reviewed and take responsibility for every change; all landing
decisions, approvals, and force-pushes were human-authorized. Raw benchmark
CSVs and the analysis are available on request.

nodejs-github-bot · 2026-07-05T20:06:15Z

Review requested:

@nodejs/streams

nodejs-github-bot · 2026-07-05T21:36:39Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74564/

nodejs-github-bot · 2026-07-05T23:25:07Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74566/

nodejs-github-bot · 2026-07-05T23:25:26Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74567/

nodejs-github-bot · 2026-07-06T01:43:30Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74576/

mcollina

lgtm

nodejs-github-bot · 2026-07-06T12:18:21Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74590/

mcollina

lgtm

nodejs-github-bot · 2026-07-06T16:11:04Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74601/

nodejs-github-bot · 2026-07-06T18:00:19Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74606/

anonrig · 2026-07-06T22:23:30Z

CI status summary for the last few runs, since the red X is misleading (each failure is a different platform, none reproduce, and none are stream-related):

run	head	result	sole red subjob
74576	`43e5e91`	✅ green	—
74590	`d4e4cf5`	✅ green	—
74601	`d4e4cf5`	❌	`plinux-rhel8-power9le` (passed in 74576/74590/74606)
74606	`d4e4cf5`	❌	`windows-fanned` (passed in 74576/74590/74601)
latest wave	`c01723a`	❌	`smartos23-x64` @ build 66505 (passed @ 66503 on the same head hours earlier)

Zero overlap between failing platforms across runs, and nodejs/reliability shows a 25.56% CI green rate with repo-wide infra errors (git reset --hard 128, workspace failures) over the same window — this looks entirely like infra flake-hopping rather than anything in the diff. The only delta since the last fully green run (74590) is the two review-suggested tweaks (listLength parameter + capacityMask = listLength - 1), both re-verified locally against the full stream/webstreams/WPT suites.

Rather than burning another full 2h matrix on the lottery, could someone with Jenkins access resume-build just the failed smartos subjob (or kick a fresh run once infra settles)? I'll keep the branch conflict-free in the meantime.

jasnell · 2026-07-07T01:51:00Z

@anonrig ... the commits, PR description, and responses to review comments here all have the appearance of being AI generated. However, there's no acknowledgement of that I can find (e.g. no Assisted-by annotations). Can I ask to what extent AI was used here (helps set context for review)

anonrig · 2026-07-07T02:03:02Z

@anonrig ... the commits, PR description, and responses to review comments here all have the appearance of being AI generated. However, there's no acknowledgement of that I can find (e.g. no Assisted-by annotations). Can I ask to what extent AI was used here (helps set context for review)

@jasnell Ah yes. You're right to think that. Is that a new thing that I completely missed? Is it only required for pr descriptions or also for commits too?

jasnell · 2026-07-07T02:24:31Z

Assisted-by annotations are strongly recommended by the Foundation policy and have been added to our commit lint checking since March (still somewhat optional). The actual policy language as been accepted in general but there are still some details being worked through here #62105 ...

Ultimately, tho, it's not so much about defined policy as it is best practice. It's good to know when we're dealing with AI agent responses vs. an actual person.

anonrig · 2026-07-07T19:20:57Z

Will update commits now. Thanks for the explanation @jasnell

fromList() re-read state.length and each chunk's length several times per call and re-loaded buffer[idx] around every copy, and read() loaded state.length three times in its read(0) check; the engine cannot fold these loads across the intervening copy and slice calls. Cache them in locals instead. Ported from Bun's fork of the same functions (src/js/internal/streams/readable.ts fromList/read), which carries these hoists on top of the shared readable-stream lineage. benchmark/compare.js against the unmodified baseline (60-run capture plus an independent 30-run repeat, Welch t-test): streams/readable-unevenread +0.65% (p=4.3e-4) / +0.59% (p=5.9e-3) and streams/pipe.js +0.74% (p=2.0e-4) / +0.52% (p=8.8e-3), with no significant regression across the captured streams benchmarks. Refs: https://github.com/oven-sh/bun/blob/main/src/js/internal/streams/readable.ts Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com> Assisted-by: Grok (Grok Build)

The [[queue]] backing every default readable/writable controller was a plain array of { value, size } wrappers consumed with ArrayPrototypeShift, so each buffered chunk allocated a wrapper object and each dequeue moved (or forced the engine to re-linearize) the remaining elements; the byte controller queue paid the same shift cost for its chunk descriptor records. Replace the array with a power-of-two ring buffer. Default controller queues store each entry as (value, size) in two consecutive slots, so the per-chunk wrapper allocation disappears; the byte controller keeps its descriptor records (they are mutated in place at the head) in single slots. Controllers start from (and are reset to) a shared immutable empty queue, so constructing a stream allocates no queue storage until a chunk is actually buffered. Enqueues measured by the internal default size algorithm (never observable by user code, always returns 1, cannot throw) skip the algorithm call and its try/catch entirely. The layout mirrors what Bun/WebKit use for the same spec structure: [[queue]] as a ring-buffer deque (WTF::Deque in Bun's src/jsc/bindings/webcore/streams/StreamQueue.h), the pure-JS ring buffer in Bun's src/js/internal/fifo.ts, and the trivial-size-algorithm bypass in src/jsc/bindings/webcore/streams/JSReadableStreamDefaultController.cpp. benchmark/compare.js against the unmodified baseline (30-run capture plus an independent 15-run repeat, Welch t-test, all p < 1e-5): webstreams/pipe-to.js +12-17% across all sixteen high-water-mark configurations, readable-read-buffered +20% (bufferSize=1) to +49% (bufferSize=1000), readable-async-iterator +21%. No stable significant regression across the rest of the webstreams suite: the creation.js and readable-read.js deltas seen in the full-suite capture disappear in isolated 60-run rechecks. Refs: https://github.com/oven-sh/bun/blob/main/src/js/internal/fifo.ts Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com> Assisted-by: Grok (Grok Build)

mcollina

lgtm

nodejs-github-bot · 2026-07-07T20:50:46Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74643/

codecov · 2026-07-07T22:01:15Z

Codecov Report

❌ Patch coverage is 94.89362% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.26%. Comparing base (4ee2117) to head (9d0477c).
⚠️ Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/webstreams/util.js	93.91%	9 Missing ⚠️
lib/internal/streams/readable.js	95.08%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #64312      +/-   ##
==========================================
+ Coverage   90.23%   90.26%   +0.02%     
==========================================
  Files         741      741              
  Lines      240979   241143     +164     
  Branches    45401    45421      +20     
==========================================
+ Hits       217449   217656     +207     
+ Misses      15112    15074      -38     
+ Partials     8418     8413       -5

Files with missing lines	Coverage Δ
lib/internal/webstreams/readablestream.js	`98.16% <100.00%> (+<0.01%)`	⬆️
lib/internal/webstreams/writablestream.js	`99.50% <100.00%> (+<0.01%)`	⬆️
lib/internal/streams/readable.js	`97.27% <95.08%> (+0.02%)`	⬆️
lib/internal/webstreams/util.js	`97.71% <93.91%> (-1.96%)`	⬇️

... and 31 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

nodejs-github-bot · 2026-07-08T14:02:00Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74662/

nodejs-github-bot · 2026-07-08T15:05:48Z

Landed in 3fc183f...1912893

fromList() re-read state.length and each chunk's length several times per call and re-loaded buffer[idx] around every copy, and read() loaded state.length three times in its read(0) check; the engine cannot fold these loads across the intervening copy and slice calls. Cache them in locals instead. Ported from Bun's fork of the same functions (src/js/internal/streams/readable.ts fromList/read), which carries these hoists on top of the shared readable-stream lineage. benchmark/compare.js against the unmodified baseline (60-run capture plus an independent 30-run repeat, Welch t-test): streams/readable-unevenread +0.65% (p=4.3e-4) / +0.59% (p=5.9e-3) and streams/pipe.js +0.74% (p=2.0e-4) / +0.52% (p=8.8e-3), with no significant regression across the captured streams benchmarks. Refs: https://github.com/oven-sh/bun/blob/main/src/js/internal/streams/readable.ts Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com> Assisted-by: Grok (Grok Build) PR-URL: #64312 Reviewed-By: Robert Nagy <ronagy@icloud.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

The [[queue]] backing every default readable/writable controller was a plain array of { value, size } wrappers consumed with ArrayPrototypeShift, so each buffered chunk allocated a wrapper object and each dequeue moved (or forced the engine to re-linearize) the remaining elements; the byte controller queue paid the same shift cost for its chunk descriptor records. Replace the array with a power-of-two ring buffer. Default controller queues store each entry as (value, size) in two consecutive slots, so the per-chunk wrapper allocation disappears; the byte controller keeps its descriptor records (they are mutated in place at the head) in single slots. Controllers start from (and are reset to) a shared immutable empty queue, so constructing a stream allocates no queue storage until a chunk is actually buffered. Enqueues measured by the internal default size algorithm (never observable by user code, always returns 1, cannot throw) skip the algorithm call and its try/catch entirely. The layout mirrors what Bun/WebKit use for the same spec structure: [[queue]] as a ring-buffer deque (WTF::Deque in Bun's src/jsc/bindings/webcore/streams/StreamQueue.h), the pure-JS ring buffer in Bun's src/js/internal/fifo.ts, and the trivial-size-algorithm bypass in src/jsc/bindings/webcore/streams/JSReadableStreamDefaultController.cpp. benchmark/compare.js against the unmodified baseline (30-run capture plus an independent 15-run repeat, Welch t-test, all p < 1e-5): webstreams/pipe-to.js +12-17% across all sixteen high-water-mark configurations, readable-read-buffered +20% (bufferSize=1) to +49% (bufferSize=1000), readable-async-iterator +21%. No stable significant regression across the rest of the webstreams suite: the creation.js and readable-read.js deltas seen in the full-suite capture disappear in isolated 60-run rechecks. Refs: https://github.com/oven-sh/bun/blob/main/src/js/internal/fifo.ts Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com> Assisted-by: Grok (Grok Build) PR-URL: #64312 Reviewed-By: Robert Nagy <ronagy@icloud.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

The [[queue]] backing the controllers became a ring buffer, but the read and write request queues (readRequests, readIntoRequests, writeRequests) were left as plain arrays consumed with ArrayPrototypeShift, which is O(n) and, even for the single pending request of the await-each regime, far slower than an indexed head advance. Back them with the same Queue, materialized lazily from the shared empty queue so acquiring a reader or constructing a writer allocates no request storage until a read or write actually parks. pipe-to: +4.7% to +9.4% (all 16 configs, ***) readable-read type=byob: +2.3% (**) parked reader.read() loop: +15%, writer.write() loop: +11% (local harness) Follow-up to nodejs#64312. Signed-off-by: Matteo Collina <hello@matteocollina.com>

The [[queue]] backing the controllers became a ring buffer, but the read and write request queues (readRequests, readIntoRequests, writeRequests) were left as plain arrays consumed with ArrayPrototypeShift, which is O(n) and, even for the single pending request of the await-each regime, far slower than an indexed head advance. Back them with the same Queue, materialized lazily from the shared empty queue so acquiring a reader or constructing a writer allocates no request storage until a read or write actually parks. pipe-to: +4.7% to +9.4% (all 16 configs, ***) readable-read type=byob: +2.3% (**) parked read loop: +15%, write loop: +11% (local harness) Follow-up to nodejs#64312. Signed-off-by: Matteo Collina <hello@matteocollina.com>

nodejs-github-bot added needs-ci PRs that need a full CI run. stream Issues and PRs related to the stream subsystem. web streams labels Jul 5, 2026

anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 5, 2026

anonrig requested review from Qard, ShogunPanda, aduh95, mcollina and panva July 5, 2026 21:18

ronag approved these changes Jul 5, 2026

View reviewed changes

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 5, 2026

anonrig added the commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. label Jul 5, 2026

anonrig added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jul 6, 2026

aduh95 reviewed Jul 6, 2026

View reviewed changes

Comment thread lib/internal/webstreams/util.js Outdated

mcollina approved these changes Jul 6, 2026

View reviewed changes

anonrig force-pushed the stream-perf-ring-buffer branch from 43e5e91 to d4e4cf5 Compare July 6, 2026 14:19

anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 6, 2026

mcollina approved these changes Jul 6, 2026

View reviewed changes

anonrig added the commit-queue Add this label to land a pull request using GitHub Actions. label Jul 6, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 6, 2026

anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 6, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 6, 2026

anonrig added request-ci Add this label to start a Jenkins CI on a PR. and removed request-ci Add this label to start a Jenkins CI on a PR. labels Jul 6, 2026

Renegade334 removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jul 7, 2026

anonrig force-pushed the stream-perf-ring-buffer branch 2 times, most recently from 9031bc2 to e578439 Compare July 7, 2026 20:02

anonrig added 2 commits July 7, 2026 16:03

anonrig force-pushed the stream-perf-ring-buffer branch from e578439 to 9d0477c Compare July 7, 2026 20:03

mcollina approved these changes Jul 7, 2026

View reviewed changes

mcollina added request-ci Add this label to start a Jenkins CI on a PR. and removed request-ci Add this label to start a Jenkins CI on a PR. labels Jul 7, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 7, 2026

anonrig mentioned this pull request Jul 7, 2026

http: cut per-message outgoing overhead #64346

Open

anonrig added the commit-queue Add this label to land a pull request using GitHub Actions. label Jul 8, 2026

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jul 8, 2026

nodejs-github-bot closed this Jul 8, 2026

mcollina mentioned this pull request Jul 11, 2026

stream: use the ring buffer for WHATWG stream request queues #64431

Open

Uh oh!

Uh oh!

Conversation

anonrig commented Jul 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1. stream: hoist repeated loads in readable paths

2. stream: use ring buffer for WHATWG stream queues

Benchmark results

Correctness

AI assistance disclosure

Uh oh!

nodejs-github-bot commented Jul 5, 2026

Uh oh!

nodejs-github-bot commented Jul 5, 2026

Uh oh!

nodejs-github-bot commented Jul 5, 2026

Uh oh!

nodejs-github-bot commented Jul 5, 2026

Uh oh!

nodejs-github-bot commented Jul 6, 2026

Uh oh!

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

nodejs-github-bot commented Jul 6, 2026

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

nodejs-github-bot commented Jul 6, 2026

Uh oh!

nodejs-github-bot commented Jul 6, 2026

Uh oh!

anonrig commented Jul 6, 2026

Uh oh!

jasnell commented Jul 7, 2026

Uh oh!

anonrig commented Jul 7, 2026

Uh oh!

jasnell commented Jul 7, 2026

Uh oh!

anonrig commented Jul 7, 2026

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

nodejs-github-bot commented Jul 7, 2026

Uh oh!

codecov Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

nodejs-github-bot commented Jul 8, 2026

Uh oh!

nodejs-github-bot commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

anonrig commented Jul 5, 2026 •

edited

Loading

1. `stream: hoist repeated loads in readable paths`

2. `stream: use ring buffer for WHATWG stream queues`

codecov Bot commented Jul 7, 2026 •

edited

Loading