Hardening wave 3b — roadmap 26: the stress rig, four-peer net-stress, and a governor that drops shadows before frames - #211
Merged
Conversation
… frame Roadmap 26 section 6. The section-2 budget numbers were estimates; this measures them. - tests/e2e/scene-stress.cjs: the manual rig (like net-stress.cjs). Per scene size: seed/import cost + long tasks, frame p50/p95/p99 idle AND orbiting, draw calls and triangles per display frame, geometries/textures, heap, object-list render ms, one autosave export (ms, bytes), optionally physics over the scene (bodies, step p50/p95, whether 26-G's stop fired) and a second peer joining (time-to-synced). Names the GPU and refuses to treat a software rasteriser as data. - tests/e2e/sceneStressProbe.cjs: the in-page probe both the rig and the suite drive, so the regression covers the real measurement code. Everything timed is timed in the page. - FOUND AND FIXED: the meter's triangles/calls read ONE fullscreen pass. renderer.info auto-resets per render() and a desktop frame is 13 render() calls, so 1,000 boxes read "1 call, 1 triangle": those budgets could never leave green and 26-G's sceneIsHeavy was asking about objects alone. sceneBudget now wraps the renderer instance's render and divides the sum by display frames. autoReset is untouched (VRStats, diagnostics and a reset-then-render test read what they always did) and it works in XR. Stopping the sampler hands back the original function. - FOUND AND FIXED: the loading stall timer (26-B M2) measured duration, not silence. A joiner receiving 3,000 boxes was still landing ~10/s when the bar cleared at 63s and a toast said 1,085 objects "never arrived"; all arrived by 180s. Every arrival re-arms it. - Metric sources the rig needed, registered from their own modules: bodies and physicsStepMs (physics.js), autosaveExportMs/autosaveBytes (autosave.js), syncMs and syncObjects (commandsHandler: announcement -> last object on the receiver's own clock; null for a batch closed unfinished). - BUDGETS retuned from the measurement (Radeon 890M, 1280x720): desktop calls [1000,2000] -> [2000,4500] (1,943 calls = 60fps; 4,446 = steady 30fps; 5,323 = p95 50ms) and triangles [1M,3M] -> [4M,8M] (6M/frame held 60fps). Required, not optional: the corrected counter reads ~2x objects (the shadow pass), so the old tiers would have made a 520-box scene "heavy" and armed 26-G's freeze streak in non-GPU suites. VR columns unchanged (owed on a headset). Full tables are in the lane handover. - Measured, not fixed here (handed to 26-D): ingest is FRAME-BOUND. 3,000 objects take ~180s to land while the joiner draws, 5.6s with drawing paused. Counterfactuals (each broken, suite red, restored): - countRenderCalls a no-op: 7 red (calls/triangles 0, not wrapped, doubling, stop/start) - 'bodies' source renamed: 4 red (no-sim reading, sim count, stopped run, rig row) - the sync `complete` flag forced true: 1 red (closed batch reports a fast sync) - 'autosaveExportMs' source renamed: 1 red - the stall re-arm removed: 1 red (batch given up on while still arriving) Suites: scene-stress (new) 28/28. Held, all green: scene-budget, overload-guard, ingest-gate, scene-poke, vr-stats, mesh-edit-materials, dispose, diagnostics, object-sync, net-handshake, physics-colliders, autosave-object-flows. svelte-check 352/47 (base 352/47). npm run build green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ling, and the rig measures presence
Roadmap 25 section 3d, and the N=4 regression 27-I's brief asked for.
- tests/e2e/localSignal.cjs: the local `peer` server on :9001 (extracted from the rig),
REUSED when one already answers (the port is machine-wide), plus LOCAL_PEER_STORAGE =
peerServerConfig {mode:'local'}. Seeding the pages is what keeps them off production;
the old "APP_URL must be localhost" check is now "must resolve to this machine", so a
lane serving theprototype.app via /etc/hosts can run the rig.
- net-stress.test.cjs: FOUR peers on the local server (was three on the shared box).
With three, the host's `hosts` roster only ever names one other peer, so a fill that
mishandled a longer list still passed; with four, six of the twelve links come from
the fill alone. Checks: every ordered pair open (pair-complete), host broadcast whole
to all three, all four blasting at once (12/12 pairs whole, counters reset first — the
running-maximum trap), fan-out bounded, and NEW: the presence stream while all four
orbit, stated as messages/s per sender against a premise that every sender drew well
above the 20/s gate (so a per-frame sender would be visible), plus long tasks.
- net-stress.cjs: default sizes 8,10,12,16; `--presence N` (every peer orbits for N
seconds, each counts camera messages RECEIVED per sender); a long-tasks/min column on
every load step; a presence table.
Measured (Radeon 890M box, ALL peers on one machine, local signaling, 20 objects):
- full mesh at 8, 10, 12 and 16 peers
- 0% loss up to 3,360 (N=8), 10,800 (N=10) and 7,920 (N=12) mesh msgs/s; 0.24% at
15,840 (N=12); at N=16 0.07% even at 10Hz, 1.87% at 28,800 msgs/s — the box is
saturated there (16 GPU contexts, idle 34fps, echo RTT p95 550ms)
- presence received per peer: 132/s (N=8), 169/s (N=10), 207/s (N=12), 260/s (N=16);
0.31-0.36 messages per sender frame at 60fps, i.e. the 25-C gate holds at scale
- long tasks/min 0 at N<=12; frame drop with N is GPU/compositor contention, not the
main thread
Counterfactuals (each broken, suite red, restored):
- mesh fill disabled (hosts -> no connectToPeer): 6 of 12 pairs missing, 6/12 pairs
deliver under four-way load
- camera gate removed (camGapMs 0): 59.9 msgs/s per sender against 18.2 with it
Suites: net-stress.test 15/15 (was 10/10 on three peers). svelte-check 352/47 (base
352/47). npm run build green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…, and a joiner stops starving its own download
Roadmap 26 section 4, Stage 1, steered by what 26-E measured.
- src/lib/qualityGovernorCore.js (PURE, import-free): the decision rule. p95 over 2s
above the trigger (or >2 long tasks in 5s) on a HEAVY scene takes one step, held 3s;
10s of p95 under 20ms walks one back. A step up within 20s of a walk down doubles the
next recovery hold (flapping), capped at 80s. A 600ms settle window after every change,
because the change itself is a hitch (a shadow toggle recompiles every lit material).
- src/lib/qualityGovernor.js: the wiring. Frames from sceneBudget's loop, published as a
LOCAL qualityOverrides store every consumer reads; never writes a preference, a
document or a message. Hidden tab / 26-G pause = no evidence.
- THE STEP ORDER IS THE MEASUREMENT'S, not the roadmap's: shadows first (the shadow pass
is the second copy of every mesh; calls, not fill, bind a many-object scene), then
resolution 85/72%, AO, 61%, the post stack, 50%, the particle cap (Stage 3's third
bullet), the presence send gap. Consumers: lightParams + environment through one
`shadowsDisabled()` (environment re-asserted the saved preference on every apply and
undid the override within a frame — found by the suite), threlte's own dpr (Scene),
AO/post filtered in Outline and the composer re-sized on a dpr change, particleRuntime,
Scene's camera gap.
- The desktop trigger is 35ms, not 33: frames are vsync-quantised, so a steady 30fps
reads 33.3-33.4ms and a 33ms trigger would walk it to the bottom of the ladder.
- Light scenes are never governed (the 26-G ruling: a slow machine is not an overloaded
scene), which also keeps SwiftShader suites untouched.
- NOT FIGHTING 26-G: the first step records the size readings (qualityBaseline) and
sceneIsHeavy judges by the larger of now and then until full quality returns — else
turning shadows off halves the calls and talks the freeze guard out of a scene that is
still too heavy. A scene that really shrinks (<70% of the baseline objects) drops it.
- THE INGEST DRAW GAP (26-E's biggest finding): while a received batch drains through
slow frames (backlog > 50, p95 > 20ms) the renderer draws 4 frames a second, sticky for
the drain. MEASURED with the rig, joiner time-to-synced for 1,000 / 2,000 / 3,000 boxes:
8.0s / 112s / ~180s before, 1.8s / 3.4s / 6.2s after, zero long tasks.
- UI: a chip beside the object count ("Reduced quality (scene is heavy)" — click to hold,
click again for full quality with a 60s snooze), a one-time toast with the same two
actions, and Settings > "Reduce quality when the scene is heavy" (LOCAL, default on).
Measured end to end in the suite on real frames (Radeon 890M): 3,000 real boxes engage the
governor on their own, it takes ONE step (shadows off), draw calls 5,312 -> 2,930, frame
p95 50ms -> 33.4ms, and it stops there.
Counterfactuals (each broken, red, restored):
- sceneIsHeavy ignoring the baseline: 2 red (26-G no longer judges heavy; no pause)
- environment re-asserting shadowMap.enabled: 2 red (shadows stay on; real calls
5,312 -> 5,310)
- Outline's composer not following dpr: 1 red (composer buffer 1280 -> 1280)
- the draw gap early return removed: 1 red (780 render calls/s against 780)
- desktop trigger back to 33ms: unit red (steady 30fps read as overloaded)
- settle window 0: unit red (the recompile's long tasks take a second step)
Suites: perf-governor (new) 39/39; unit qualityGovernor (new) 21, all unit 119/119.
Held, green: scene-stress, overload-guard, scene-budget, ingest-gate, scene-poke,
object-sync, view-mode, shadows, environment, environment-v2, env-preset-broadcast,
scene-post, scene-post-ui, post-play-mode, particles, flow-particle, net-stress,
camera-pip, settings-toasts-ux, settings-labels; net-handshake red once on a two-peer
join then green on re-run. scene-post-effects 4.5 ("assigning a LUT PUSHES its bytes")
is red IDENTICALLY with this diff reverted to HEAD — pre-existing, not chased.
svelte-check 352/47 (base 352/47). npm run build green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 17, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave 3's second half: the rigs that were supposed to replace roadmap 26's estimated budget numbers, and the governor that steers by them.
37b4ecdd9a15edcf1c200Two defects in shipped code that the rig found
These are the reason a measuring phase comes before a governing one, and both are fixed in
37b4ecd.renderer.infoauto-resets perrender()call, and a desktop frame is 13 of them (composer: scene, N8AO, outline, final quad). 1,000 boxes readtriangles: 1, calls: 1. So those two budgets could never leave green — and 26-G'ssceneIsHeavy()gate, which Hardening wave 2b — roadmap 26: the overload gateway, from one poke per frame to a window that pauses instead of freezing #209 added, was really only asking about object count. Fixed by wrapping the renderer instance'srenderand dividing by display frames; XR-safe, andinfo.autoResetis untouched so VRStats, diagnostics and the existing tests read exactly what they always did.The measurements (roadmap 26 §2 was estimates; these are measured)
Rig
node tests/e2e/scene-stress.cjs. ANGLE Vulkan, AMD Radeon 890M (RADV) — an integrated GPU, headless Chromium 1280×720, vsync-capped so frames quantise to 16.7/33.3/50ms. 4s windows.Post and AO cost almost nothing here —
shadedwith no composer reads 1,905 calls at 1,000 boxes against 2,943 with it, and p95 is identical. The cost is the shadow pass (~2× calls) and per-call CPU. That is what 26-D gives up first.Full tables — dense GLB models, presence, and the four-peer runs — are in
~/.code/handovers/done/STATUS-26-measure.md, written for the orchestrator to fold into roadmap 26 §2.Metric sources wired
bodiesandphysicsStepMs(physics.js, p95 of a 120-step ring, null when no run) ·autosaveExportMs,autosaveBytes(autosave.js) ·syncMs,syncObjects(commandsHandler — announcement to last object, on the receiver's own clock, null for a batch closed unfinished).Not wired, listed for later:
bufferedAmounthigh-water per conn (peerHandler, which #210 owns), IDB write latency, script-node ms per frame.Evidence
New:
scene-stress28/28. Held green:scene-budget,overload-guard,ingest-gate,scene-poke,vr-stats,mesh-edit-materials,dispose,diagnostics,object-sync,net-handshake,physics-colliders,autosave-object-flows. Build green, counterfactuals in each commit body.🤖 Generated with Claude Code