diff --git a/CHANGELOG.md b/CHANGELOG.md index 496c4a0..b523bd2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -194,7 +194,7 @@ recorded trace hash moves, once, for every workload. the syncs Raft owes before it answers an RPC; drop the sync before an append is acknowledged and the first seed loses a committed entry. - Campaign evidence at scale, regenerable via `benchmarks/campaign.py`: - 100,000 seeds of jobqueue chaos green in just over six minutes on a laptop, + 100,000 seeds of jobqueue chaos green in just over four minutes on a laptop, every ablation caught with its failure density recorded, and 20 sampled failing seeds replaying with identical trace hashes across 100 re-runs apiece. A small nightly CI sweep keeps the numbers honest. diff --git a/README.md b/README.md index 391c252..512ff67 100644 --- a/README.md +++ b/README.md @@ -277,7 +277,7 @@ recording included — about 3.7× faster than the stock loop, which pays a selector syscall per iteration), compresses sleep-heavy workloads ~2,000× against wall clock, and with `--simloop-jobs` fanning seeds across processes, the full jobqueue chaos scenario sweeps 100,000 seeds in just -over six minutes (~260 seeds/second) on an M4 MacBook Air. Methodology, numbers, +over four minutes (~390 seeds/second) on an M4 MacBook Air. Methodology, numbers, and the campaign results: [benchmarks/README.md](https://github.com/dhruvl/simloop/blob/main/benchmarks/README.md). diff --git a/benchmarks/README.md b/benchmarks/README.md index 1995c91..9641509 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -119,9 +119,16 @@ Results, recorded 2026-08-04 on the M4 MacBook Air (10 jobs): | campaign | scale | result | |---|---|---| -| green | 100,000 seeds, 6.3 min, 263.1 seeds/s | green — no invariant violated | -| ablations | 6 mutations × 10,000 seeds, 2.7 min | every ablation caught, densities below | -| replay stability | 20 failing seeds × 100 re-runs, 14.2 s | identical trace hash on every run | +| green | 100,000 seeds, 4.2 min, 394.5 seeds/s | green — no invariant violated | +| ablations | 6 mutations × 10,000 seeds, 3.0 min | every ablation caught, densities below | +| replay stability | 20 failing seeds × 100 re-runs, 15.7 s | identical trace hash on every run | + +The green row is where the workers' frozen heap shows up: nearly every green +seed reaches the settle phase and pays the end-of-run collections in full, +so taking the imported heap out of them moved this number from 263 to 394 +seeds/s in one sitting on this machine. The ablation runs barely moved — +most of those seeds fail within a few steps and never accumulate the work +the collection was re-walking. Per-ablation failure density: