[vitecopeter]

Problem and result

Results only, no code. Two older data-centre cards on the CUDA 12 engine, each in its own Hyper-V VM on the same host (2x Xeon E5-2630 v3, 8 vCPUs and 66 GB RAM per VM), running Qwen3.8-Flash-Next IQ3_S at 262K context on release v0.2.12 (commit 146df3c, engine 0.1.58):

Run	Tesla V100-PCIE 32 GB (Windows Server 2022, 130 W)	Tesla P40 24 GB (Ubuntu 24.04, 150 W)
BENCH, short chat decode	57.2 tok/s (49.5-67.9)	29.0 tok/s (26.8-35.9)
BENCH, agent decode	67.3 tok/s	35.2 tok/s
BENCH, 4K / 16K prompt	1,083 / 1,300 tok/s	352 / 379 tok/s
Standard draft.1, status	complete, 21/21, 0 failed	complete, 21/21, 0 failed
Standard, short / 4K / 16K decode	64.1 / 58.5 / 59.3 tok/s	34.6 / 30.0 / 28.5 tok/s
Standard, 4K / 16K prompt	1,043 / 1,276 tok/s	347 / 396 tok/s
Standard, sanity checks	12/12	12/12

Both setups turned KV streaming on (3.6 GB of KV in RAM at 262K) and calibrated the options themselves; the V100 holds 12,115 experts (23.0 GiB) and the P40 7,930 (15.0 GiB) in VRAM.

The PR also carries the four P40 reports measured on 0.1.43 on 2026-10-13 (IQ3_S, Coder IQ1_M, Unsloth UD-IQ4_XS, UD-Q4_K_XL; first submitted in #2328, which was closed in the 0.2.0 clean-up). They are labelled 0.1.43 in their folders and rows. The same P40 went from 21.5 chat / 26.2 agent tok/s on 0.1.43 to 29.0 / 35.2 on 0.2.12 with IQ3_S, though the VM also changed (Windows with 98 GB RAM then, Ubuntu with 66 GB now).

Related discussion

#2328 (the 0.1.43 P40 reports, closed with "Closed here does not mean rejected"). No scope agreement needed: results only.

Approach

• bench/results/2026-11-08-community-tesla-{p40,v100-pcie-32gb}-qwen3-8-flash-next-iq3-s/: BENCH.bat / bench.sh reports, unchanged except a "Submitter's notes" section with what the tool cannot see (VM, power limit, storage, install).
• bench/results/2026-11-08-standard-tesla-p40-bb7bd2c6/ and 2026-11-08-standard-tesla-v100-pcie-32gb-260c2c3b/: strata bench run output as written, protocol strata-qwen38-thinking-v1-draft.1, track custom (262K context instead of setup's recommended 128K). The engine.log files are added with -f because .gitignore excludes *.log; every artifact hash in results.json matches the committed bytes.
• bench/results/2026-10-13-community-tesla-p40-* (four folders): as in Community benchmark: Tesla P40 (Windows Server 2022 VM), four models at 256K context #2328.
• bench/results/COMMUNITY.md: one row per folder.

Validation

Passed:

• strata bench run --profile strata-iq3_s.json --track custom ... on both VMs from a freshly started engine: status complete, 21 of 21 required requests, 0 failed, 0 skipped, sanity 12/12. P40 from the git checkout (build_kind: source); V100 from the tag archive.
• BENCH (tools/strata_bench.py, suite 1, 3 runs per test) right after setup's calibration on both VMs.
• The SHA-256 of every artifact listed in both results.json files was recomputed from the staged blobs: all 12 match.
• Redaction checked by hand: no keys, user or host names, or private addresses. The V100 engine log keeps the drive path of the data folder (E:\Strata\...), which holds nothing private; it was left as written because the log is hashed.

Failed:

• A first standard run on the V100 was nonconforming because the tag archive has no .git and no COMMIT file. A COMMIT file with the tag's commit (146df3c468739147f520ac3816de806f71232571) was then added to the install by hand and the run repeated from a fresh engine; only the repeat is in this PR. Its build_kind therefore reads release, although the CUDA 12 engine was compiled by setup (stated in --build-options and --track-notes).

Not run:

• The optional concurrency stage of the standard protocol.
• Correctness beyond the twelve sanity checks. (We have LiveCodeBench v6 runs of the same model on these cards on 0.1.43, outside this PR; not repeated on 0.2.12.)
• --model-revision / --source-revision: not passed, so they are null with a reason.

Limits

• One host, VMs with DDA passthrough; other VMs on the host kept running and their load was not recorded.
• The V100 is power-limited to 130 W for cooling; an unlimited card will be faster.
• Hyper-V's virtual disk reports itself as rotational to Linux, so setup warned about a rotational disk on the P40 VM although the VHDX is on an NVMe SSD. A udev rule fixed the flag after the run; it does not change the engine's options.
• 0.2.x has no ready-made CUDA 12 engine for a source install, so both engines were compiled locally with CUDA 12.9.

🤖 Generated with Claude Code