fulmine, fulmine-tuned: fulmine.js 5.18.1, gzip on json-comp, pg-telaio 0.1.2 - #1255
Conversation
|
/benchmark -f fulmine-tuned -t production-stack |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
|
/benchmark-multiple -f fulmine,fulmine-tuned --save |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFramework:
Full log |
|
Wait for #1255 (comment) |
f1ef7d8 to
ca23933
Compare
|
/benchmark-multiple -f fulmine,fulmine-tuned --save |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFrameworks: 2 | Test: ✅
|
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 1,251,529 | 6770.9% | 2.6GiB | -2.8% | ~0% |
| baseline | 4096 | 1,234,471 | 6595.3% | 2.7GiB | -2.8% | ~0% |
| pipelined | 512 | 31,433,334 | 6688.8% | 2.1GiB | -2.1% | ~0% |
| pipelined | 4096 | 31,540,985 | 6711.9% | 2.1GiB | -1.7% | ~0% |
| limited-conn | 512 | 1,051,872 | 6221.4% | 2.6GiB | -1.2% | -3.7% |
| limited-conn | 4096 | 1,133,960 | 6439.8% | 2.7GiB | -0.7% | ~0% |
| json | 4096 | 1,123,330 | 6474.7% | 6.9GiB | +0.5% | -15.9% |
| json-comp | 512 | 455,335 | 6201.6% | 7.3GiB | +43.3% | -5.2% |
| json-comp | 4096 | 477,613 | 6421.6% | 7.5GiB | +43.2% | -12.8% |
| json-comp | 16384 | 492,726 | 6257.4% | 8.5GiB | +38.4% | -18.3% |
| json-tls | 4096 | 1,046,604 | 6456.2% | 7.0GiB | +0.5% | +1.4% |
| upload | 32 | 2,024 | 2988.6% | 6.7GiB | -0.4% | -1.5% |
| upload | 256 | 2,117 | 6366.0% | 7.4GiB | -1.2% | +2.8% |
| api-4 | 256 | 36,187 | 388.9% | 536MiB | -2.0% | +2.9% |
| api-16 | 1024 | 125,579 | 1535.4% | 2.2GiB | +4.7% | ~0% |
| static | 1024 | 381,761 | 6570.4% | 6.3GiB | +0.9% | +1.6% |
| static | 4096 | 504,353 | 6573.3% | 7.3GiB | +0.3% | -3.9% |
| static | 6800 | 558,475 | 6528.5% | 7.7GiB | +0.6% | ~0% |
| static-tls | 1024 | 323,561 | 6557.6% | 6.2GiB | +0.9% | ~0% |
| static-tls | 4096 | 410,633 | 6461.0% | 6.9GiB | +0.6% | -1.4% |
| static-tls | 6800 | 438,209 | 6397.1% | 7.3GiB | +0.6% | -1.4% |
| async-db | 1024 | 226,186 | 6008.3% | 3.7GiB | -2.2% | -14.0% |
| crud | 4096 | 358,248 | 4342.1% | 8.2GiB | +0.8% | -2.4% |
| fortunes | 1024 | 68,266 | 6498.5% | 4.2GiB | +0.4% | ~0% |
| gateway-64 | 512 | 91,011 | 4123.7% | 5.2GiB | +0.2% | +4.0% |
| gateway-64 | 1024 | 81,506 | 3912.8% | 7.7GiB | +9.2% | +10.0% |
| gateway-h3 | 64 | 90,120 | 3339.5% | 3.4GiB | -1.1% | +17.2% |
| gateway-h3 | 256 | 62,493 | 2429.9% | 4.6GiB | ~0% | +12.2% |
| production-stack | 256 | 59,608 | 4134.5% | 3.2GiB | +1.0% | -3.0% |
| production-stack | 1024 | 50,177 | 4202.6% | 4.9GiB | +6.9% | -12.5% |
| echo-ws | 512 | 3,749,580 | 6530.1% | 2.4GiB | -0.2% | ~0% |
| echo-ws | 4096 | 3,837,599 | 6519.4% | 2.4GiB | +0.3% | ~0% |
| echo-ws | 16384 | 3,600,528 | 6496.3% | 2.5GiB | -0.1% | ~0% |
| echo-ws-pipeline | 512 | 24,932,364 | 6812.1% | 2.4GiB | -0.2% | ~0% |
| echo-ws-pipeline | 4096 | 25,124,326 | 6593.1% | 2.4GiB | +0.2% | ~0% |
| echo-ws-pipeline | 16384 | 23,794,067 | 6392.9% | 2.6GiB | +0.1% | ~0% |
| echo-ws-limited | 512 | 2,093,837 | 5523.0% | 2.4GiB | +0.1% | ~0% |
| echo-ws-limited | 4096 | 2,571,048 | 6239.1% | 2.4GiB | +0.4% | ~0% |
✅ fulmine-tuned
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 1,257,752 | 6768.8% | 2.6GiB | -4.6% | ~0% |
| baseline | 4096 | 1,241,907 | 6571.4% | 2.7GiB | -3.4% | ~0% |
| pipelined | 512 | 33,090,694 | 6780.3% | 2.1GiB | -1.9% | ~0% |
| pipelined | 4096 | 33,462,227 | 6726.2% | 2.1GiB | -0.2% | ~0% |
| limited-conn | 512 | 1,064,430 | 6138.9% | 2.7GiB | -1.3% | +3.8% |
| limited-conn | 4096 | 1,143,274 | 6469.5% | 2.7GiB | -1.1% | ~0% |
| json | 4096 | 1,131,255 | 6430.7% | 6.8GiB | -0.8% | ~0% |
| json-comp | 512 | 459,348 | 6194.1% | 7.3GiB | +43.1% | -3.9% |
| json-comp | 4096 | 479,916 | 6383.1% | 7.6GiB | +40.5% | -13.6% |
| json-comp | 16384 | 496,994 | 6310.5% | 8.5GiB | +36.1% | -16.7% |
| json-tls | 4096 | 1,068,089 | 6459.8% | 6.6GiB | +1.4% | -1.5% |
| upload | 32 | 2,033 | 3024.4% | 6.8GiB | +0.6% | +4.6% |
| upload | 256 | 2,121 | 6292.2% | 7.2GiB | -0.4% | -2.7% |
| api-4 | 256 | 36,814 | 397.1% | 552MiB | +2.1% | -1.1% |
| api-16 | 1024 | 127,669 | 1562.1% | 2.2GiB | +0.4% | ~0% |
| static | 1024 | 424,399 | 6560.1% | 6.3GiB | +0.8% | +1.6% |
| static | 4096 | 574,516 | 6541.8% | 7.4GiB | -0.2% | ~0% |
| static | 6800 | 645,855 | 6512.3% | 8.1GiB | +2.7% | +3.8% |
| static-tls | 1024 | 356,600 | 6536.8% | 6.1GiB | +0.5% | ~0% |
| static-tls | 4096 | 460,444 | 6462.2% | 6.9GiB | ~0% | ~0% |
| static-tls | 6800 | 501,523 | 6391.9% | 7.3GiB | +0.5% | +1.4% |
| async-db | 1024 | 251,202 | 6114.7% | 6.2GiB | +0.7% | ~0% |
| crud | 4096 | 377,971 | 3711.1% | 8.2GiB | -0.1% | ~0% |
| fortunes | 1024 | 68,661 | 6487.7% | 6.5GiB | -1.9% | ~0% |
| gateway-64 | 512 | 92,036 | 4208.2% | 5.3GiB | +1.8% | +12.8% |
| gateway-64 | 1024 | 80,342 | 4002.6% | 7.8GiB | -1.4% | +18.2% |
| gateway-h3 | 64 | 90,874 | 3320.5% | 3.5GiB | +1.0% | +16.7% |
| gateway-h3 | 256 | 62,851 | 2423.1% | 4.7GiB | +3.5% | +11.9% |
| production-stack | 256 | 58,793 | 4262.4% | 3.1GiB | +0.2% | -3.1% |
| production-stack | 1024 | 42,988 | 4030.9% | 4.2GiB | +4.2% | -17.6% |
| echo-ws | 512 | 3,779,060 | 6428.7% | 2.3GiB | +0.7% | ~0% |
| echo-ws | 4096 | 3,836,177 | 6318.3% | 2.4GiB | +0.2% | ~0% |
| echo-ws | 16384 | 3,599,054 | 6197.9% | 2.5GiB | -0.3% | ~0% |
| echo-ws-pipeline | 512 | 25,103,993 | 6797.4% | 2.4GiB | +0.4% | ~0% |
| echo-ws-pipeline | 4096 | 24,947,062 | 6581.5% | 2.4GiB | -0.7% | ~0% |
| echo-ws-pipeline | 16384 | 23,818,524 | 6342.4% | 2.6GiB | +0.5% | ~0% |
| echo-ws-limited | 512 | 2,105,837 | 5621.8% | 2.4GiB | +0.7% | ~0% |
| echo-ws-limited | 4096 | 2,576,206 | 6054.4% | 2.4GiB | +0.3% | ~0% |
|
/benchmark-multiple -f fulmine,fulmine-tuned --save |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
|
|
…orts the driver rows in place
…made the cheaper call
6eb425f to
bb13e15
Compare
|
/benchmark-multiple -f fulmine,fulmine-tuned --save |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFrameworks: 2 | Test: ✅
|
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 1,348,644 | 6779.8% | 2.7GiB | +4.8% | +3.8% |
| baseline | 4096 | 1,340,956 | 6784.6% | 2.7GiB | +5.5% | ~0% |
| pipelined | 512 | 31,577,351 | 6767.5% | 2.1GiB | -1.7% | ~0% |
| pipelined | 4096 | 31,380,352 | 6716.2% | 2.1GiB | -2.2% | ~0% |
| limited-conn | 512 | 1,124,445 | 6168.9% | 2.7GiB | +5.6% | ~0% |
| limited-conn | 4096 | 1,215,520 | 6462.1% | 2.8GiB | +6.5% | +3.7% |
| json | 4096 | 1,123,852 | 6443.2% | 6.9GiB | +0.6% | -15.9% |
| json-comp | 512 | 460,547 | 6195.2% | 7.3GiB | +44.9% | -5.2% |
| json-comp | 4096 | 479,964 | 6382.0% | 7.6GiB | +43.9% | -11.6% |
| json-comp | 16384 | 497,788 | 6361.9% | 8.5GiB | +39.9% | -18.3% |
| json-tls | 4096 | 1,055,889 | 6495.0% | 6.7GiB | +1.4% | -2.9% |
| upload | 32 | 2,030 | 3086.7% | 6.8GiB | -0.1% | ~0% |
| upload | 256 | 2,163 | 6331.5% | 7.3GiB | +0.9% | +1.4% |
| api-4 | 256 | 36,634 | 393.3% | 539MiB | -0.8% | +3.5% |
| api-16 | 1024 | 126,735 | 1540.7% | 2.2GiB | +5.6% | ~0% |
| static | 1024 | 382,422 | 6562.3% | 6.2GiB | +1.1% | ~0% |
| static | 4096 | 506,071 | 6556.6% | 7.6GiB | +0.6% | ~0% |
| static | 6800 | 560,864 | 6520.3% | 7.7GiB | +1.1% | ~0% |
| static-tls | 1024 | 325,511 | 6553.8% | 6.3GiB | +1.5% | +1.6% |
| static-tls | 4096 | 411,486 | 6493.8% | 7.0GiB | +0.8% | ~0% |
| static-tls | 6800 | 439,419 | 6380.8% | 7.3GiB | +0.9% | -1.4% |
| async-db | 1024 | 232,808 | 5694.4% | 4.3GiB | +0.7% | ~0% |
| crud | 4096 | 358,930 | 4162.2% | 8.4GiB | +1.0% | ~0% |
| fortunes | 1024 | 68,448 | 6509.7% | 4.3GiB | +0.6% | +2.4% |
| gateway-64 | 512 | 91,443 | 4069.9% | 5.4GiB | +0.7% | +8.0% |
| gateway-64 | 1024 | 81,711 | 3951.0% | 7.6GiB | +9.5% | +8.6% |
| gateway-h3 | 64 | 90,305 | 3327.5% | 3.4GiB | -0.9% | +17.2% |
| gateway-h3 | 256 | 62,793 | 2423.1% | 4.5GiB | +0.6% | +9.8% |
| production-stack | 256 | 59,185 | 4212.5% | 3.2GiB | +0.3% | -3.0% |
| production-stack | 1024 | 48,040 | 3908.7% | 4.1GiB | +2.3% | -26.8% |
| echo-ws | 512 | 3,770,125 | 6492.8% | 2.3GiB | +0.3% | -4.2% |
| echo-ws | 4096 | 3,835,606 | 6331.5% | 2.4GiB | +0.3% | ~0% |
| echo-ws | 16384 | 3,600,730 | 6169.3% | 2.5GiB | -0.1% | ~0% |
| echo-ws-pipeline | 512 | 24,973,673 | 6800.0% | 2.4GiB | ~0% | ~0% |
| echo-ws-pipeline | 4096 | 25,110,668 | 6591.2% | 2.4GiB | +0.1% | ~0% |
| echo-ws-pipeline | 16384 | 24,018,432 | 6374.9% | 2.6GiB | +1.1% | ~0% |
| echo-ws-limited | 512 | 2,089,376 | 5485.6% | 2.4GiB | ~0% | ~0% |
| echo-ws-limited | 4096 | 2,579,391 | 6029.6% | 2.4GiB | +0.7% | ~0% |
✅ fulmine-tuned
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 1,348,833 | 6834.1% | 2.7GiB | +2.3% | +3.8% |
| baseline | 4096 | 1,339,524 | 6794.1% | 2.7GiB | +4.2% | ~0% |
| pipelined | 512 | 33,194,000 | 6780.7% | 2.1GiB | -1.6% | ~0% |
| pipelined | 4096 | 33,681,708 | 6510.7% | 2.1GiB | +0.4% | ~0% |
| limited-conn | 512 | 1,138,874 | 6149.4% | 2.7GiB | +5.6% | +3.8% |
| limited-conn | 4096 | 1,233,753 | 6456.9% | 2.8GiB | +6.7% | +3.7% |
| json | 4096 | 1,140,337 | 6629.5% | 8.2GiB | ~0% | +20.6% |
| json-comp | 512 | 460,937 | 6171.4% | 7.3GiB | +43.6% | -3.9% |
| json-comp | 4096 | 480,926 | 6436.5% | 7.6GiB | +40.7% | -13.6% |
| json-comp | 16384 | 496,034 | 6332.7% | 8.6GiB | +35.9% | -15.7% |
| json-tls | 4096 | 1,071,284 | 6494.2% | 6.8GiB | +1.7% | +1.5% |
| upload | 32 | 2,047 | 3146.2% | 6.7GiB | +1.3% | +3.1% |
| upload | 256 | 2,157 | 6213.1% | 7.1GiB | +1.3% | -4.1% |
| api-4 | 256 | 36,632 | 395.0% | 545MiB | +1.6% | -2.3% |
| api-16 | 1024 | 127,296 | 1552.7% | 2.2GiB | ~0% | ~0% |
| static | 1024 | 426,047 | 6559.2% | 6.3GiB | +1.1% | +1.6% |
| static | 4096 | 576,698 | 6538.1% | 7.2GiB | +0.2% | -2.7% |
| static | 6800 | 647,192 | 6506.8% | 7.9GiB | +2.9% | +1.3% |
| static-tls | 1024 | 358,551 | 6532.1% | 6.2GiB | +1.1% | +1.6% |
| static-tls | 4096 | 466,500 | 6460.9% | 6.9GiB | +1.4% | ~0% |
| static-tls | 6800 | 500,290 | 6418.0% | 7.2GiB | +0.3% | ~0% |
| async-db | 1024 | 248,561 | 6024.2% | 6.2GiB | -0.4% | ~0% |
| crud | 4096 | 375,368 | 3520.3% | 8.4GiB | -0.8% | +2.4% |
| fortunes | 1024 | 70,511 | 6492.2% | 6.5GiB | +0.8% | ~0% |
| gateway-64 | 512 | 91,458 | 4147.9% | 5.6GiB | +1.1% | +19.1% |
| gateway-64 | 1024 | 83,905 | 3849.5% | 7.0GiB | +3.0% | +6.1% |
| gateway-h3 | 64 | 90,478 | 3300.6% | 3.4GiB | +0.6% | +13.3% |
| gateway-h3 | 256 | 60,821 | 2385.2% | 4.6GiB | +0.2% | +9.5% |
| production-stack | 256 | 59,230 | 4217.2% | 3.3GiB | +1.0% | +3.1% |
| production-stack | 1024 | 47,743 | 4208.9% | 5.1GiB | +15.8% | ~0% |
| echo-ws | 512 | 3,765,345 | 6454.5% | 2.3GiB | +0.3% | ~0% |
| echo-ws | 4096 | 3,826,471 | 6349.2% | 2.4GiB | ~0% | ~0% |
| echo-ws | 16384 | 3,606,493 | 6218.5% | 2.5GiB | -0.1% | ~0% |
| echo-ws-pipeline | 512 | 25,147,430 | 6812.0% | 2.4GiB | +0.6% | ~0% |
| echo-ws-pipeline | 4096 | 25,058,201 | 6609.7% | 2.4GiB | -0.2% | ~0% |
| echo-ws-pipeline | 16384 | 23,832,268 | 6350.1% | 2.6GiB | +0.6% | ~0% |
| echo-ws-limited | 512 | 2,107,514 | 5523.1% | 2.4GiB | +0.7% | ~0% |
| echo-ws-limited | 4096 | 2,583,634 | 6062.4% | 2.4GiB | +0.6% | ~0% |
pg-telaio 0.1.2 fixes a false positive in its slow-query guard, and the guard is then turned off for this entry: every query these profiles run is a point read of a few milliseconds, so there is no slow query for it to catch. production-stack is 50,177 rps at 1024 connections, +6.9% on top of the +21.8% the earlier run on this branch measured.
Both entries build the json items with a preallocated loop instead of slice().map(), and fortunes pushes the runtime row onto the driver rows and sorts them in place instead of copying them first.
json-comp changes encoding, which reverses what this entry did before, and fulmine.js is why: 5.18.0 gzips a whole body on a stream it keeps rather than on one built and thrown away per call, and a brotli stream cannot be kept that way. Brotli's body is 10% smaller and the score squares that, so this was only worth trying because the call got cheaper. The run says it was: +43.3%, +43.2% and +38.4% rps on the three rows, with bytes per request going 1379 to 1506, which is a net gain on the scored figure.
5.18.1 then copies a request body chunk out of a view of the uWS buffer instead of out of a slice of it, which is the same bytes without an ArrayBuffer allocated per chunk. It is not what these rows measure; it is here because the entry should not pin an older version than the one it was built against.