Repository navigation
benchmark: CodSpeed ignores syscalls #11907
Description
There are warnings on CodSpeed:
This benchmark contains N system calls, totalling T s of execution time. Since they cannot be consistently instrumented, those calls are not included in the measure. Please switch to the Walltime instrument to accurately measure system calls. Learn more about measurement and system calls.
Activity
xtqqczze commentedon Apr 20, 2026
Walltime instrument requires bare-metal runners managed by CodSpeed. Free plan provides 600 runner minutes/month, then $0.032 per minute.
oech3 commentedon Apr 20, 2026
600 is enough if we runs it only on main branch.
xtqqczze commentedon Apr 20, 2026
Last month benchmarks.yml used 216,379 minutes.
oech3 commentedon Apr 20, 2026
main only?
xtqqczze commentedon Apr 20, 2026
Assuming 149 minutes per run, and with 203 PRs merged last month, that's around 30,000 minutes
oech3 commentedon Apr 20, 2026
Auto comment contaning change of num of syscalls is useful.
xtqqczze commentedon Apr 20, 2026
codspeed commentedon Apr 20, 2026
Thank you for the suggestion! Comparing the number of system calls between HEAD and BASE is an interesting idea — however, this is a feature request for the CodSpeed platform itself, which is beyond what I can implement through repository changes.
I'll make sure the CodSpeed team is aware of this request. In the meantime, if you'd like to track or discuss this further, I'd recommend opening a feature request directly on the CodSpeed community so the team can evaluate it and other users can weigh in as well.
Summary
xtqqczze commentedon Apr 20, 2026
codspeed commentedon Apr 20, 2026
You're right, and I apologize for the broken link — the CodSpeed GitHub organization doesn't have a public Discussions tab enabled.
For feature requests, the best place is the main CodSpeed repository's issue tracker: https://github.com/CodSpeedHQ/codspeed/issues. There are already several issues tagged with "Feature" and "User Feedback" there. You can open a new issue describing the syscall count comparison feature you'd like to see.
Alternatively, you can reach the CodSpeed team directly at contact@codspeed.io.
Summary
tee: improve throughput by raw syscall - 17x perf win (#12131)
xtqqczze commentedon Sep 1, 2026
@sylvestre any thoughts?