Measure benchmark memory usage natively instead of via /usr/bin/time - #307
Merged
guitargeek merged 4 commits intoSep 3, 2026
Merged
Conversation
RB::RunCommandMeasuringRss() runs a command like std::system() and additionally reports the maximum resident set size of the child process tree via wait4(). This is the same value that "/usr/bin/time -v" reports as "Maximum resident set size", but without depending on an external time binary, temporary files or output parsing. This supersedes the approach of PR root-project#190, which kept shelling out to the time binary.
The maximum resident set size was obtained by prefixing the tutorial invocation with a hardcoded /usr/bin/time, parsing its output file with grep and awk, and passing the result through a temporary file shared by all benchmarks. On systems without the time binary, the invocation failed before even running the tutorial, so the benchmark silently measured a failed shell command and reported an RSS of zero. Use RB::RunCommandMeasuringRss() instead, and fail the benchmark with a clear error if the tutorial cannot be run.
The RSS counter of the PyROOT benchmarks was always reported as zero, since no memory measurement was implemented at all. Worse, the shell quoting of the TPython::Exec() invocation was broken, so the tutorials never actually ran: root only reported an interpreter parse error, which does not even show up in its exit status. Fix the quoting, propagate a failure of the interpreted code into the exit status via gSystem->Exit(), measure the memory usage with RB::RunCommandMeasuringRss(), and fail the benchmark with a clear error if the tutorial cannot be run.
The Python tutorials moved with the big tutorials reorganization in ROOT, and several of them do not exist anymore. Update the paths of the six remaining tutorials that still run standalone, following the same convention as the interpreter benchmarks, and drop the others.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This supersedes #190 (itself a reincarnation of #95): same goal, but with a native implementation instead of shelling out to the
timebinary.Motivation
The interpreter benchmarks obtained their
RSScounter by prefixing the tutorial invocation with a hardcoded/usr/bin/time -v, parsing its output file withgrep/awk, and passing the result through atmp_mem_filein the working directory shared by all benchmarks. On systems without thetimebinary, the shell command failed before ever running the tutorial, so the benchmarks silently measured a failed shell invocation and reportedRSS=0.The PyROOT benchmarks were in even worse shape: no memory measurement at all (
RSSwas always 0), and the shell quoting of theTPython::Exec()call was broken, so the tutorials never actually ran — root only printed an interpreter parse error, which doesn't show up in its exit status either.Changes
RB::RunCommandMeasuringRss()toRBSupport: runs a command likestd::system()and reports the maximum resident set size of the child process tree viawait4(). This is the same value that/usr/bin/time -vreports as "Maximum resident set size", but with no external binary, no temporary files and no output parsing.SkipWithError) when a tutorial cannot be run, instead of silently reporting garbage timings.TPython::Exec()quoting and propagate failures of the interpreted code into the exit status viagSystem->Exit().geometry.py/na49view.py, which cannot run standalone (they need a geometry created by other macros — a failure the new error reporting surfaced).Verification
Against a recent ROOT master build:
Test_nopch_mlp_higgswent from a bogus 18 ms /RSS=0(on a machine without/usr/bin/time) to a real 4.2 s /RSS≈262M.Note for the dashboards
Timings and RSS values for these benchmarks will jump discontinuously once this is merged: the old numbers were measuring a failing shell command, so the apparent "regression" on the graphs is the benchmarks starting to work.