Repository navigation
fix: require numeric EPS/throughput fields before scoring sysbench results - #2
Closed
BlueBirdBack wants to merge 1 commit into
Closed
BlueBirdBack wants to merge 1 commit into
BlueBirdBack wants to merge 1 commit into
Conversation
…sults
A sysbench run that exits 0 without an 'events per second' field produced
'CPU single thread: events/s, p95 1.00 ms', and the scorer then parsed the
'95' inside the 'p95' label as CPU throughput — a missing measurement was
scored as a real (FULL) result.
Now each sysbench metric validates its parsed value: missing/empty EPS or
memory-throughput fields mark the metric FAILED ('no events-per-second
field'), so the score degrades to PARTIAL/not-comparable instead of
borrowing digits from display text.
Adds a regression test with a p95-only sysbench fixture.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Contributor
Author
|
Superseded by #3 (includes this fix commit); closing to avoid duplicate merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A
sysbenchrun that exits 0 but emits noevents per secondfield produced a results row likeCPU single thread: events/s, p95 1.00 ms. The scorer'sfirst_numthen grabbed the 95 inside thep95label and counted it as CPU throughput — a missing measurement scored as a real,comparable=trueFULL result.Fix: each sysbench metric now validates its parsed field before being recorded — missing/empty EPS or memory-throughput output marks the metric
FAILED; no events-per-second field, so the score degrades toPARTIAL / not comparableinstead of silently borrowing digits from display text.Changes
is_numhelper;cpu-1t,cpu-all,mem-read,mem-writeblocks check the extracted value beforeadding itmemspeedpreviously recorded a blank row)test_missing_cpu_eps_cannot_score_from_p95_label: p95-only sysbench fixture → assertsstatus=partial,comparable=false,cpuinmissing_components, and both CPU metrics FAILEDshellcheck abs.shclean;python3 -m unittest: 19/19 pass (was 18 + 1 new).Devin session: https://app.devin.ai/sessions/c74df4e507504d63aa16cf898640ae58
Requested by: BlueBirdBack