IBMCEPH-17498: implement Elbencho S3 run loop and pdsh command builder (Story 3) - #360
Open
gitkenan wants to merge 3 commits into
Open
IBMCEPH-17498: implement Elbencho S3 run loop and pdsh command builder (Story 3)#360gitkenan wants to merge 3 commits into
gitkenan wants to merge 3 commits into
Conversation
…are factory bypass Add benchmark/elbencho.py (Elbencho class), wire it into benchmarkfactory.py with _WORKLOADS_BENCHMARKS guard to prevent all_configs() Cartesian expansion, and cover both with 25 unit tests in tests/test_bm_elbencho.py. Signed-off-by: Kenan Al-Shamie <kenan.al-shamie@ibm.com> Assisted-by: Bob 2.0
This was added to elbencho as part of the integration work, during which it was realised that this kind of validation also suits hsbench due to it also being a package external to tools included with Linux Assisted-by: Bob 2.0 Signed-off-by: Kenan Al-Shamie <kenan.al-shamie@ibm.com>
…ory 3)
Three-tier nested loop (blocksize → threads → iodepth) in _run_workloads(),
fanning out one elbencho process per client node via pdsh for each run cell.
_build_elbencho_cmd() assembles the full CLI string from workload params:
all YAML schema keys map to their documented elbencho flags, --s3region is
always emitted (defaulting to 'default'), and the bucket path is suffixed
with $(hostname -s) at shell-evaluation time to prevent silent object-key
collisions when multiple clients write to the same bucket concurrently.
stat and list modes are skipped with a logged warning per Story 5's deferral
decision. The run directory follows the {mode}_{bs_bytes}/threads-NNN/
iodepth-MMM/ convention so the formatter (Stories 4/5) can reconstruct all
test parameters from the path without re-reading the YAML.
32 new tests across TestParseBlocskizeToBytes, TestBuildAuthFlags,
TestBuildElbenchoCmd, and TestRunLoop. Total: 54 tests, all passing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Jira: IBMCEPH-17498 | Epic: IBMCEPH-15080 — Story 3 of 11
This fills in the _run_workloads() stub from Story 2. The three-tier nested loop (blocksize → threads → iodepth) now runs for real: for each cell it creates the run directory, builds the elbencho shell command, and fans it out to all client nodes via a single pdsh broadcast. The one non-obvious decision is the namespace-collision fix — rather than looping per-host, the bucket path is suffixed with $(hostname -s) inside the command string so the remote shell expands it independently on each node, matching the pattern already used by rbdfio for RBD volume naming.
stat and list modes are skipped with a warning per Story 5's explicit deferral decision; they produce no throughput or blocksize metrics so the formatter isn't ready for them yet.
32 new tests cover the blocksize parser, auth flag builder, command builder, and run-loop call count/path structure. Total test count: 54, all passing.