Repository navigation
Benchmark sharding basic functionality #1813
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
63 commits
Select commit
Hold shift + click to select a range
1c7665d
Cached benchmark IO sketch
cmdupuis3 787e2d4
benchmark debugging
cmdupuis3 96ce618
Sample large benchmarks fewer times
cmdupuis3 4060897
Temp fork/thread speedup for connectivity. REEVALUATE OR REMOVE WHEN …
cmdupuis3 7ab239d
Cache benchmark sampling across forks too
cmdupuis3 c63676c
Per-resolution async benchmarks
cmdupuis3 85f7c0c
Benchmark IO caching fixes
cmdupuis3 9cd8fa8
cached IO benchmarks deslopping
cmdupuis3 b1a6be6
cached IO deslop global variables
cmdupuis3 41f68b2
bench cached IO: more global variable deslopping
cmdupuis3 4dad004
Prime the fixture cache before asv run, not during it
cmdupuis3 60cf1db
Merge branch 'main' into cmd/bench_io
cmdupuis3 ec83767
Lazy neighborhood filter kernel compilation
cmdupuis3 8564410
Lazy nb kernels with functools cache instead
cmdupuis3 7127726
Move kernels inside
cmdupuis3 d1ab4e0
merge cmd/lazy_nb_kernels to cmd/bench_io
cmdupuis3 c00fd1d
Cached IO: try to impose more CI thread safety
cmdupuis3 526e320
Cached IO: lazy interpreter warming; NetCDF warming
cmdupuis3 a064b30
Cached IO: configuration tweaks
cmdupuis3 b17b7a5
Merge branch 'main' into cmd/bench_io
cmdupuis3 a0f4a80
Partially revert a064b30, add cpu info dump
cmdupuis3 f54d634
Cache the asv environment and fixtures, and report durations
cmdupuis3 8f66e30
Fix 'run benchmark' action for manual runs
cmdupuis3 3b46b08
Cache the asv environment and fixtures on main too
cmdupuis3 04f96f2
Split the suite into shards of roughly equal cost
cmdupuis3 f9e5546
ASV workflow branch for pre-caching IO
cmdupuis3 966e5a8
ASV benchmarks skip existing commits
cmdupuis3 6ad7a7f
ASV diff against merge base rather than main
cmdupuis3 d472982
ASV multithreading
cmdupuis3 dbbd6ca
Merge branch 'UXARRAY:main' into cmd/bench_io
cmdupuis3 aaf1dd8
Merge branch 'cmd/bench_io' into cmd/bench_shards
cmdupuis3 5114500
Generalize caching across machine names
cmdupuis3 4f044ff
Fully async sharding (?)
cmdupuis3 8da3f54
Benchmark sharding fix
cmdupuis3 f7d8118
Sharding fix 2
cmdupuis3 99c49b5
Rework shard partitioning
cmdupuis3 8428e1d
Load balancing and removing thread sweeps
cmdupuis3 6bcf1e7
HPC benchmark scipts
cmdupuis3 63dfedb
HPC bugs
cmdupuis3 4803894
Benchmark machine names
cmdupuis3 6a4882d
Thread/core numbers for benchmarks
cmdupuis3 cd8d5ed
ASV machine bug
cmdupuis3 98e9c9d
benchmark shard bug
cmdupuis3 8066798
Merge main into cmd/bench_io
cmdupuis3 f417df8
Cached benchmarks IO deslop p1
cmdupuis3 7bb949a
Cached benchmark IO deslop p2
cmdupuis3 353805c
Cached benchmark IO deslop p3
cmdupuis3 937fb06
Merge branch 'main' into cmd/bench_io
cmdupuis3 152e418
Merge branch 'main' into cmd/bench_io
cmdupuis3 5cfac6d
Merge origin/main into cmd/bench_shards
cmdupuis3 7c1a1cc
Merge cmd/bench_io into cmd/bench_shards
cmdupuis3 2739e6f
Benchmark shards: asv run instead of asv continuous
cmdupuis3 cc1d850
Drop merge leftovers in gcagca and mpas_ocean benchmarks
cmdupuis3 aa5a267
Condense comments and docstrings added on the benchmark sharding branch
cmdupuis3 189e78f
Trim neighborhood benchmark radii
cmdupuis3 dc3df00
Simplify the benchmark sharding helpers, workflow and HPC scripts
cmdupuis3 d136d95
Remove hpc scripts (maybe re-add on a new branch)
cmdupuis3 c88d68d
Build neighborhood benchmark setups once per process; seed PR duratio…
cmdupuis3 00506a1
Simplify the shard merge; restore the results artifact's runner.os name
cmdupuis3 41913f8
Weigh shards by the median recorded duration
cmdupuis3 5abfba4
Read result files' duration column directly in load_weights
cmdupuis3 63f795c
Name plan's groupings for what they hold
cmdupuis3 7a82d11
Merge branch 'main' into cmd/bench_shards
cmdupuis3 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.