Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 35 additions & 4 deletions .github/workflows/perf-gate.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: Perf gate

# Hot-path instruction counts, PR against its merge base
# Hot-path instruction counts, PR against its merge base and against the last
# release tag, a cumulative budget per release cycle
# (benchmarks/hotpath.py gate; docs/TESTING.md, "Performance gate").
# The workflow always runs so the check can be required; the gate itself is
# skipped (and passes) unless runtime source, dependencies, toolchain or driver changed.
Expand All @@ -9,6 +10,10 @@ on:
pull_request:
branches: ["main"]
merge_group:
# On main only the release binary is built (when missing) and cached, so PRs,
# which can only restore caches saved on main, reuse it.
push:
branches: ["main"]

concurrency:
group: perf-gate-${{ github.event.pull_request.number || github.ref }}
Expand All @@ -19,11 +24,13 @@ env:
CARGO_INCREMENTAL: "0"
# Maximum growth of instructions per operation, in percent.
HOTPATH_THRESHOLD: "3"
# Maximum growth over the last release (git describe --match 'v*'), in percent.
HOTPATH_RELEASE_THRESHOLD: "3"

jobs:
perf-gate:
runs-on: ubuntu-latest
timeout-minutes: 40
timeout-minutes: 50
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -34,6 +41,14 @@ jobs:
PR_BASE: ${{ github.event.pull_request.base.sha }}
MQ_BASE: ${{ github.event.merge_group.base_sha }}
run: |
release=$(git describe --tags --abbrev=0 --match 'v*' HEAD)
echo "release=$release" >> "$GITHUB_OUTPUT"
echo "release_sha=$(git rev-parse "$release^{commit}")" >> "$GITHUB_OUTPUT"
if [ "${{ github.event_name }}" = push ]; then
echo "run=false" >> "$GITHUB_OUTPUT"
echo "warm=true" >> "$GITHUB_OUTPUT"
exit 0
fi
base=$(git merge-base HEAD "${PR_BASE:-$MQ_BASE}")
echo "base=$base" >> "$GITHUB_OUTPUT"
changed=$(git diff --name-only "$base" HEAD)
Expand All @@ -44,13 +59,29 @@ jobs:
echo "Perf gate skipped: no runtime source, dependency, toolchain or driver change." >> "$GITHUB_STEP_SUMMARY"
fi
- uses: dtolnay/rust-toolchain@1.93.0
if: steps.scope.outputs.run == 'true'
if: steps.scope.outputs.run == 'true' || steps.scope.outputs.warm == 'true'
# The release binary: same driver, same toolchain, same release commit.
- name: Release binary cache
if: steps.scope.outputs.run == 'true' || steps.scope.outputs.warm == 'true'
uses: actions/cache@v4
with:
path: ${{ runner.temp }}/hotpath-release
key: hotpath-release-${{ steps.scope.outputs.release_sha }}-${{ hashFiles('benchmarks/hotpath/driver.rs', 'rust-toolchain.toml') }}
- name: Build the release binary
if: steps.scope.outputs.warm == 'true'
run: >
python3 benchmarks/hotpath.py build-release
--release ${{ steps.scope.outputs.release }}
--release-cache ${{ runner.temp }}/hotpath-release
--work ${{ runner.temp }}/hotpath
- name: Install valgrind
if: steps.scope.outputs.run == 'true'
run: sudo apt-get update -q && sudo apt-get install -y -q valgrind
- name: Instruction counts, head vs merge base
- name: Instruction counts, head vs merge base and last release
if: steps.scope.outputs.run == 'true'
run: >
python3 benchmarks/hotpath.py gate
--base ${{ steps.scope.outputs.base }} --head HEAD
--release ${{ steps.scope.outputs.release }}
--release-cache ${{ runner.temp }}/hotpath-release
--work ${{ runner.temp }}/hotpath
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,15 @@
non-blocking disposal and checks the queue drains and every worker retires
after its keep-alive (`disposal.keep_alive` failpoint with `failpoints`).
The soak harness judges the three gauges.
- `perf-gate` also holds every PR to a cumulative budget: the head may grow
at most 3% instructions per operation over the last release tag
(`git describe --match 'v*'`), not only over its merge base, so small
per-PR regressions cannot add up across a release cycle. The job summary
table shows release, base and head with both changes. The release binary
is built once per release and cached (`hotpath.py build-release` on pushes
to main). `threads_request_seq`, whose thread
interleaving varies on a loaded host, takes the median of nine run pairs
instead of three.
- Fix the flaky `supervision::transient_startup_timeout_retries_after_actual_cleanup`
(a 20 ms startup deadline raced the fast starts on a loaded host, which also
made cargo-mutants report surviving mutants as caught): the replacement's
Expand Down
4 changes: 3 additions & 1 deletion benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,9 @@ python3 benchmarks/hotpath.py abba --cores 12-15 --reps 30 # wall clock, before
```

`gate` is the `perf-gate` CI check: it fails when instructions per operation
grow more than 3% over the merge base. `abba` alternates base and head on the
grow more than 3% over the merge base, or more than 3% over the last release
tag (`--release`; the cumulative budget per release cycle, so small per-PR
regressions cannot add up). `abba` alternates base and head on the
same cores and reports the median and spread. See
[performance gate](../docs/TESTING.md#performance-gate).

Expand Down
Loading
Loading