Skip to content

ci: split PR benchmarks by topic#2303

Open
zeitlinger wants to merge 1 commit into
mainfrom
ci/pr-benchmark-topic-split
Open

ci: split PR benchmarks by topic#2303
zeitlinger wants to merge 1 commit into
mainfrom
ci/pr-benchmark-topic-split

Conversation

@zeitlinger

Copy link
Copy Markdown
Member

Summary

Replace the parallel base/head PR benchmark comparison with topic-split benchmark jobs. Each topic compares base and head sequentially on the same runner, while topics still run in parallel for better wall-clock time.

Changes

  • Split PR benchmarks into topic jobs: counter, histogram, and exposition.
  • Run base and head sequentially inside each topic job so each comparison uses the same runner.
  • Merge per-topic JMH JSON results in the summary job before generating the PR comment.
  • Add a comparison note clarifying that base/head are compared on the same runner within each topic.
  • Add histogram benchmark variants for classic-only histograms:
    • shared contended 4-thread benchmark remains as prometheusClassic
    • single-thread benchmark: prometheusClassicSingleThread
    • per-thread 4-thread benchmark: prometheusClassicPerThread

Why

Running base and head in separate jobs was faster but not reliable enough because GitHub Actions can schedule them on different CPU models. Topic splitting keeps the reliable same-runner comparison within each topic while still allowing unrelated benchmark topics to run in parallel.

Validation

  • Ran mise run lint:fix.
  • Ran mise run build.
  • Ran python3 -m py_compile .mise/tasks/generate_benchmark_summary.py.
  • Ran git diff --check.
  • Verified the JMH include patterns select the intended benchmarks with java -jar benchmarks/target/benchmarks.jar -l ....
  • Smoke-tested the topic result merge and summary generation using the latest perf: optimize classic-only histogram observe path #2252 benchmark artifact.

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant