Repository navigation
One script for the synthetic use cases; systems runner takes any spec, error budgets and raw samples - #248
Merged
Conversation
--spec runs a spec file on VictoriaMetrics and ASAPQuery-backend; --arm picks the error budgets. Queries that do not plan together plan alone, one instance each; queries the planner or the data plane refuses are recorded as unsupported. Each budget writes a summary with per-query agreement against VictoriaMetrics and the planner's time and memory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oards run_all_use_cases.sh builds planeval, runs the use cases one after another and starts Grafana. tools/grafana/run.sh keeps Grafana's state and the Infinity plugin under target/grafana-home and finds Grafana through Homebrew or GRAFANA_HOME. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and current dashboards Arms that keep raw samples tell the planner so and give the ASAPQuery-backend data plane a VictoriaMetrics instance holding the raw samples. The docs use the scripts/use_cases paths and the canary-release dashboard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GordonYuanyc
added a commit
that referenced
this pull request
Nov 16, 2026
… needless_range_loop Clippy in Rust 1.100 rejects indexing self.quantiles by the loop variable, which fails format-and-lint on main since #248 and on this branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GordonYuanyc
added a commit
that referenced
this pull request
Nov 16, 2026
… by default (#251) * grafana: describe each use case's data and rules on its dashboard; synthetic runs only by default Each use case dashboard now opens with its scenario figure, a plain description of the synthetic data (every column, its usual values and the events injected over time), and every query and rule with what it does. The rule dropdown names the threshold rules by their condition, entity dropdowns use each use case's own word (server, container, subpopulation, client), and injected problem periods and the times approx decides differently from exact are always drawn on the time charts. generate_dashboards.py takes --phase, --out, --no-figure and --real-data; by default it shows synthetic runs only, which is what run_all_use_cases.sh produces. run.sh takes the same --phase and --real-data and serves the scenario figures on port 8001. The overview table gains its own row, split plan columns and a target error column; panel heights follow their text. Dataset and policy come from the run name, so the dashboards read only fields the current runner writes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * polars_shared: fill the quantile array through iter_mut, for clippy's needless_range_loop Clippy in Rust 1.100 rejects indexing self.quantiles by the loop variable, which fails format-and-lint on main since #248 and on this branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
scripts/use_cases/run_all_use_cases.sh(new): buildsplaneval, runs each use case on its synthetic workload one after another, writes the rows totarget/grafana-data, then serves the dashboards.--use-case 1,2,...,--quick(one approximate arm per use case) and--no-grafana. On an 8-core laptop: build about 11 min,--quickabout 11 min, every arm about 1 h 40 min; peak memory under 4 GB.tools/grafana/run.sh: Grafana's state and the Infinity plugin live undertarget/grafana-homeinstead of the user's Grafana; Grafana is found through Homebrew orGRAFANA_HOME;--checkinstalls the plugin and exits.scripts/use_cases/run_use_case_on_systems.py:--specruns any spec file and--arm(repeatable) sets the error budgets; VictoriaMetrics runs once per spec. Arms that keep raw samples tell the planner so and give the ASAPQuery-backend data plane a VictoriaMetrics instance holding the raw samples.docs/aqpbm-planeval.md,docs/phase2-use-cases/canary.md: script paths underscripts/use_cases/, and the current dashboard URL.Checked:
run_all_use_cases.shwith--quickand with every arm from an emptytarget, all 5 phase 2 dashboards return rows;test_run_use_case_on_systems.pypasses.🤖 Generated with Claude Code