Declare named and filtered thresholds in the v1 report payload

[epompeii]

Built on #1075, which counts a report's new thresholds against the creation ceiling. The entries this layer adds are more creates in that same count.

The list shape

A report has always carried its thresholds as a map of measure to model, and a map key names a measure and nothing else. Every threshold a report could declare was therefore the bare one: the conventional value name of every variant.

At BMF version 1 thresholds.models is a list:

{
  "bmf_version": 1,
  "thresholds": {
    "models": [
      { "measure": "latency", "metric": "p99", "model": { "test": "t_test", "upper_boundary": 0.98 } },
      { "parameters": [{ "size_mb": 16 }], "measure": "latency", "metric": "value", "model": { "test": "percentage", "upper_boundary": 0.25 } }
    ]
  }
}

An entry is parameters, measure, metric, model: the dimensions a threshold hangs off that the report does not already state, in their canonical order, and the model to check with. measure is the same name, slug, or UUID the map key is today, and a name or slug the project has never seen creates the measure. metric is required, and value names the conventional metric; the value default is only for version 0's map. An absent parameters checks every variant, so an entry naming a measure, value, and a model is the map pair written out longhand. One measure may carry several entries, and each creates or updates the threshold with those dimensions under the report's branch and testbed, through the same null collapse and canonical filter storage every other writer goes through.

A threshold the report declares checks the very report that declared it. Nothing moved to make that true: thresholds are resolved before results are parsed, which is where they already were. It is pinned by a test whose history is five unchecked reports and whose alerts all belong to thresholds that did not exist when the request arrived.

One shape per version

The report's BMF version, the one it declares or else the project's default, says which shape to expect, and only that shape is accepted. A list at version 0 and a map at version 1 are both a 400 naming that version and the shape it calls for, an empty map at version 1 included, while a list whose entry is malformed, an entry with no metric included, is refused first by that entry's field error. The check runs before anything is created for the report. A project whose bmf_version is 1 takes its thresholds as a list, and a test pins that a report declaring no version takes its shape from the project.

One set of dimensions is one threshold

One set of dimensions declared twice in one version 1 list is one threshold: the position is where it was first written and the model is what it was last told, resolved without an error. Two spellings of one filter address one threshold, because a filter canonicalizes before it is compared. A version 0 map merges through the same path, so a measure keyed by both its name and its UUID is one threshold rather than a failed insert; which of its two models wins is not defined. The creates a payload plans count against the threshold creation ceiling after they merge, in one check whichever shape they arrived in.

Reset

reset means one thing at both versions: it takes the model away from every threshold on the report's branch and testbed that the payload does not name, including a payload that names none at all. A version 0 reset therefore reaches the named and filtered thresholds its map has no way to spell, where before it reached only the bare ones. reset stays on the report's own branch and testbed.

Adjudicated calls worth a second opinion

• The entry nests model rather than flattening it. JsonNewThreshold flattens its Model, which is the sibling precedent. The entry is the map pair unrolled into fields, and the map's value is the model, so model is a key here. It also keeps the entry legible beside metric and parameters.
• Deserialization is a hand-written visitor, not serde(untagged). An untagged enum buffers the input and, on failure, discards every inner error for one opaque sentence. The two shapes are distinguished by the first JSON token, so the visitor dispatches a map to the map's own deserializer and a list to the list's, and their field-level errors survive verbatim, path prefixes included. Two messages change. A models value that is neither shape is now told the field expects a map of measure to threshold model (BMF version 0) or a list of threshold entries (BMF version 1); the old text named only the map, and a test pins the new sentence. A list sent at version 0, which serde used to refuse as invalid type: sequence, expected a map, now parses and is refused by the shape check, or by its entry's field error when an entry is malformed.
• A shape-refused payload creates nothing for the report. The shape check runs right after the idempotency check, ahead of branch and testbed creation, and the list and map refusal tests assert that the report's branch and testbed were never created.
• The two shapes are a hand-written oneOf, not a derived anyOf. A derived untagged enum emits anyOf, and the client generator turns an anyOf of two subschemas into one struct of flattened optional members. A list cannot be flattened into a struct. oneOf is also what the shapes are: mutually exclusive. The generated client gets a clean untagged enum with Map and List arms, named from the schema titles.
• The CLI construction site changed mechanically (models: models.map(JsonReportThresholdModels::Map)), and the payload it sends is byte identical. The CLI declares bare thresholds, which is what the version 0 map spells, so it does not yet send thresholds to a project whose bmf_version is 1. --thresholds-reset follows reset, and its help no longer carves out named and filtered thresholds. Naming a metric or a variant from the command line is a separate piece of work.
• Creation order within one version 1 list is the list's order. A test pins it.

Version 0 behavior

Version 0 compatibility was capture-proven at 2927c700: the map ingest, map update, and map-with-reset responses, and the malformed-map 400 bodies with their field paths, were captured on the base branch and on that head and diffed empty, apart from the two messages above. Since then two version 0 behaviors changed on purpose, each pinned by a test: reset reaches named and filtered thresholds, and a map that names one measure twice merges instead of failing on the unique index.

Gates

Locally, at 749c1a55 on devel at #1077's merge: cargo fmt -- --check, cargo clippy --no-deps --all-targets --all-features -- -Dwarnings, cargo nextest run --all-features --profile ci -p bencher_schema -p api_projects -p bencher_json -p bencher_adapter -p api_run -p api_runners -p bencher_cli -p bencher_comment, doc tests, cargo check --no-default-features, and cargo gen-types with only the intended openapi.json change. Rebased since onto #1083's merge as 9dcb9989, where fmt and clippy pass. No migration in this layer.

[github-actions]

Bencher Report

Project	Bencher
Branch	u/ep/parameters-api/thresholds-payload
Testbed	intel-v1

Click to view all benchmark results

Benchmark	Latency	Benchmark Result microseconds (µs) (Result Δ%)	Upper Boundary microseconds (µs) (Limit %)
Adapter::Json	📈 view plot 🚷 view threshold	5.25 µs (+4.64%) Baseline: 5.02 µs	6.10 µs (86.12%)
Adapter::Magic (JSON)	📈 view plot 🚷 view threshold	5.06 µs (+4.45%) Baseline: 4.85 µs	5.77 µs (87.68%)
Adapter::Magic (Rust)	📈 view plot 🚷 view threshold	27.12 µs (+0.77%) Baseline: 26.91 µs	30.13 µs (90.01%)
Adapter::Rust	📈 view plot 🚷 view threshold	4.81 µs (+12.73%) Baseline: 4.27 µs	6.59 µs (73.02%)
Adapter::RustBench	📈 view plot 🚷 view threshold	4.82 µs (+13.00%) Baseline: 4.26 µs	6.58 µs (73.20%)

🐰 View full continuous benchmarking report in Bencher

[epompeii]

Review after the rebase onto devel

Devel retired the BMF project version gate: a project's bmf_version is now the default for a report that declares none, never a ceiling. This layer was reworked for that. The gate refusal test is gone, the thresholds shape check runs right after the idempotency check (so a refused payload still creates nothing), and version comparisons use equality. Two independent reviews and an adjudication agree on all three.

Major: the inherited project version picks the thresholds shape and the reach of reset

report/mod.rs resolves json_report.bmf_version.unwrap_or(query_project.bmf_version) and hands that version to both check_models_shape and the reset query in threshold/mod.rs. The CLI never declares a version (every report path sends bmf_version: None) and always sends thresholds as a map. On a project whose bmf_version an admin set to 1, which is the only CLI route to BMF v1:

• bencher run --threshold-* is a 400: "The report payload declared BMF version 1, so thresholds.models must be a list of threshold entries, but a map was sent." The payload declared nothing, and the CLI cannot send a list. The same request succeeds on devel. Reproduced against a running server.
• bencher run --thresholds-reset passes the shape check, then runs reset at version 1 reach, and returns 201 after stripping the model from every threshold on the branch and testbed, named and filtered ones included. Reproduced against a running server.
• A payload that explicitly declares version 1 and sends a map is also refused, where devel accepts it.

No test sends the CLI's shape: every report in report_thresholds.rs declares bmf_version.

Recommended fix, in line with accepting input rather than rejecting it: accept a map as bare thresholds at every version; accept a list unless the payload explicitly declares version 0; let reset reach only what the sent shape can address, and let a models-free reset reach every threshold only on an explicit version 1. Pin it with tests that omit bmf_version on a version 1 project, in exactly the CLI's shape, for a map and for a reset alone. The alternative keeps the refusals but keys the shape check and the reset reach on the explicit declaration only.

Cleanup applied from review meanwhile: the refusal now reads "The report is read as BMF version N", which is true whether the version was declared or inherited.