Repository navigation
Description
Example
A daily distinct-user count over the synthetic clicks table (2,000,000 rows/s, user_id declared with 5,000,000 distinct values), every 60 s:
SELECT COUNT(DISTINCT user_id) FROM clicks
WHERE ts >= now() - INTERVAL '86400 seconds'On main 2f3664d1f, with the sketch-heavy study's planning driver (default deployment and cost model), every HLL candidate is pruned with NoGuarantee, and KMV is selected. The table varies the target and the source's declared max_distinct_per_evaluation:
| Target | max_distinct_per_evaluation |
Selected | Cost /s | HLL prune causes |
|---|---|---|---|---|
| relative 0.02, no δ (the workload as written) | not declared | KMV k = 1,118 | 184.12 | NoGuarantee × 4 |
| relative 0.05, δ 0.01 | not declared | KMV k = 4,323 | 201.04 | NoGuarantee × 4 |
| relative 0.05, δ 0.01 | 5,000,000 | KMV k = 4,323 | 201.04 | NoGuarantee × 4 |
| relative 0.05, δ 0.01 | 9,000 | HLL precision 18 | 196.72 | NoGuarantee × 1, BelowRequiredAccuracy × 2 |
| relative 0.05, no δ | 9,000 | HLL precision 17 | 169.97 | NoGuarantee × 1, BelowRequiredAccuracy × 1 |
So HLL is selected only when the source declares a bound small enough for the classic certificate. A real distinct count (millions of users) never gets HLL, whatever its precision. The workload is tools/dag-viewer/workloads/sketch-heavy/blocked-sql-distinct-users-no-delta-hll-shape__2000000rps-86400s-every60s-releps0.02.json, added by #1388. The other rows are the same file with the target or the bound changed.
Why it happens
- Without a usable bound, the failure probability is unknown.
guarantee::hll::guarantee(accuracy/src/guarantee/hll/mod.rs:19-38) returns classic HLL's certified guarantee only when the source declaresmax_distinct_per_evaluationand the bound is inside the certificate's contract (classic_guarantee_at, line 54). Otherwise it returns the relative standard error1.04 / √mwithProbabilityExpr::Unknown { statistic: "hll_estimator_failure_probability" }. - Pass 2 rejects an unknown failure probability. Pass 2's edge rule (
accuracy/src/logical_edge/mod.rs:136-149) rejects a readout whose failure probability does not evaluate:NoGuarantee. It does so even when the target sets no δ. - The certificate covers only small populations. It comes from HLL's linear-counting branch. At precision 18 and δ = 0.01, a bound of 9,636 certifies a relative error just under 0.05, and a bound of 1,000,000 certifies none (the doc example on
classic_guarantee_at). Distinct counts in the millions are outside it at every listed precision. - What already works. The
max_distinct_per_evaluationcertificate path exists and works, as the last two rows show.
Proposal
- Measured accuracy (P3). The measured sketch accuracy table (feat(accuracy, cost, devtools): measured sketch accuracy table; delete the cost table's provenance and legacy format #1372) gives
AccuracyEvidence::measured_guarantee, which Pass 2 already consults before the theoretical guarantee (logical_edge/mod.rs:115). A measured HLL row for the precisions and cardinalities of interest, at the requested failure probability, makes HLL selectable for large populations without a closed-form certificate. - A certificate for the full estimator. Alternatively, certify the full estimator, not only linear counting, under explicit assumptions ([Design] Add an estimator-aware HLL failure-probability model #459).
Whichever path lands, the NoGuarantee rule stays for HLL configurations that neither covers.
Scope and acceptance
- Re-check after feat(accuracy, cost, devtools): measured sketch accuracy table; delete the cost table's provenance and legacy format #1372 lands. Re-run the example and the table above. If HLL readouts for the 5,000,000-distinct source then have a measured guarantee, this issue reduces to pinning that with a test.
- Acceptance. For the example at relative 0.05, δ 0.01:
- the HLL candidates are not pruned
NoGuarantee; - Stage 3 compares HLL with KMV, and selects HLL where it is cheaper;
- a
sketch_plan_snapshotsentry pins the result.
- the HLL candidates are not pruned
- Out of scope. HLL++ or sparse-mode variants that the executor does not build.
Related
- [Design] Add an estimator-aware HLL failure-probability model #459: the HLL failure-probability model design.
- feat(accuracy, cost, devtools): measured sketch accuracy table; delete the cost table's provenance and legacy format #1372 (P3): the measured accuracy table that partly resolves this.
- Found by the sketch-heavy workload study.
Activity