Repository navigation
Description
Problem, by example
An exact COUNT(DISTINCT user_id) over 12,000,000 rows of 1,000,000 distinct users, ungrouped:
SELECT COUNT(DISTINCT user_id) FROM http WHERE ts >= now() - INTERVAL '20 minutes'The cost (exact_distinct_value_operations_per_row in crates/cost/src/node/operations_per_unit.rs) charges a cache-miss term for its value table:
207 × cache_miss_fraction(1,000,000 values, EXACT_DISTINCT_VALUES_IN_CACHE = 200,000)
= 207 × (1 − 200,000 / 1,000,000) = 207 × 0.8 = 166 operations per row
The 200,000 (values whose table still fits in the cache) and EXACT_DISTINCT_PAIRS_IN_CACHE = 500,000 ((group, value) pairs) were fitted on one machine type: 35 MB of L3 cache per socket. They are constants, so every machine gets the same knee:
| machine | value table fits up to | charge for the example |
|---|---|---|
| the fitted machine (35 MB L3 per socket) | 200,000 values | 166 per row for misses |
| a machine with half the cache | about 100,000 values (expected) | still 166 per row; should be about 186 (0.9 miss) |
| a machine with 4× the cache | about 800,000 values (expected) | still 166 per row; should be about 41 (0.2 miss) |
The rows for other machines are what the miss form predicts, not measurements; no other machine has been measured.
The measured time also moves with how busy the shared cache is. The same shape measured 393.5 ns per row on an idle core and 553.6 ns per row while other builds shared the cache (1.4×). By service (100 groups) it measured 510 against 653.
Why it happens
- The exact distinct-count cost (refit in the exact
COUNT(DISTINCT)refit PR, fix(cost): charge an exact distinct count per value and pair lookup that misses the cache; declare the production workloads' column-wide distinct users #1368) models a lookup into a hash table that has outgrown the cache asmax(0, 1 − K / entries).Kis a property of the machine (cache size, entry size), not of the algorithm. Kis stored as twoconsts inoperations_per_unit.rs. Unlike other per-operation costs, nothing measures it per machine. Their docs say they are machine-specific defaults.- The fit only weakly pins the pair knee: the best value was about 3,000,000 pairs on one grid and 200,000–1,000,000 on the other, with errors within 0.01 of each other. Both grids ran on the same CPU model, so "across grids" was not a cross-machine check.
Proposal
- The calibration harness (
asap_run calibrate, [Design] asap_run: one binary to execute plans, calibrate measured unit costs per machine, and bench predicted vs measured #1317) measures the two cache sizes per machine key. It runs exact distinct counts over a small grid of distinct values (1e3 to 1e7) and group counts, and fitsKfor the value table and for the pair set. It writes both to<machine-key>/costs.jsonnext to the other measured unit costs. CostInputscarries them.exact_distinct_value_operations_per_rowreads them from there, and the two constants stay only as the defaults used when no calibration exists (with the existing missing-calibration warning).- Alternative: derive
Kfrom the machine's cache size and the executor's entry size, without a measured fit. It is simpler, but it misses the effect of a busy shared cache and of the hash table's layout.
Second dependency: value skew
The fit also assumes values are uniform. The measurement grid draws user_id uniformly from U values (the expected distinct and pair counts are computed that way), so every value is equally likely to be a cache miss once the table outgrows the cache.
Example: SELECT service, COUNT(DISTINCT user_id) FROM requests GROUP BY service over 12M rows and 1M users.
- Uniform users: the value table holds 1M entries, the miss fraction is 1 − 200,000 / 1,000,000 = 0.8, and the charge includes 207 × 0.8 = 166 operations per row for misses.
- Zipf-skewed users (a few heavy users make most requests): the same 1M entries exist, but most lookups hit the hot entries already in cache. The real miss rate is well below 0.8. The charge stays at 166, so exact is over-charged and the planner leans toward a sketch.
This has not been measured.
Proposal (skew)
- Add skew as a grid dimension: uniform, plus Zipf with exponent 0.8, 1.0 and 1.2, over the same distinct values, rows and group counts.
- If predicted ÷ measured leaves 0.76–1.32 under skew, make the miss fraction depend on skew. Use the fraction of lookups that land on values outside the
Khottest, instead of1 − K/x. Under a Zipf law this fraction has a closed form in the exponent. - The planner then needs the skew as a data fact. Following the rule for undeclared facts, use a written default (uniform, the conservative choice for exact) and report it in
SelectedPlanReport.assumed_data_factswhen it is not declared. - Done (skew part) when the skewed grid is measured, and predicted ÷ measured stays within 0.76–1.32 on it, with or without the new fact.
Scope and acceptance
- In scope: the two exact distinct-count cache sizes. Other constants fitted on one machine may follow the same path, tracked in [Design] Price CPU and latency from a measured cost table per state family (scan, update, merge), not built-in unit costs #1314.
- In scope: value skew (see above).
- Out of scope: modelling cache contention from other workloads.
- Done when a calibration run on a second machine type writes both sizes, the planner reads them, and the predicted ÷ measured time of the exact distinct-count grid stays within 0.76–1.32 on that machine (the range on the fitted machine).
Related
- fix(cost): charge an exact distinct count per value and pair lookup that misses the cache; declare the production workloads' column-wide distinct users #1368: the refit that introduced the two constants.
- [Design] asap_run: one binary to execute plans, calibrate measured unit costs per machine, and bench predicted vs measured #1317:
asap_run, the calibrate subcommand that would measure them. - [Design] Price CPU and latency from a measured cost table per state family (scan, update, merge), not built-in unit costs #1314: a measured cost table per state family.
Activity
zzylol commentedon Nov 16, 2026
Skew measurement (no code change)
Under Zipf skew the cost charged for an exact
COUNT(DISTINCT)(main 79d9932) is too high by up to 5×. The skew fact is needed.Setup
Example (U = 1e7, no grouping), in ns/row:
Measured ÷ charged over the whole grid:
What this shows:
Candidate formula (variant C). It keeps the merged form and takes everything from (U, s, rows, groups G):
D and P match the counts actually seen to within 0.05%. At s = 0 every part reduces exactly to the current formula.
Leave-one-setting-out over 155 points (this grid plus the two uniform grids):
C does not yet meet the 0.76–1.32 acceptance. The residual follows s:
The hit mass of the hottest K values looks too optimistic at s = 0.8. A fitted factor on K under skew may close the gap; it is untested.
Next
assumed_data_facts.