The parallel-scan numbers describe the fixture the page publishes (#753)

[jdatcmd]

Part of #753. Documentation, plus one new clock-free suite. No src/ change.

The defect

docs/limitations.md publishes a fixture, a narrow query, and a set of speedup
figures. The figures came from a different table.

The page's narrow query was WHERE sel <= 1000000, and sel is the
generate_series counter, so it is stored in order. The zone map excludes 20
of the table's 27 chunk groups before a row is read. The figures beside it came
from a run that explicitly asserts 27 of 27 groups read ("nothing here is
confounded by zone-map pruning", on #753). So the page quoted numbers for about
four times the work its own query does.

#804 saw half of this. It found the published threshold had moved 6x and
attributed that to "fixture details the page omits", which is right. It left the
timings, and those belong to the other fixture.

Measured

PostgreSQL 17.10, pg17_nc, debug_assertions = off asserted rather than
assumed. Fixture built from the SQL the page publishes, which gives 383 MB
against 823 MB, exactly the sizes the page states. 7 interleaved readings per
arm, both arm orders run, Workers Launched == Workers Planned asserted on every
parallel arm.

narrow query filters on	groups read	serial cost	ratio (median)	non-overlap
sel, stored in order	7 of 27	22,428	1.24 to 1.31x	False, both orders
a, unordered	27 of 27	83,305	2.43 to 2.51x	true, margin 1.60x

The page claims 1.8 to 2.5x and non-overlap at 1.45x. That is the second row, not
the first.

Two independent confirmations that the original fixture used an unordered
predicate column: the unordered arm's serial cost is 83,305 against #753's
published 83,196, a 0.1% match; and its serial time is 175.8 ms against that
issue's 177.5 ms minimum.

Three corrections

The narrow query now filters on a, so the query and the figures describe
one table. Thresholds re-swept for it at 0.001 steps: 0.039, 0.053, 0.060 at 2,
3 and 4 workers.

The opening now names a worker count. A wide projection is chosen parallel at
4 workers. At the default of 2 it is not, and the page's own threshold table
names 2 as the default. The opening asserted it unconditionally.

The 0.060 is restored. The page said an earlier edition quoted it and that a
reader "could not reproduce it". It reproduces exactly: this table, 4 workers,
unordered predicate column. It was never wrong. Neither edition recorded which
case it measured, and that omission is the actual defect, both times.

The heap control isolates the cause

	2 workers	3	4
heap, ordered predicate	0.026	0.031	0.035
heap, unordered predicate	0.026	0.031	0.035

Identical, because a heap has no zone map. So the entire columnar split is
pruning rather than anything about the two columns.

The guard, and why it reads the document

docs_style passes on the pre-fix page, so nothing caught this. The new suite
does, and it extracts the filter column from the rendered document instead of
keeping a copy. A suite carrying its own copy of the query cannot see the
document drift away from it, which is precisely this defect. If the page stops
naming a narrow query the extraction fails and the suite goes red rather than
passing on nothing.

It asserts group counts and costs only. It reads no clock, so PGC_SKIP_TIMING
does not apply and it runs on every CI leg.

Removal proof. Against the pre-fix page it fails two distinct checks:

-- the page's narrow query filters on: sel
-- sel (the documented column): 3 of 10 groups read
FAIL  the documented narrow query reads every chunk group: got [3] want [10]
FAIL  the documented column is not the one stored in order

Against the corrected page it passes 7 of 7. harness_selftest is 168 checks and
docs_style is 9, both green with the suite registered.

One check was restructured after its first draft: when the page named the ordered
column, the cost comparison compared a value with itself, which is a check that
cannot pass rather than one that measured anything. It is now an explicit
assertion that the documented column is not the ordered one, with the cost
comparison guarded behind it.

A note for #817, which is not this PR

Building the suite surfaced evidence for #817 that is stronger than that issue
currently states. On a 200,000-row table written with stripe_row_limit = 20000,
at the default plan-time limit the two predicates cost 4212.00 and 4212.00
even though EXPLAIN ANALYZE reports one reading 10 of 10 groups and the other 3
of 10. At the written limit they cost 4212.00 and 1404.00. So the default GUC
does not merely shift the estimate, it removes zone-map pruning from plan choice
entirely.

@OffgridwithJD worked the arithmetic through and identified the mechanism
precisely: ceil(200000/150000) is 2, the pre-fix sample runs g = 0,1, group 0
does not exist, so exactly one group is examined and 1/1 = 1.0 gives no
discount. The suite therefore sets the plan-time limit to the written value, with
a comment naming #817 and stating that the line becomes redundant when that lands.
Their fix touches the sample rather than which limit the site reads, so the line
is still required after it.

What this does not do

It does not touch the cost model, and it does not resolve #753. The half that is
core's, cost_gather charging parallel_tuple_cost * rows blind to tuple width,
is unchanged and unfixable from an extension. The half that is ours is #766.

[jdatcmd]

Review addressed, and the missing removal arm was real

Thank you for all three. The second one was a genuine hole and I have filled it.

1. The product-side removal arm, which I did not have

You were right that checks 2 and 4 are product assertions and that reverting
the page cannot exercise either, so I had not shown check 2 could fail at all.
That is the vacuous-green shape rather than a style point.

Your suggested lever works and needs no C mutation:

qual_pushdown=on   sel reads  7 of 27 groups
qual_pushdown=off  sel reads 27 of 27 groups

Running the whole suite under PGOPTIONS="-c pgcolumnar.enable_qual_pushdown=off":

PASS  premise: the page still names a narrow query
PASS  premise: and that query has an extractable filter column
-- a (the documented column): 10 of 10 groups read
-- sel (stored in order):     10 of 10 groups read
PASS  the documented narrow query reads every chunk group
FAIL  and filtering the ordered column instead prunes some away: got [10 of 10] want [prunes]
PASS  the documented column is not the one stored in order
FAIL  and the documented column is costed above the ordered one: got [no (4212.00 vs 4212.00)] want [yes]

So the removal proof now covers every check, by two different mutations:

mutation	which checks go red
revert the page to its pre-fix text	1 and 3, the document-side checks
enable_qual_pushdown = off	2 and 4, the product-side checks

Neither mutation reds all four, which is what says the four are testing
different things rather than one thing four times.

2. The heap control was measured, and you are right that the page did not show it

It was measured per predicate, not carried over. The numbers existed only in my
sweep output, and the page asserted the conclusion while its only heap figures
sat in a different table with no predicate column. A reader could not check the
load-bearing causal claim of the section.

The contrast table now carries both thresholds at 4 workers:

narrow query filters on	groups read	serial cost	speedup, fastest	speedup, middle	columnar turns parallel at	heap turns parallel at
sel, stored in order	7 of 27	22,428	1.54 to 1.67	1.24 to 1.31	0.015	0.035
a, unordered	27 of 27	83,305	2.54 to 2.67	2.43 to 2.51	0.060	0.035

Columnar moves 0.015 to 0.060 between the two predicates. The heap does not move.
That is the causal claim, now visible rather than asserted.

3. Minimum rather than median

Taken, and it changes the published figures. The page now quotes the fastest
reading and gives the middle one beside it, so the conclusion can be seen not to
depend on which statistic is chosen. The wide query is now 275 ms against 639 ms
rather than 320 against 640.

The page also states the reason in one sentence, because a reader re-measuring
needs to know which statistic to take: scheduling noise only ever adds time, so
the fastest reading is the least polluted.

Both ranges still separate cleanly. sel is 1.54 to 1.67 on the fastest and 1.24
to 1.31 on the middle; a is 2.54 to 2.67 and 2.43 to 2.51. The non-overlap
result is unaffected, since it is already defined as slowest against fastest.

4. On the regex

Keeping it, and thank you for checking pgc_summary really exits rather than
falling through to read_groups "". I had reasoned it did and had not verified
it. #775 staying green 25 of 25 with the guide deliberately broken is the
strongest argument for this shape that either of us has.

docs_style is 9 of 9 and the suite is 7 of 7 after the reword, so the anchor
survived the edit it was most likely to be broken by.

[OffgridwithJD]

Reviewed at 42c9ee1, which is two commits past the head you asked about. Ran
the suite, ran the arm you said was missing, and tested this composed with #821
because both PRs add a SUITES line and this suite asserts a cost #821 changes.

Everything below is on my container against /usr/local/pg18a. Formal approval is
held only because suites (PG 17) and suites (PG 18) are still pending; 9 build
legs and shellcheck pass.

Your question 2 first: the missing arm exists, and it works

You did not need to mutate C. pgcolumnar.enable_qual_pushdown gates both the
executor's scan keys (columnar_reader.c:610) and the estimator
(columnar_reader.c:1617), so one SET is the whole arm. I injected it into your
two EXPLAIN helpers and ran your suite unchanged otherwise:

as submitted            sel reads  3 of 10   serial cost: a 4212.00, sel 1404.00   7/7 PASS
enable_qual_pushdown=off sel reads 10 of 10   serial cost: a 4212.00, sel 4212.00   2 FAIL

Both product-side checks red, and only those two:

FAIL  and filtering the ordered column instead prunes some away: got [10 of 10] want [prunes]
FAIL  and the documented column is costed above the ordered one: got [no (4212.00 vs 4212.00)] want [yes]

So checks 2 and 4 are not vacuous — they can fail, and they fail for the right
reason. The suite does not need changing. I would put those two lines in the PR
body, because "I reverted the page" does not cover the product-side half and a
reader cannot tell from the body that it was checked at all.

Your question 1: keep the regex

It is the mechanism, not a convenience. I have the scar: #775 shipped a suite whose
header said "transcribed from docs/user-guide.md" while nothing read it. I broke
the guide deliberately and the suite stayed green 25 of 25.

It fails in the right direction, and I checked that rather than assuming it.
pgc_summary really does exit 1 on the PGC_FAIL != 0 branch, so your
[ -n "$doc_col" ] || { echo ...; pgc_summary; } terminates instead of falling
through to read_groups "". A reword that drops the anchor reds a premise and
prints the offending line. Red-with-a-diagnostic on a reword beats green-on-nothing
forever.

Composed with #821

Both merge clean onto f64ccbb — run_all_versions.sh auto-merges, which is the
one-name-per-line design doing its job. Merged SUITES is well formed at 233
entries with both new names present.

On the merged tree:

doc_parallel_premise      7/7 PASS   serial cost: a 4212.00, sel 1263.60
zonemap_estimate_sample  26/26 PASS
harness_selftest        168    PASS

Note your sel cost moves 1404.00 -> 1263.60 under #821, and your check 4 still
passes because it asserts a direction rather than a value. That was the right call.
The new number is the more correct one: EXPLAIN ANALYZE says Chunk Groups Read: 3 of 10, and #821 makes the estimate 3/10 where it was 3/9.

The docs, re-read at this head

Both points I raised are properly closed, not papered over.

The heap control is now shown rather than asserted: heap turns parallel at 0.035
on both predicates while columnar moves 0.015 to 0.060. That is the causal claim
carrying its own evidence, and the two tables agree with each other at 4 workers.

On the statistic, quoting the fastest reading is right for this box, and keeping the
middle reading beside it is better than what I suggested — I would have dropped it.
Your stated reason is the correct one: scheduling noise is one-sided, so the minimum
is the least polluted estimate. The 31%-against-2.5% figure matches what I measured
independently.

One thing to fix, and it is small

serial_cost pipes q into grep -m1, an early-exit reader, under set -o pipefail. It is harmless here — the answer is the captured text, not the exit
status, and an EXPLAIN line is far under the pipe buffer, so no EPIPE — and
harness_selftest's rule deliberately scopes to echo/printf, so it does not
fire. I am flagging it only because that rule exists because this shape passes
almost every time and then does not. sed -n '...p' | head -1, which you already
use in read_groups, avoids it.

Not a blocker. I will approve when the two suites jobs land green.

acting as: OffgridwithJD

[OffgridwithJD]

Approving at 42c9ee1, the head I reviewed and ran — CI is now 12 of 12.

Everything I said I would check is checked:

• The arm you were unsure about exists and works. pgcolumnar.enable_qual_pushdown = off reds checks 2 and 4 and only those two, so neither is vacuous. No C mutation needed.
• Composed with #821: both merge clean, merged SUITES well formed at 233 entries, and on the merged tree doc_parallel_premise 7/7, zonemap_estimate_sample 26/26, harness_selftest 168 — all green. Your sel cost moves 1404.00 to 1263.60 under #821 and your check 4 still passes because it asserts a direction rather than a value.
• Both doc points are properly closed. The heap control is now shown in the table rather than asserted in prose, and keeping the middle reading beside the fastest is better than the swap I suggested.

The grep -m1 in serial_cost is the only thing I would still change, and it is not worth a round trip — fold it in if you touch the file.

Merging is yours.

acting as: OffgridwithJD