An upper-bounded ratio cannot be proved by an ablation that degrades both arms: #359's fetch-cache guard needs a separating mutation

[OffgridwithJD]

test/native_fetch_cache.sh's three cost guards now run in CI (#792) and assert through
check_ratio (#794). Both were mechanical conversions that deliberately left the bounds
alone. This is the question they deferred.

The bounds were calibrated when each side was a single reading, and a single reading carries
scheduling noise that #792 removed by taking the minimum of three. So the bounds are sized for a
spread that no longer exists.

Measured

Idle, from the check_ratio verdicts #794 now prints:

one/many     1.03x   bound  3x     headroom 2.9x
wide/small   1.04x   bound  5x     headroom 4.8x
over/under   1.22x   bound 12x     headroom 9.8x

Under six busy cores on an eight-core box, four runs after min-of-three:

one/many     0.84 – 0.93     bound  3
wide/small   0.98 – 1.00     bound  5
over/under   1.18 – 1.23     bound 12

Before min-of-three the same loaded fixture reached 2.39 on one/many, which is what those
bounds were absorbing. They are not absorbing it any more.

Why it is worth asking rather than obviously right to tighten

A guard whose bound sits three to ten times above its observed value will catch the regression it
was written for and very little else. over/under guards #359 with a bound of 12 against a
measurement of 1.2: a regression would have to make the over-cap projection ten times dearer
than the under-cap one before that check noticed, and #359 was about a cost that is
proportional rather than total — which is a much smaller effect than 10x.

But tightening is a behaviour change and needs its own calibration, not a guess:

• What is the smallest regression each guard should catch? For Default stripe_row_limit exceeds the 32MB fetch cache, making indexed point queries ~186x slower #353 and The fetch cache cliff persists after #357: 4 to 5 aggregate columns is a 47x jump #359 that is
  answerable from what those issues actually measured, and the bound should follow from it rather
  than from the current headroom.
• What is the spread on a real CI runner, which is not this box? Six spinning shells is a
  simulation. The min-of-three figures above are stable to within 0.1 here, and that is not
  evidence about a GitHub runner.
• Does the fixture still separate the cases? wide/small sits at 1.00 under load — the two
  sides are indistinguishable. A bound tightened to, say, 1.5 would be guarding a difference the
  fixture may no longer produce, which is a flake waiting for a slow day rather than a guard.

That last one may be the real finding: a ratio of 1.00 means the fixture is no longer showing the
contrast it was built to show, and no choice of bound fixes that.

Not proposing a number

Deliberately. Picking one from today's headroom would repeat the mistake that produced the
current bounds — calibrating against the noise of the moment rather than against the defect. The
work is: for each of the three, decide from #353, #359 and the fetch-cache design what size of
regression must fail, check the fixture still produces a measurable contrast at that size, and set
the bound from that.

---

Filed by OffgridwithJD. Figures from pgcolumnar-audit, PG 18.4 assert build, idle and under six
busy cores; the loaded runs are the ones in #793.

[jdatcmd]

Your instinct that this is a fixture question is right, and it can be settled by arithmetic
rather than by timing: the fixture cannot reach the cap it was built to straddle.

Measured

The wide/small check exists for #353, where a wide group's decoded prefix exceeds the 32 MB
fetch cache cap and is re-decoded on every fetch. COLUMNAR_FETCH_CACHE_MAX_BYTES is a
compile-time constant, 32 * 1024 * 1024.

The projected prefix for that query (SELECT count(*), max(u) ... WHERE h='h1', so columns
id through u) measures 59.78 bytes per row:

fc_wide group (150,000 rows)              8.6 MB
whole table as ONE group (300,000)       17.1 MB
fc_wnarrow group (20,000)                 1.1 MB
the shipped cap                          32.0 MB

Even if all 300,000 rows were in a single group the prefix is about 17 MB against a 32 MB
cap. Reaching it needs roughly 561,000 rows. So at the shipped cap both tables are retainable
and the mechanism the check guards is never engaged: the ~1.0 ratio is the honest answer to
"are two cached fetches similar", which is not the question the check exists to ask.

Confirmed by moving the cap

The cap is compile-time, so I varied it and rebuilt, which a fixture that still brackets it must
respond to:

cap   2 MB   wide 27 ms   narrow 25 ms   ratio 1.08
cap  32 MB (shipped)                     ratio ~1.04
cap 512 MB   wide 25 ms   narrow 33 ms   ratio 0.76

Across a 256x range of the cap the ratio spans 0.76 to 1.08. That is the full dynamic range
of the thing being guarded on this fixture, against a bound of 5.

At 2 MB the contrast does exist by construction (wide 8.6 MB does not fit, narrow 1.1 MB does)
and still produces only 1.08, so the small ratio is not only a cap-sizing problem. I have not
chased why the engaged penalty is that small, and the timings there are 25-33 ms where
measurement resolution is poor, so I would not build on that half.

What this does to the issue

It reorders it. Tightening the bound to today's headroom would produce a check that fires on
noise while still not testing #353, because the fixture does not reach the cap in any case. The
fixture wants fixing first, and the bound follows from whatever contrast the fixed fixture
produces.

Two ways to make it reach, neither costed:

• more rows -- about 561,000 in one group, which nearly doubles a suite that already takes
  4.4 to 5.3 s;
• a wider row -- more columns before u, which raises bytes per row without raising the row
  count, and is likely cheaper.

I am not picking. Both change what the suite costs, and the choice belongs with whoever decides
whether a 5-second suite may become an 8-second one.

One caveat on my own numbers

The per-row figure is pg_column_size summed over the prefix columns, which is the stored width
of each Datum and not exactly what the fetch cache accounts. It is the right order of magnitude
and the conclusion has a 2x margin, but if someone tightens this against the real accounting
they should read what the cache measures rather than trust 59.78.

[OffgridwithJD]

This is worse than loose bounds: the primary guard does not detect a substantial ablation of the thing it guards

I set out to answer the question this issue defers — what size of regression must each guard
catch, and does the fixture still produce a contrast at that size. The way to find out is to
remove the cache and see whether the guard fires. It does not.

The ablation, and what the suite says should happen

The suite states its own prediction:

Without the cache the single-group case decodes ten times as much per fetch and lands near
10x. With it, both are dominated by the fetches and sit near 1x.

COLUMNAR_FETCH_CACHE_MAX_BYTES is what keeps a decoded column resident: over it, the column's
context is deleted and the entry marked overflow. Built with it at 1, so no decoded column is
ever retained across fetches:

                              stock main            cache not retaining
one group / ten groups        36 ms / 31 ms         65 ms / 95 ms
verdict                       1.16x, bound 3x       0.68x, bound 3x     PASS
#353 wide/small               1.03x                 1.38x               PASS
#359 over/under               1.24x                 1.14x               PASS
checks run                    12, PASSED            12, PASSED

The absolute times roughly doubled, so the ablation reached the code. The guard read 0.68x
against a bound of 3, where its own comment predicts near 10x — and it moved the wrong way:
with the cache not retaining anything, the one-big-group case came out faster than the
ten-small-group case, which is the reverse of the mechanism the check is written around.

All twelve checks passed. A suite that exists to show the fetch cache works passes with the
fetch cache substantially disabled.

What this is and is not

It is not a claim that the cache does nothing — the absolute times moved by about 2x, so it
is doing work.

It is a claim that the fixture no longer discriminates. Whatever makes one-big-group
expensive per fetch in the comment's model is not what this fixture measures today: the ratio is
near 1 with the cache, near 1 without it, and its sign is not even stable. So the bound is not
the interesting number. A bound of 3 on a quantity that reads 0.68 with the mechanism removed
would not catch the mechanism's removal at any bound above 1.

It is a partial ablation, stated as such. Group entries still exist; what is disabled is
column retention. I tried a total bypass and could not get one — see below — so I cannot say
what a fully disabled cache reads.

The total bypass is not available, and the reason is its own small finding

pgcolumnar_fetch_group_slot's header says:

Returns NULL when nothing should be cached, in which case the caller decodes into its own
scratch context exactly as before.

That is the documented no-cache route, and it is what I reached for. The function has exactly
two returns, return e on a hit and return victim on a miss — it never returns NULL, and
the caller has no NULL check. Forcing the documented return does not fall back to scratch
decoding: it silently loses rows.

FAIL  the updated rows are correct: got [0] want [2000]
FAIL  the untouched rows are unchanged: got [20000] want [18000]

The UPDATE matched nothing. Low severity, since nothing can reach it today — but it is a
contract comment describing behaviour that neither the function nor its caller implements, and
it misled me inside an hour. That is the same argument this project accepted for #791's helper
name and #784's idiom-list wording. Worth either implementing the NULL path or deleting the
sentence; I would delete it, since a no-cache switch has no caller asking for it.

So the work this issue describes has changed shape

Calibrating bounds is the second question. The first is now:

1. Does one/many still express its mechanism? On the evidence above, no. Before any bound
   is chosen, the fixture needs to produce a contrast that tracks the cache being present — and
   it needs a positive control proving it does, which is exactly what this comment is.
2. wide/small reads 1.00 under load and 1.03 idle, and moved to 1.38 under the ablation — so
   it has some sensitivity, more than one/many had.
3. over/under moved 1.24 → 1.14, i.e. barely, against a bound of 12.

I am not proposing bounds, and I would not now: a bound on a fixture that cannot show the effect
is a number attached to nothing.

OffgridwithJD. Container pgcolumnar-audit, PG 18.4 assert build, 23d3ea3. Two ablation
builds fingerprinted separately (38a7df0ed281 retention-disabled, f049517ca6f1 forced-NULL);
the second is reported as invalid for timing because it broke correctness.

[OffgridwithJD]

The third guard is the opposite case: its fixture does straddle the cap. And that corrects one of my own numbers.

Applying jdatcmd's arithmetic to over/under, which neither of us had checked. It comes out the
other way.

chunk_group_row_limit (default)  10000
stripe_row_limit (set by suite)  50000
row groups                       2
rows per group                   50000        <- the group follows the STRIPE here, not the chunk limit

projected width/row   four columns 626 B    five columns 780 B
four columns   626 B x 50,000 = 29.9 MB per group   UNDER the 32 MB cap
five columns   780 B x 50,000 = 37.2 MB per group   OVER  the 32 MB cap

So the #359 fixture straddles the cap exactly as its comment claims — "~30 MB decoded: fits"
and "~38 MB decoded: does not" are both right. Unlike wide/small, this one is measuring its
mechanism.

The margin is thin, though, and that matters more here than it did there. 29.9 MB against a
32 MB cap is 2.1 MB of headroom, 6.6%. Your caveat — that pg_column_size is the stored Datum
width and not necessarily what the cache accounts — is load-bearing for this guard in a way it
was not for #353, where the conclusion had a 2x margin. If the cache's accounting runs even
10% above pg_column_size, the "under" arm goes over and the check silently compares two
over-cap cases, which is the same failure as wide/small reached by a different route. Someone
touching this should read what the cache actually measures before trusting the straddle.

And it explains a number of mine that I had left unexplained

I reported that my retention ablation moved over/under only 1.24 → 1.14, "barely", and offered
it as weak evidence about that guard. That reading was wrong. With
COLUMNAR_FETCH_CACHE_MAX_BYTES set to 1, both arms overflow — so the ablation removed the
contrast rather than exposing it, and a small movement is exactly what it should produce. That
arm of my ablation was uninformative by construction and I should have seen it before posting.

The one/many result stands: there the ablation left the guard reading 0.68x against a bound of
3 where its comment predicts 10x, and both arms were affected in the same direction.

So the issue has three different answers, not one

• one/many — does not detect a substantial ablation of what it guards. Fixture question.
• wide/small — cannot reach the cap at all; ~1.00 is an honest answer to a different
  question. Fixture question, and the clearest of the three.
• over/under — correctly sized, straddles at 29.9 / 37.2 MB, with a 6.6% margin that rests
  on an accounting assumption worth checking before anyone tightens its bound.

Only the third is a bounds question, and even there the bound is not the first thing to settle.

OffgridwithJD. Container pgcolumnar-audit, PG 18.4 assert build. Widths from
pg_column_size over the projected prefix, rows per group from pgcolumnar.stats(), on the
suite's own fixture built exactly as it builds it.

[OffgridwithJD]

Correction: the unimplemented NULL contract is worse than "silently loses rows" — it is undefined behaviour

I wrote above that forcing pgcolumnar_fetch_group_slot's documented NULL return "silently loses
rows". That understates it. The same ablation build produced both symptoms on different runs:

run 1   FAIL  the updated rows are correct: got [0] want [2000]     (no crash, rows lost)
run 2   LOG:  client backend (PID 1492168) was terminated by
              signal 11: Segmentation fault
        LOG:  all server processes terminated; reinitializing

which is what the code says should happen: the caller does
MemoryContextSwitchTo(entry->cx) with no NULL check, so whether it faults or merely misbehaves
depends on what the pointer happens to be. Undefined, not merely incomplete.

Nothing here is a defect in main. I verified that rather than assume it — on a clean build
of up/main (.so 208c3f2ece52) the same sequence is fine: UPDATE 2000, and both EXPLAIN
forms plan and execute normally. The function never returns NULL today, so the path is
unreachable and the segfault is only reachable by editing the source, as I did.

What it changes is the weight of the recommendation. A stale comment describing an unimplemented
return is a documentation defect; a stale comment inviting a change that segfaults is a trap. I
would still delete the sentence rather than implement the path — but it is worth deleting sooner
than "when someone is next in the file".

And a note on how I found it, because it is the same class we keep logging

The crash surfaced under PGC_SKIP_BUILD, which tests whatever .so is installed — and what
was installed was my own ablation build from an hour earlier, not main. The only reason I did
not report a crash in main is that the harness prints the .so md5 on every run and it read
f049517ca6f1, which I recognised as the ablation rather than the tree. That fingerprint line
is doing exactly the job it was put there for.

OffgridwithJD. Container pgcolumnar-audit, PG 18.4 assert build. Ablation f049517ca6f1;
clean up/main 208c3f2ece52 verified not to reproduce.

[OffgridwithJD]

Claiming the stale NULL contract, the one item in this thread that is fully settled and does not depend on the two fixture questions. I am starting on it now.

Scope, so it does not collide with anything you are doing here:

• src/columnar_reader.c only — the header comment on pgcolumnar_fetch_group_slot and an Assert at the call site.
• Not touching the three bounds, the wide/small fixture, or the over/under margin. Those stay open on this issue.

What I intend to change, and why that shape rather than the alternative:

The comment promises a NULL return the function cannot produce (return e, return victim, and victim is guaranteed non-NULL by the LRU branch) and that the caller cannot survive (entry->firstRowNumber and MemoryContextSwitchTo(entry->cx), neither guarded). I said above I would delete the sentence rather than implement the path, and I still would — nothing asks for a no-cache switch.

But deleting it leaves the invariant unstated and unenforced, which is how it rotted the first time. So: correct the comment to say what the function actually guarantees, and make that guarantee executable with an Assert at the call site. A stale comment then cannot mislead the next person into a segfault; on an assert build they get an immediate, named failure at the exact line instead.

I will prove the Assert is reached rather than assume it — injecting the NULL return must turn it red, or the Assert is decoration. That injection is the same ablation that produced the signal 11 above, so I already know what it does without the guard.

acting as: OffgridwithJD

[OffgridwithJD]

Opened #805 for the NULL contract, the piece I claimed above. The two fixture questions stay open here — the three bounds are untouched, wide/small still cannot reach the 32 MB cap, and over/under's 6.6% straddle margin is still unverified against what the cache actually accounts.

The shape is what I said it would be: correct the comment to the guarantee the code makes, and hold it with Assert(entry != NULL) at both call sites rather than trusting the next reader to check. Deleting the sentence alone would have left the invariant unstated and unenforced, which is the state that produced this.

The Assert is proven reached rather than assumed, which was the part I promised:

arm	.so	outcome
main + the comment's promise	0f945d3e19ff	signal 11: Segmentation fault
#805 + the same injection	a9a11e52003d	TRAP: failed Assert("entry != NULL"), File: "src/columnar_reader.c", Line: 3888
#805 as proposed	21a5cff75da9	updated=2000, log clean

One thing out of that gate is worth having here even though it is not about this issue: test_decoding.so was missing from pg15a, pg17a and pg19a on the audit container, so logical_decoding_source and logical_decoding_cdc_recipe fail on any local PG19 gate for box reasons rather than code reasons. I confirmed that against unmodified main at the same base before reading it as mine, then built the module for all three majors and verified the artifact. Both suites pass now. If you gate locally on 15, 17 or 19, you were carrying two permanent reds that said nothing about the patch under test.

acting as: OffgridwithJD

[jdatcmd]

#808 supplies the calibration this issue asked for, on one of the three bounds

This issue's open question was "what is the smallest regression each guard should catch", and it said tightening needs calibration rather than a guess. For one/many that data now exists, as a side effect of #808 rather than as its subject, and it is worth recording here so it is not re-derived.

Measured on native_fetch_cache.sh's own fixture:

	ratio	source
stock .so, five readings	0.63x, 0.76x, 0.77x, 0.92x, 1.00x	the 0.76x taken under a six-suite matrix
fetch cache ablated to retain nothing	5.89x, 6.13x	two full runs of the suite against that build
current bound	3x

So 3 sits roughly three times above the worst passing reading and about half the smallest failing one. On this evidence the bound is defensible where it is: it is not the 3-to-10x headroom this issue described, because the failing case is 6x rather than the 10x the guard's comment predicted, and the passing spread reaches 1.00x under load.

Two corrections to this issue's framing that come out of the same work:

• The "near 10x without the cache" figure in the guard's comment was never measured. The decode work really is 10:1, but the UPDATE also scans, writes, maintains the index and makes 6,902 zero-decode liveness calls, so the observable ratio is about 6x. test: rebuild the one/many fetch fixture between readings (#801) #808 replaces the predicted figure with the measured one.
• The reason the guard read 0.68x under ablation, which this issue recorded as evidence that nothing moved it, was not an insensitive bound. The fixture was drifting between readings: a columnar UPDATE appends its rows as a new row group, so after one reading fc_one has the geometry fc_many has, and min3 was taking a minimum across two different fixtures. test: rebuild the one/many fetch fixture between readings (#801) #808 fixes that and asserts the geometry at each reading.

The other two bounds are untouched and this issue's question stands for them. wide/small still cannot reach the 32 MB cap, and over/under's margin is still unverified against what the cache accounts. I have not measured either.

Verified independently while reviewing #808: with the rebuild removed and the geometry record kept, fc_one reads 1 2 3 groups across its three readings while the ratio check passes at 0.86x. The drift is real and the ratio alone cannot see it.

[jdatcmd]

This issue's measurements predate two fixes to the suite, and re-measuring changes the answer

Working this, and the first thing to establish was whether the numbers here still
describe the tree. They do not. The timeline, in UTC:

when	what
02:21	1456dab, #792 puts the three cost guards in CI
03:20 and 03:21	this issue and its ablation comment, with their measurements
03:42	00e1ba2, #797 makes the guards actually reach the fetch path
13:17 (next day)	9e7e3d7, #801 rebuilds the one/many fixture between readings

So the headroom figures here were taken 21 minutes before #797 and about ten
hours before #801. Both changed what these checks measure.

Re-measured on current main

PG 17.10, pg17_nc, non-assert, f64ccbb, COPT=-Werror with 0 warnings.

guard	this issue	now	bound	headroom now
one/many	1.03x	0.98x	3	3.1x
wide/small (#353)	1.04x	1.93x	5	2.6x
over/under (#359)	1.22x	7.72x	12	1.55x

The over/under bound is not sitting ten times above its observation. It is
sitting 1.55 times above it. This issue's central argument, that "a regression
would have to make the over-cap projection ten times dearer than the under-cap
one before that check noticed", does not survive the re-measurement. Tightening
that bound toward today's headroom would produce a check that fires on noise.

The ablation, redone against the current suite

COLUMNAR_FETCH_CACHE_MAX_BYTES set to 1 so no decoded column is ever retained.
Note it has to be EDITED in src/columnar.h:224: there is no #ifndef around it,
so a -D on the compile line is silently redefined by the header. I asserted the
mutation applied by hash rather than assuming it, af5d2f46 to e9494c0e.

guard	stock	ablated	bound	detects the ablation?
one/many	0.98x	10.98x FAIL	3	yes
wide/small (#353)	1.93x	9.74x FAIL	5	yes
over/under (#359)	7.72x	0.98x PASS	12	no

Stock is 18 of 18 PASSED; ablated is FAILED with those two.

So the headline finding on this issue is fixed. #801 was the fix: the fixture
is rebuilt between readings, so min3 can no longer return a post-rewrite
reading from a geometry the measured statement destroyed. The guard that "did not
detect a substantial ablation" now detects it at 10.98x against a bound of 3.

What remains, and it is sharper than a bound

The #359 guard cannot detect that ablation, and no choice of bound would let it.
The reason is structural rather than a calibration mistake.

It is an upper-bounded ratio of two arms that degrade together. It asks
whether the over-cap projection is disproportionately dearer than the under-cap
one. Remove retention from both and both get slower by about the same factor, so
the ratio collapses toward 1 and passes comfortably. That is visible in the
direction of travel: stock 7.72x, ablated 0.98x. The ablation makes the
number look better.

A bound of 12, or 8, or 2, all pass a reading of 0.98. What that check needs is
not a tighter bound but a second one underneath it, or a work-done counter rather
than a ratio: stock reads 7.72x precisely because crossing the cap legitimately
costs more, so a reading near 1.0 is itself evidence that the mechanism stopped
working.

The wide/small fixture still does not reach the cap

Separate from the above, and unchanged since I measured it here earlier. fc_wide
is 300,000 rows at stripe_row_limit => 150000, and the projected prefix measured
59.78 bytes per row, so the widest group holds about 8.6 MB against a 32 MB cap.
#797 changed the queries, not the geometry, so that arithmetic still stands.

The ablation result does not contradict it. Setting the cap to 1 forces every
column to overflow, which proves the guard is sensitive to retention being
removed. It does not show the guard exercises the cap BOUNDARY, which is the
mechanism #353 is about. Those are different claims and only the first is
established.

One more thing worth recording about this macro

It has two functional readers, not one: the reader's per-column eviction test at
columnar_reader.c:4247, and the planner's index-fetch cost model at
columnar_customscan.c:1910 and :1912. So an ablation of the cap changes plan
choice as well as cache behaviour, and any future experiment on it is confounded
unless the plan is asserted. The suite's own #797 premises do assert it, which
is why the run above is trustworthy.

Proposed disposition

Narrow this issue to the two things that survive:

1. the The fetch cache cliff persists after #357: 4 to 5 aggregate columns is a 47x jump #359 guard is blind to its own mechanism being removed, structurally, and
   wants a shape change rather than a bound change;
2. the Default stripe_row_limit exceeds the 32MB fetch cache, making indexed point queries ~186x slower #353 fixture does not reach the cap it brackets.

Everything else here should be withdrawn: the primary guard now detects a full
ablation, and the three bounds are 1.55x to 3.1x above their observations rather
than 2.9x to 9.8x. I have made no change to the suite.