Object-storage tiering is rejected, not deferred (#403 item 5b)

[jdatcmd]

The last of the #403 remaining items, and per your decision it terminates at a document rather than an implementation. Independent of #814 and #815 — docs only, branched from main.

The answer is no, and not because it is hard

The seam is small: 4 call sites read through PgColumnarReadLogicalData, 1 writes through PgColumnarWriteLogicalData, and the object-storage API from #393/#394 already offers exactly the ranged reads a tier needs. Feasibility is not the problem.

The obstacle is what the bytes currently are. Every columnar page goes through the buffer manager and is WAL-logged as a full-page image — log_newpage_buffer(buffer, true), on written pages and gap pages alike. Five properties follow, and a tier keeps none: crash recovery, physical replication, self-contained backup, PITR, and rollback of an aborted rewrite.

One of the five settles it

docs/limitations.md already records that logical decoding cannot carry columnar tables, because their data reaches WAL as full-page images with no tuple structure, and directs users to physical replication.

Physical replication works precisely because the bytes are in WAL. Tiering removes them, so a standby replays the WAL and finds nothing — and it cannot fetch them, because nothing in the stream says they exist. Tiering removes the only supported way to replicate a columnar table, for any table using it.

The other four are engineering problems with known, expensive answers. This one is a change to what the product is.

Why rejected rather than deferred

Keeping replication would mean putting a record in the WAL that a standby acts on by fetching remotely: a new WAL semantic, in a new record type, interpreted by a replay path this extension does not own. The project's standing constraint rejects that rather than deferring it.

So this is not a large feature waiting for time, and recording it as deferred would invite someone to re-derive all of this in a year.

What to build instead

The user-facing goal is reachable today by composing what exists, with none of the five invariants touched: export the cold range (export_parquet / parallel_export_parquet), drop the local rows (pgcolumnar.expire from #815, or DELETE plus compact), read it back through the external Parquet reader or iceberg_scan.

The cost — the cold data is no longer part of the same table, so a spanning query has to union them — is stated plainly rather than dressed up.

Every fact was checked, including one I had wrong

An earlier draft said the seam had 5 read and 2 write call sites. That counted the function definitions along with the calls; it is 4 and 1. Each row of the document's evidence table names the file and construct it comes from.

I also verified the three functions the recommendation tells people to use actually exist in the shipped script before recommending them — expire is marked as PR #815, not yet merged, rather than implied to be available.

Also in this PR

design/ISSUE_403_REMAINING_ITEMS.md gains an outcome table for every item, including the two that changed shape once measured: item 3 was not built, and the <> substitute proposed in its place was withdrawn after measurement too.

Refs #403.

[OffgridwithJD]

Approving 3044091. Independent of #814 and #815 -- one commit off main, not
stacked -- so it can go in whatever happens to those.

Docs only, so the review is whether the facts are facts. I checked each one
against the tree rather than reading it:

The seam counts. Confirmed independently: 4 real calls to
PgColumnarReadLogicalData (columnar_reader.c 1996, 2296, 4085, 4138) and 1
to PgColumnarWriteLogicalData (columnar_write_state.c:2755). Everything else
matching those names is the definition, the header declaration, or a comment.
Your correction from 5 and 2 was the right correction, and the document's "all in
columnar_reader.c" holds too.

The WAL claim. Eight log_newpage_buffer(..., true) in
columnar_storage.c, including line 481 on the gap page, so "on written pages
and gap pages alike" is exact. The two false calls are the visibility map, a
different fork, which does not weaken the argument.

The replication argument's premise. docs/limitations.md:624 does say it:
"does not carry it. Use physical replication for columnar tables." So the
document is not inventing the constraint it leans on, it is citing one the
project already published -- which is what makes "tiering removes the only
supported way to replicate a columnar table" land.

The recommendation is buildable today. export_parquet,
parallel_export_parquet, iceberg_scan, read_parquet and compact are all
in pgcolumnar--1.0-alpha3.sql. expire is not, and the document says so --
"#403 item 5a, PR #815, not yet merged" -- rather than implying it is available.
A rejection document whose alternative pointed at a function that does not exist
would be worse than no document, and this one does not.

I went looking specifically for the stale-number failure: a count corrected in a
PR description while the document keeps the old one. There is no 5 read or
2 write left anywhere in the file.

On the substance

The argument is that four of the five lost properties are expensive engineering
and the fifth is a change to what the product is, and I think that is the right
shape for a rejection: it does not rest on effort, so it does not expire when
someone has more time. Recording it as deferred really would invite the whole
derivation again in a year.

Nothing to request. The one thing I would keep an eye on is that the "what to
build instead" section will go stale the moment expire merges -- the status
note is correct today and becomes wrong when #815 lands.

acting as: OffgridwithJD