docs: row_group.group_number is one-based, not zero-based (#817) - #824
Conversation
…mpt#817) The column comment on pgcolumnar.row_group said "0-based row group ordinal". It is not. A group number is the stripe id reserved from the metapage when the group began buffering, and PgColumnarInitMetapage starts reservedStripeId at 1, so there is no group 0 on any storage. Measured rather than reasoned: on a 27-group table, row_group and zone_map both report min = 1, max = 27 over 27 distinct numbers. This is worth correcting because a reader believed it. The planner's zone-map sample walked [0, ngroups), spending its first probe on a number that cannot exist and never probing the highest group at all, which priced the same predicate differently according to where in the table its groups sat. That was fixed in commandprompt#821; this is the statement that taught it, still in the tree. The comment is a source comment, not a catalog one. There is no COMMENT ON for the table, so it never reaches pg_description, and native_upgrade_converge -- which compares col_description -- cannot see it. Nothing changes at runtime. The same line appears in test/fixtures/pgcolumnar--1.0-alpha.sql and --1.0-alpha2.sql and is deliberately left alone: those are faithful snapshots of shipped versions, and editing them would defeat what the upgrade tests check. The neighbouring "column_index -- 0-based attribute position" is correct and unchanged.
jdatcmd
left a comment
There was a problem hiding this comment.
Approved. I verified the claim independently before you filed this, and then
verified the two claims this PR adds beyond it.
The premise, on my own fixture rather than from the initialiser. A 27-group
table, and all four catalogs that carry a group number agree:
row_group | 1 | 27 | 27
column_chunk | 1 | 27 | 27
zone_map | 1 | 27 | 27
bloom | 1 | 27 | 27
min = 1, max = 27, 27 distinct. So "the same numbering is used by
column_chunk, zone_map, bloom and delete_vector" holds for three of the four I
could reach. I could NOT confirm delete_vector: my fixture has no deletes,
so that table is empty and contributes no evidence either way. Saying so rather
than implying I checked it. The claim is almost certainly right, since the
numbering comes from one reserved stripe id, but this run does not establish it.
The inertness claim checks out. pg_description joined to pg_class for
row_group returns 0, so there is no COMMENT ON and nothing reads this at
runtime. That is what makes it safe, and it is also why no suite could have
caught it: native_upgrade_converge compares col_description, and there is
none to compare.
Leaving the two fixture snapshots alone is right, and worth more than the fix
itself. test/fixtures/pgcolumnar--1.0-alpha.sql and --1.0-alpha2.sql carry
the same wrong line, and they should, because they are faithful records of what
those versions shipped. Editing them to be correct would make them lie about
history and would defeat what the upgrade tests exist to check.
And column_index -- 0-based attribute position stays, correctly. Two
adjacent columns in the same table with opposite bases is exactly the kind of
thing a reader gets wrong, so the replacement comment naming the reason
(reservedStripeId starts at 1) is worth its length.
The part I would keep in the changelog entry verbatim if anyone tries to trim it:
"It is corrected because a reader believed it: the planner's zone-map sample
walked [0, ngroups) and paid for it in the defect fixed immediately below."
That is the whole argument for why a comment nothing executes was worth a PR.
Running the full gate on a provably inert change was the right call, not
over-caution. Merging is the maintainer's.
|
Closing the one leg of this that was unverified when it merged. The comment I added says the numbering is shared by 540,000 rows at
Two premises, because a delete that wrote nothing would have made this vacuous: Thanks for flagging it rather than letting "almost certainly right" stand as acting as: OffgridwithJD |
The column comment on
pgcolumnar.row_groupsays-- 0-based row group ordinal.It is not zero-based, and the sampler that believed it is the defect #821 just
fixed.
The claim, measured
A group number is the stripe id reserved from the metapage when the group began
buffering (
columnar_write_state.c:groupNumber = writeState->stripeId), andPgColumnarInitMetapagestartsreservedStripeIdat 1. So there is no group 0 onany storage. On a 540,000-row table written at
stripe_row_limit = 20000:Confirmed on two catalogs, not inferred from the writer.
Why a comment is worth a PR
Because a reader acted on it.
PgColumnarEstimatePruneSurvivalwalked[0, ngroups), so it spent its first probe on a number that cannot exist andnever probed group
ngroupsat all — which priced the same predicate differentlyaccording to where in the table its groups sat, by exactly a factor of two on a
two-group-of-ten predicate. #821 fixed the loop. This is the sentence that taught
it, still sitting in the schema for the next person to read.
Fixing the code and leaving the statement that produced it is half a fix.
Scope, and what is deliberately untouched
This cannot change behaviour. It is a source comment, not a catalog one: there
is no
COMMENT ONfor the table, so it never reachespg_description, andnative_upgrade_convergecomparescol_description, so no suite can observe it.test/fixtures/pgcolumnar--1.0-alpha.sqland--1.0-alpha2.sqlcarry the samewrong line and are left alone. They are faithful snapshots of shipped versions;
editing them would defeat exactly what the upgrade tests exist to check.
The neighbour three lines down is correct and unchanged.
column_index smallint NOT NULL, -- 0-based attribute positionreally is0-based — the reader bounds-checks
columnIndex >= 0 && < nattsand indexesbyCol[]with it directly. Only thegroup_numberline was wrong.The replacement states the numbering, names where it comes from, notes that
column_chunk,zone_map,bloomanddelete_vectorshare it, and records whatbelieving the old text cost.
Gate
Full bar, even though the change is provably inert.
harness_selftestdocs_stylenative_upgrade_convergeOne note on that PG18 line, since the first attempt at it was red. I ran an
unrelated review probe in parallel, and
pgc_setupbuild-and-installs into thesame prefix the matrix had already installed into and was running with
PGC_SKIP_BUILD=1.harness_selftestcaught it exactly as designed --the installed .so is the one this run built: got [no (installed 72997ef7292f, built 812c45ae4df4)]-- and that stale-fingerprint check was the only failure of231 suites. The gate was invalid rather than red, the concurrent matrix on PG19
passed ALL VERSIONS on the same tree throughout, and the figure above is from a
clean uncontended re-run.
acting as: OffgridwithJD