Conversation
The snapshot cursor queries sorted every matching snapshot of a team with all of its columns, metadata and config included, before LIMIT could apply: a LIMIT cannot bound a sort through the lateral joins above it. For large teams the planner's parallel bitmap plan spilled that sort to disk on every call. Each query now picks the page in a subquery that returns only the id, the sort keys and the build columns, then reads the full row and the aliases for the page alone. The ready-build lookup stays in the subquery because it drops rows and must run before LIMIT. Generated row types and parameters are unchanged.
PR SummaryMedium Risk Overview Reviewed by Cursor Bugbot for commit 571b78f. Bugbot is set up for automated code reviews on this repo. Configure here. |
|
Opened in the wrong repository; moving to belt. |
Description
In production,
GetSnapshotsWithCursor(the paused half ofGET /sandboxes) accounts for essentially all of the database's temp-file writes. It launches about 1.8 parallel workers per call, and its slowest calls take over a minute. The temp-file log shows the query's leader and worker each spilling 80–115 MB on a parallel bitmap scan.In that plan Postgres sorts every matching snapshot of the team, with all columns including
metadataandconfig, and only then runs the build and alias lookups and appliesLIMIT. ALIMITcan't bound a sort through the lateral joins above it, so the sort holds the whole team.This PR splits each of the four cursor queries in two:
id, the sort keys and the build columns. The ready-build lookup stays in this subquery because it drops rows and has to run beforeLIMIT. Moving it afterLIMITwould let snapshots without a ready build take page slots.The generated row structs, parameters and scan order are unchanged, so callers don't change.
Plans
These come from a Postgres 18 test container with one team holding 100k snapshots (~1 KB of
configeach) andwork_memat 4 MB.Default settings: both queries walk
idx_snapshots_team_time_idin order, and nothing is sorted. This change doesn't affect that plan (1–1.6 ms atLIMIT 101).Production's plan shape, forced with
enable_indexscan = off(parallel scan → Sort → Gather Merge):LIMIT 101LIMIT 5101* There's also a 6.7 MB re-sort of the page at the top. That's an artifact of disabling index scans: the primary-key lookup became a hash join. With index scans available it's a nested loop that keeps the page order, as in the
LIMIT 101plan.I couldn't reproduce production's planner statistics locally, so the "after" numbers are for the forced plan shape. After deploy,
temp_blks_writteninpg_stat_statementsfor the new query, and thetemporary filelog lines, will show whether it holds in production.Tests
TestSnapshotCursorQueriesShareOneProjectionnow compares the shared text around each query'sWHEREandORDER BY. It also checks that each query's outerORDER BYmatches its pageORDER BY.TestGetSnapshotsWithCursor_OrdersNewestFirstAndPaginatescovers newest-first order, keyset pages with no gaps or overlaps, and a snapshot without a ready build not taking a page slot.Not in this PR: the handler still adds one to the
LIMITfor each running sandbox it excludes (sandboxes_list.go), which keeps the page large for teams with many running sandboxes.