Build rows lazily: components, minimum_spanning_forest, vector_topk, most_similar

DESIGN.md

@@ -1587,12 +1587,12 @@ A pooled backend's own memory is 4–12 MB with the shared cache against the ver
 - *Cold load:* 1.03 / 0.77 s privately, 1.06 / 0.79 s into the shared cache.
 - *Lookups:* `neighbours` of one IRI 0.1 / 0.1 ms (median of 20 calls), `most_similar` 1.1 / 1.0 ms, `similarity` of 100,000 pairs 1.31 / 0.41 s (past 65,536 lookups, so through a per-call index, which costs 40 MB at 5M nodes).
 - *Traversal:* `bfs` to depth 6 (1.07M rows) 0.95 / 0.49 s, `reachable_set` (5.0M rows) 3.72 / 2.22 s. Every row's `text` datum is built and checked against the encoding, so row emission, already most of these calls' time (above), about doubles. The kernels that return no ids are unchanged: `components` 2.77 / 2.74 s, `eigen` for 200 steps 75.9 / 77.3 s, and the library within noise (`clustering` 27.0 / 28.8 s, betweenness 40.5 / 40.1 s, `minimum_spanning_forest` 22.5 / 22.0 s).
-- *Peaks:* the stages that read the version peak about 0.12 GB higher privately, the difference in the versions' sizes (`pagerank` 1.28 / 1.16 GB, `eigen` 2.28 / 2.17 GB). The calls that return a row per node peak higher still: `components` 1.28 / 0.80 GB and `minimum_spanning_forest` 1.77 / 0.98 GB, for their rows of text ids (explained at full scale, below).
+- *Peaks:* the stages that read the version peak about 0.12 GB higher privately, the difference in the versions' sizes (`pagerank` 1.28 / 1.16 GB, `eigen` 2.28 / 2.17 GB). The calls that return a row per node peak higher still: `components` 1.28 / 0.80 GB and `minimum_spanning_forest` 1.77 / 0.98 GB, for their rows of text ids. *Fixed since:* both built every row before returning the first, a `Vec` of rows whose text ids were each `palloc`ed (`cstring_to_text_with_len`) in the set-returning function's multi-call memory context, which lives until the call ends: an IRI and its 4-byte header round to a 64-byte AllocSet chunk plus an 8-byte header, about 72 bytes an id (68–78 measured here, 69–72 at full scale, where it was 0.72 and 1.38 GB). They, `vector_topk` and `most_similar` now return an iterator over the kernel's result, which stays in the extension's own buffer (charged as before), and the loaded version, and make each row's external ids in `next()`. pgrx calls `next()` after the first row in the caller's memory context, the per-tuple context Postgres resets before each call (`ExecMakeTableFunctionResult` for a call in `FROM`, `ExecProjectSet` in the select list), so only the current row's ids are held; the iterator's `Arc` keeps the version alive, private or mapped from the shared cache, until the call ends, by its last row, a `LIMIT`, an error or a cancel (`shared_cache_lazy_rows.sql`; a cursor read across statements keeps it after its statement's pin has ended, even once it is evicted). Each has a `pg_test` that opens a cursor on the call and checks that what it holds after its first row does not grow with the rows to come (3.6–7.3 MB more for 50,000 rows before, none since). Measured again at half scale (`bench/results/soak/a-scale2*-m10-lazy.json`, 8d759e5, the same flags without the second version), text / `bigint`, peak RssAnon and what the call added to the backend: `components` 0.84 / 0.73 GB, +112 / +63 MB (was 1.28 / 0.80 GB, +555 / +166 MB); `minimum_spanning_forest` 1.10 / 0.98 GB, +326 / +326 MB (was 1.77 / 0.98 GB, +1,002 / +326 MB), what remains being the kernel's own result and scratch: the forest's text peak now differs from `bigint`'s by the versions' sizes alone, as the other stages' do, and `components`' by 49 MB more (about 10 bytes a node). Times are unchanged within noise (`components` 2.87 / 2.00 s, `minimum_spanning_forest` 17.0 / 16.4 s, a load average of 1–3). The SRFs that already made each row as it is returned are `bfs`, `reachable_set`, `vector_values`, `matrix_rows`, `id_map`, `similarity`, `similar_pairs` and `gather`. Two still build their rows up front, *left for M11* (§13): `walk`, whose rows are made as the walks run, into a reserved buffer of datums bounded by `max_results` and `warren.max_walk_paths`, so streaming it means keeping each row as dense ids (and a path's as a slice) and building its arrays in `next()`; and `neighbours`, whose rows are one node's neighbours, bounded by its degree, made inside a closure that borrows the loaded version and its adjacency, so streaming it needs an adjacency iterator that owns what it reads. Neither is the small change the four above were, and both are bounded by their arguments or by one node.
 - *Shared cache:* the shared version is 719.5 / 600.6 MB, and the private peaks fall by the version, as for `bigint` ids: `pagerank` 0.56 / 0.56 GB, `bfs` 0.12 / 0.12 GB, `components` 0.56 / 0.20 GB.
 
 Every §12 target that ran was met at this scale except `reachable_set`, which misses for both id types (12–13 / 22–23M edges/s).
 
-*At full scale* (10M nodes, 100M edges; `bench/results/soak/a-text-m10.json` and `a-m10.json`): case a with every stage, the recursive CTE baseline, the second version (`diff`, `distance`), the library and both community detections included, a release build of 7934941 on the soak's machine and settings, text then `bigint` back to back with no other agent building (load average 1.0–1.5, the soak's own). Neither run came within `--disk-floor` (3 GiB, of about 20 GB free). Figures are text / `bigint`; peaks are private RssAnon:
+*At full scale* (10M nodes, 100M edges; `bench/results/soak/a-text-m10.json` and `a-m10.json`): case a with every stage, the recursive CTE baseline, the second version (`diff`, `distance`), the library and both community detections included, a release build of 7934941 on the soak's machine and settings, text then `bigint` back to back with no other agent building (load average 1.0–1.5, the soak's own). Neither run came within `--disk-floor` (3 GiB, of about 20 GB free). Both ran *before the lazy-rows fix* (below), so the `components` and `minimum_spanning_forest` peaks are the eager ones. Figures are text / `bigint`; peaks are private RssAnon:
 
 | Stage | Text | `bigint` | Text / `bigint` |
 |---|---|---|---|
@@ -1619,7 +1619,7 @@ Every §12 target that ran was met at this scale except `reachable_set`, which m
 
 - *Size and build.* The text version is 28% larger, its id map the IRIs' bytes plus offsets and order; cached, 20% larger, since it builds no dense-id direction when it loads. The build costs 31–39% more (the half-scale 35%) and peaks 0.25 GB higher, the larger id map it holds. Every stage that reads the version peaks about 0.24 GB higher, the difference in the versions' cached sizes.
 - *Where text costs.* Calls that emit a row per reached node pay for each id's `text` datum (built and checked against the encoding): `bfs` and `components` take twice as long, `reachable_set` 42% longer. `similarity`'s 100,000 pairs pass 65,536 lookups, so it builds its per-call index (80 MB at 10M nodes, the 8 bytes a node of §5.7) and maps each pair through it, 1.7 s more. Kernels that map no ids are within 12% (`pagerank`, `eigen`) or unchanged (the library, the community detections, which return a modularity).
-- *Why `components` and `minimum_spanning_forest` peak higher.* Both build all their rows before they return the first: a `Vec` of `(AnyElement, …)` whose text datums are each `palloc`ed (`cstring_to_text_with_len`) in the set-returning function's multi-call memory context, which lives until the call ends. An IRI here is 33–36 bytes and its 4-byte varlena header, so AllocSet rounds it to a 64-byte chunk plus its 8-byte header: 72 bytes an id. Measured past each call's starting RssAnon, `components` holds 1,031 / 314 MB (717 MB more for 10M ids: 72 bytes each) and `minimum_spanning_forest` 2,030 / 646 MB (1,384 MB more for 20.0M ids: 69 bytes each); at half scale 78 and 68 bytes. With `bigint` ids the datum is by value and only the `Vec` is held (24 and 40 bytes a row). *The fix* (not built here): return the rows lazily, a `TableIterator` over the kernel's result (the labels, 4 bytes a node, or the forest's edges) and the loaded version, which makes each row's external ids in `next()`. pgrx calls `next()` after the first row in the caller's memory context, which for a function in `FROM` is the per-tuple context that `ExecMakeTableFunctionResult` resets before each call, and the row is then copied into the tuplestore (bounded by `work_mem`, then spilled). The datums would then live one row each, and no `Vec` of rows would be held: here the text peaks would fall by about 1.0 and 1.8 GB (the datums and the `Vec`) and the `bigint` ones by about 0.24 and 0.40 GB (the `Vec`), which leaves the kernel's result (4 bytes a node, or 16 bytes a forest edge). Other functions that build every row up front before they return one could be made lazy the same way.
+- *Why `components` and `minimum_spanning_forest` peak higher.* Both build all their rows before they return the first: a `Vec` of `(AnyElement, …)` whose text datums are each `palloc`ed (`cstring_to_text_with_len`) in the set-returning function's multi-call memory context, which lives until the call ends. An IRI here is 33–36 bytes and its 4-byte varlena header, so AllocSet rounds it to a 64-byte chunk plus its 8-byte header: 72 bytes an id. Measured past each call's starting RssAnon, `components` holds 1,031 / 314 MB (717 MB more for 10M ids: 72 bytes each) and `minimum_spanning_forest` 2,030 / 646 MB (1,384 MB more for 20.0M ids: 69 bytes each); at half scale 78 and 68 bytes. With `bigint` ids the datum is by value and only the `Vec` is held (24 and 40 bytes a row). *Fixed since* (§5.7, the half-scale *Peaks* note above, which has the mechanism and the tests): `components`, `minimum_spanning_forest`, `vector_topk` and `most_similar` now make each row's external ids as the row is returned, so only the current row's are held. The figures in this table are from before the fix and were not re-measured at full scale. At half scale the text peaks fell from 1.28 and 1.77 GB to 0.84 and 1.10 GB (`bench/results/soak/a-scale2*-m10-lazy.json`); here they should fall by about the datums' 0.72 and 1.38 GB.
 - *Disk.* The community detections wrote, text / `bigint`: `louvain` 4.31 / 4.64 GB of segments and 6.14 / 6.14 GB of WAL, `leiden` 9.51 / 13.10 GB and 15.09 / 15.06 GB (the harness drops the edge table and the first version first), within the disk.
 
 At full scale every §12 target was met for both id types except `reachable_set` (13 / 19M edges/s, against 50M) and `bfs` against the recursive CTE (5.2 / 10.5×, against 20×); `components` (10.6 s) also misses its 10 s target with text ids only, which the lazy rows above would not change (it is time, not memory: the datums are built either way).
@@ -1742,7 +1742,8 @@ Test it by installing as a non-superuser in CI and running the suite. **Make the
   - many backends refreshing and querying one graph at once;
   - physical streaming replicas, which §5.1 claims and nothing tests;
   - crash recovery during `refresh`;
-  - SQL-surface fuzzing (sqlsmith-style), beside the segment decoder fuzzing that exists.
+  - SQL-surface fuzzing (sqlsmith-style), beside the segment decoder fuzzing that exists;
+  - the two set-returning functions that still build every row's external ids before returning the first, `walk` and `neighbours` (§12, text node ids): made as each row is returned, as the others are.
 - Distribution: CI builds and tests on arm64 as well as x86_64 (most hosted Postgres runs on arm64), prebuilt packages for PG 15–18 on both, and a PGXN release.
 
 **M12 — 1.0.0: the stable surface.** A review of the whole SQL surface before it is frozen:

bench/results/soak/README.md

@@ -392,3 +392,21 @@ free at the start, and neither run came within the floor.
         --pg-config /var/tmp/m10-pgtree/usr/lib/postgresql/16/bin/pg_config --parent /var/tmp/m10-soak \
         --extension-revision 7934941 --extension-profile release --threads 4 --statement-timeout 20min
     bench/soak.py a --ids bigint --label=-m10 …   # otherwise as above
+
+## Rows made as they are returned, at half scale (§5.7, §12)
+
+`a-scale2-text-m10-lazy.json` then `a-scale2-m10-lazy.json`: case a at
+half scale (5M nodes, 50M edges) with IRIs and with `bigint` ids, after
+`components`, `minimum_spanning_forest`, `vector_topk` and
+`most_similar` were changed to make each row's external ids as it is
+returned, to compare their peaks with `a-scale2-text-m10.json` and
+`a-scale2-m10.json`. PostgreSQL 16.15 from a private tree, a release
+build of 8d759e5, harness 8d759e5, the same 4-vCPU Xeon, one after the
+other on 2026-10-24, load average 1.0–2.9 (another agent's work during
+the bigint run):
+
+    bench/soak.py a --ids text --label=-m10-lazy --scale 2 --no-cte --no-eigen-steps \
+        --eigen-max-iter 200 --skip communities --skip second-version --disk-floor 3221225472 \
+        --pg-config /var/tmp/m10-pgtree/usr/lib/postgresql/16/bin/pg_config --parent /var/tmp/m10-soak \
+        --extension-revision 8d759e5 --extension-profile release --threads 4 --statement-timeout 20min
+    bench/soak.py a --ids bigint --label=-m10-lazy …   # otherwise as above