[wfaithfull]

M10 step 4 (DESIGN.md §5.7, §14 item 41): text node ids work everywhere an external id is mapped, built on the TextKeys id map that merged in #58. Design decisions and deviations are recorded in §5.7, "Built (step 4)".

What changes

• Id types. pg_warren is generic over the id map's key storage: FixedKeys<i32>, FixedKeys<i64>, FixedKeys<[u8; 16]> and TextKeys. The old IdDatum trait is split by concern into four:
  • source::IdDatum
  • source::InPlaceId
  • segment::StoredIds
  • cache::CachedIds
• Storage. A text version's id map is stored as the id_bytes, id_offsets and id_order segments. In a shared-cache region these are arrays 7–9; the region format is still 1.
• Build intake. For every id type, edge rows are copied into batches (EdgeBatch, 65,536 rows or 1/32 of the build budget). Each batch's endpoints are mapped on the compute pool, through a TextIndex charged to the build budget. A property test checks that this gives exactly the row-by-row edge list.
  • The compute pool moved from compute_sql to interrupts::pool, so build can use it. CLAUDE.md's dependency list gains interrupts → build.
• Bulk lookups in one call. The first 65,536 lookups use binary search. After that, a text graph builds a lookup index charged to warren.max_compute_memory (80 MB at 10M nodes). This applies to:
  • vector_from_query
  • gather's messages
  • similarity's pairs
• Encoding. An id is checked against the database encoding when its text value is built: 22021 if it is invalid, 54000 if it is longer than a text can hold. A load checks only the id_bytes header's encoding (55000), so a version restored into a database with another encoding is refused.

Tests

• test/sql/text_ids.sql: runs every function that maps ids on a text graph and on the same graph with bigint ids, and compares the results.
• a4_typed_retrieval_iris.sql: the Appendix A.4 use case written with IRIs, checked against recursive CTEs.
• shared_cache_text.sql: shares a text version across backends.
• Dump and upgrade: dump.py and upgrade.py now cover a text graph.
• Soak: soak.py --ids text is new.

Run locally:

• run.py: 55/55 plain, 55/55 --trusted, 55/55 --shared-cache preload.
• pgrx: 721 on PG 16, 721 on PG 18, 719 on PG 15.
• dump.py and upgrade.py pass.
• Miri, with CI's flags: 415 passed. The new pool tests are kept out of Miri because of crossbeam-epoch's known report, as the crate already does for its other pool tests.
• fmt, clippy and rustdoc are clean.

Soak: 5M nodes, 50M edges (half scale for disk, §12)

	text	bigint
version size (id map)	719.5 MB (238.9 MB)	560.6 MB (80 MB)
refresh	56.4 s	41.8 s
cold load (private)	1.03 s	0.77 s
bfs depth 6	0.95 s	0.49 s
similarity, 100k pairs	1.31 s	0.41 s
pagerank	9.1 s	6.7 s

• The build takes about 35% longer with text ids.
• Calls that return one row per id take about twice as long, because each text value is built and its encoding checked.
• Kernels that return no ids are unchanged. The pagerank gap is probably noise: the machine had a load average of 2–4.
• The full 10M text soak needs about 16 GB of free disk. It is still to do.

---

Generated by Claude Code