From 7d129002bbb621056cf72796bc63a2cad8e83241 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 24 Oct 2026 10:11:35 +0000 Subject: [PATCH 1/4] release: 0.6.0 (M10, identity, lifecycle and visibility) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CHANGELOG's 0.6.0 section is final (dated 2026-10-24) and reads as release notes in 0.5.0's order: an introduction to M10, then upgrading, behaviour changes, new, fixes and performance. It now covers every M10 pull request (#53-#63): text node ids (#56, #58, #60), set_owner (#55), dropped roles (#57), quotas (#59), observability and progress (#61), the 10M-node soak (#62) and rows made as they are returned (#63). The upgrade list names everything sql/pg_warren--0.5.0--0.6.0.sql does: twelve functions created, the quota's column, table and index, and the library's eleven PL/pgSQL functions replaced to report progress; text ids need no catalog change. Performance records the soak with IRIs against bigint ids, the build's parallel edge intake (79 s against 0.5.0's 173 s), 0.5.0's Louvain and Leiden fixes run at 10M nodes, and the lazy rows' peaks. README's Status describes 0.6.0 as released, with its caveats (walk and neighbours still build their rows first; other backends' rows need the preload on 15 and 16); the install checks out v0.6.0; "Not there yet" starts at M11. docs/semantics.md documents text ids, and docs/limits.md that warren.threads also maps a refresh's edge endpoints. DESIGN §13 marks M10 done, and §5.7 steps 4 and 10. No code or SQL changes: this is the commit to tag v0.6.0. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke --- CHANGELOG.md | 137 ++++++++++++++++++++++++++++++++++++++++++++-- DESIGN.md | 14 ++++- README.md | 36 ++++++++---- docs/limits.md | 2 +- docs/semantics.md | 31 ++++++++++- 5 files changed, 200 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 83a8e81c..8e1396c8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,9 +1,94 @@ # Changelog -## 0.6.0 — in development +## 0.6.0 — identity, lifecycle and visibility (2026-10-24) -### Added +Milestone M10 of DESIGN.md §13. Nodes can be keyed on strings: `text` is +a fourth id type, for IRIs and slugs, and every function takes and +returns `text` ids with no new signature. A graph's ownership can be +transferred (`warren.set_owner`), and the graphs, vectors and reader +entries of a dropped role no longer linger unseen: `reassign_owned` and +`drop_owned` run before `DROP OWNED`, a superuser may opt into a `DROP +OWNED` trigger, and `orphans()` and `drop_orphans()` find what is left. A +graph's owner can bound the vectors other roles persist on it. A +monitoring query can see what each backend's cache holds and counts +(`warren.cache()`, `warren.backends()`), and how far each long call has +got (`warren.progress()`). In the 100M-edge soak with IRIs as node ids, a +text graph's version is 28% larger than with `bigint` ids and builds in +1.39× the time; `pagerank` and `eigen` are within 12%, and the +community detections within 3%. The build itself, for every id type, +now maps edge endpoints on the compute pool, and the `bigint` graph +builds in 79 s where 0.5.0's soak took 173 s (under Performance). +### Upgrading + +- **Run `ALTER EXTENSION pg_warren UPDATE;` in every database that has the + extension**, after installing the new build and before using the new + version. The new library is loaded at once, but the catalog changes only + with the update, so calls in between can fail. The update: + - creates twelve functions: `set_owner`; `reassign_owned`, + `drop_owned`, `on_drop_owned`, `orphans` and `drop_orphans`; + `set_vector_quota` and `vector_usage`; `cache` and `backends`; + `progress` and `report_progress` (all under New); + - adds the column `warren.graphs.reader_vector_quota` (existing graphs + have no quota), the table `warren.vector_quotas` (dumped with the + extension) and an index on `warren.vectors (graph_id, created_by)`; + - replaces the library's PL/pgSQL functions, `aggregate_neighbours`, + `components_vector`, `eigenvector_centrality`, `hits`, + `katz_centrality`, `label_propagation`, `label_propagation_majority`, + `pagerank`, `sssp_vector`, `louvain` and `leiden`, with the same + signatures, so that they report their progress. Their results are + unchanged. +- The update never creates the `DROP OWNED` event trigger: a superuser + opts into it (under New). +- `text` ids change nothing in the catalog. Existing `integer`, `bigint` + and `uuid` graphs, their segments and their shared versions are + unchanged, and read, refresh and compact as before. A build older than + 0.6.0 cannot read a `text` graph, and downgrading is not supported. +- The shared cache's registry is unchanged, so on PostgreSQL 17 and 18 + versions stay shared across the upgrade without a restart. + +### Behaviour changes + +- `refresh` maps each batch of edge rows' endpoints to dense ids on the + compute pool, on up to `warren.threads` threads, for every id type; the + main thread still reads the rows. The edge list, and so the version, is + the one the row-by-row intake made. Each batch is charged to + `warren.max_build_memory`. +- `grant`, `revoke` and `drop_graph` take the graph's row lock, as + `refresh` and the other writers do, so they wait behind a running + `refresh` of the same graph (under Fixes). +- `components`, `minimum_spanning_forest`, `vector_topk` and + `most_similar` make each row's external ids as the row is returned, + rather than all rows before the first. Their rows and order are + unchanged. Such a call keeps the version it reads, private or mapped + from the shared cache, until its last row is read or it ends: a cursor + on one, read across statements, keeps the version after the statement + that opened it, even once it is evicted (under Performance). + +### New + +- **`text` node ids** (DESIGN.md §5.7, §14 items 41 and 42; + docs/semantics.md, "Text ids"). A graph's node query + may return `text` or `varchar` ids, such as IRIs or slugs; its id type + is then `text`. Every function that takes or returns ids does so in + `text`, with no new signature: a string literal needs its cast + (`warren.neighbours('kg', 'https://example.org/a'::text)`), `ARRAY['…']` + is already `text[]`, and the type witness is `NULL::text`. + - Ids compare bytewise in the database encoding (`COLLATE "C"` order), + which is SQL's own equality under every deterministic collation. An + id column whose collation is nondeterministic is refused by + `create_graph` and `refresh` (42804, hinting `COLLATE "C"` or + `lower()`), as are `bpchar`, `name` and `citext` ids, each with the + cast to write. There is no length limit beyond memory. + - A text version stores each id's bytes once, in dense-id order, with + their offsets and their sorted order: at 10M nodes 120 MB plus the + ids' bytes, against 240 MB for `bigint`. It is shared across backends + and read by parallel workers like any other version. One restored + into a database of another encoding is refused (55000). + - Calls that map more than 65,536 ids (`vector_from_query`, `gather`'s + messages, `similarity`'s pairs) build a hash index of the id map once + per call, charged to `warren.max_compute_memory` (80 MB at 10M + nodes), and keep the binary search if it does not fit. - **`warren.set_owner(name, owner regrole)`** gives a graph a new owner, as `ALTER MATERIALIZED VIEW … OWNER TO` does (DESIGN.md §5.7, §14 item 43). The caller must be a member of the current owner and able to `SET ROLE` @@ -36,7 +121,6 @@ owner (or a member with its privileges or the right to `SET ROLE` to it) as well as by a superuser (`drop_graph`, `drop_orphans`), and adopted by a superuser with `warren.set_owner`, which refused it before ("the owner no longer exists"). - - A `drop_vector`, a replacement or `compact`'s move that checked a vector's creator, and then waited behind a `reassign_owned` that changed it, fails with 40001 instead of deleting the new creator's vector. @@ -110,7 +194,7 @@ extension), an index on `warren.vectors (graph_id, created_by)` and the two functions; existing graphs have no quota. -### Fixed +### Fixes - `refresh`, `compact`, `promote`, `drop_version`, `grant`, `revoke`, `drop_graph` and `contract` into an existing quotient check ownership @@ -119,6 +203,51 @@ transfer ran with an ownership its caller no longer had (harness test `set_owner_race`). +### Performance + +- **The 100M-edge soak with `text` ids** (DESIGN.md §12, "Text node ids"; + `bench/results/soak/a-text-m10.json` and `a-m10.json`; PG 16, 4 vCPUs, + `warren.threads = 4`, private cache). Case a with IRIs of 33–36 bytes as + node ids, against the same graph with `bigint` ids, every stage run: + - the version is 1.44 GB against 1.12 GB, and `refresh` takes 110.2 s + against 79.1 s, peaking 0.25 GB higher; + - calls that return a row per reached node build each id's `text` datum: + `bfs` (1.47M rows) 1.42 s against 0.66 s, `components` 10.6 s against + 5.2 s, `reachable_set` 1.42× as long; + - `similarity` of 100,000 pairs builds its per-call index: 2.15 s + against 0.47 s; + - `pagerank` and `eigen` are within 12%; `eigenvector_centrality`, + `katz_centrality` and `hits` take 1.2–1.4× as long, the rest of the + library and `louvain` and `leiden` within 8%, with the same modularity + to 0.001. + + §12's targets missed there, for both id types: `reachable_set` (13 and + 19M edges/s, against 50M) and `bfs` against the recursive CTE (5.2× and + 10.5× faster, against 20×); with `text` ids, `components` also misses + its 10 s. +- **The build maps edge endpoints in parallel** (under Behaviour changes). + In the same soak the `bigint` graph builds in 79.1 s from its table + (0.5.0's soak: 173.1 s), 99.1 s from the generator query (184.5 s) and + 88.3 s again with 0.1% fewer edges (165.3 s), on the same machine. +- **0.5.0's fixes to level graphs and Leiden's refinement at 10M nodes**, + run at that scale for the first time in the same soak: on the + one-relation graph with little community structure, `leiden` completes + in 804 s (16 levels, modularity 0.186, an 8.10 GB peak), where 0.5.0's + soak was refused (53400), and `louvain` takes 528 s (627.5 s). The two + wrote 13.1 and 4.6 GB of level segments and 15.1 and 6.1 GB of WAL, + held until the run's transaction ends (docs/limits.md). +- **Rows made as they are returned** (DESIGN.md §12): `components`, + `minimum_spanning_forest`, `vector_topk` and `most_similar` no longer + hold every row's ids until the call ends. At 5M nodes and 50M edges, + `components` peaks at 0.84 GB with `text` ids (1.28 GB before) and + 0.73 GB with `bigint` (0.80 GB), and `minimum_spanning_forest` at + 1.10 GB with `text` (1.77 GB), its `bigint` peak unchanged; times are + unchanged. `walk` and `neighbours` still build their rows first, bounded + by their arguments or by one node's degree (M11). +- Progress reporting costs a few atomic stores per statement and per + poll: within noise for one `degree` per transaction and for `pagerank`, + and about 1.5 µs a `report_progress` call. + ## 0.5.0 — the shared cache (2026-10-23) Milestone M9 of DESIGN.md §13, the first on the road to 1.0. A loaded diff --git a/DESIGN.md b/DESIGN.md index 1294891c..380f45e7 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -430,7 +430,7 @@ M9 (0.5.0) moves loaded versions into dynamic shared memory so backends share on ### 5.7 Identity, lifecycle and visibility (M10 design) -*Designed 2026-10-23 for M10 (0.6.0), after 0.5.0's release; built so far: steps 1 (the spike), 2 (`IdMap`), 3 (`TextKeys`), 4 (text graphs end to end), 5 (ownership transfer), 6 (dropped roles), 7 (quotas), 8 (the activity table and the cache counters) and 9 (progress). The decisions are §14 items 41–49: items 44, 45 and 46 were decided by the owner on 2026-10-23, as recommended, and the rest are decided here. Claims about Postgres were read in the PG 15–18 sources (`tcop/utility.c`, `tcop/cmdtaglist.h`, `catalog/pg_shdepend.c`, `commands/event_trigger.c`, `utils/backend_progress.h`, `utils/adt/pgstatfuncs.c`, `utils/wait_event.h`, `utils/misc/guc_tables.c`); where a claim needed code to confirm it, step 1 of the work breakdown did so on PG 15, 16 and 18 (Spike results, below), and the text reflects what it found.* M10 has five items, which share little code. `text` node ids change the id map and everything that maps external ids. Ownership transfer, cleanup after a dropped role, and quotas change the catalog's lifecycle. Observability adds a per-backend table in shared memory beside the registry of §5.6. Each part states what Postgres allows an extension on PG 15–18 and where pg_warren stops. Nothing in M10 changes a kernel, the CSR layout, the segment format of existing id types, or the registry's format, so 0.6.0 needs no restart to keep sharing on PG 17 and 18 (§5.6). +*Designed 2026-10-23 for M10, after 0.5.0's release, and released in 0.6.0 (2026-10-24): every step of the work breakdown below is done. The decisions are §14 items 41–49: items 44, 45 and 46 were decided by the owner on 2026-10-23, as recommended, and the rest are decided here. Claims about Postgres were read in the PG 15–18 sources (`tcop/utility.c`, `tcop/cmdtaglist.h`, `catalog/pg_shdepend.c`, `commands/event_trigger.c`, `utils/backend_progress.h`, `utils/adt/pgstatfuncs.c`, `utils/wait_event.h`, `utils/misc/guc_tables.c`); where a claim needed code to confirm it, step 1 of the work breakdown did so on PG 15, 16 and 18 (Spike results, below), and the text reflects what it found.* M10 has five items, which share little code. `text` node ids change the id map and everything that maps external ids. Ownership transfer, cleanup after a dropped role, and quotas change the catalog's lifecycle. Observability adds a per-backend table in shared memory beside the registry of §5.6. Each part states what Postgres allows an extension on PG 15–18 and where pg_warren stops. Nothing in M10 changes a kernel, the CSR layout, the segment format of existing id types, or the registry's format, so 0.6.0 needs no restart to keep sharing on PG 17 and 18 (§5.6). #### Text node ids (§14 items 41, 42) @@ -703,13 +703,13 @@ Each step is a pull request that leaves `main` green. - The text id map at 10M nodes: build, sort, merge and hash-index time, lookup latency, and memory, at *L* = 24, 48 and 64. 2. *`warren-core`: `IdMap` with `FixedKeys`,* with no behaviour change: every existing test and proptest passes unchanged, and Miri. *Done:* tests changed only where the types did (`IdMap>`, and `external_id` returning a reference). 3. *`warren-core`: `TextKeys`:* the rebuild with its arena and hash index, the three segment kinds, validation, renumbering, compaction, `diff` and alignment, with the tests above. **Done** (see "Code (item 41)", *Built (step 3)*). -4. *`pg_warren`: text graphs end to end:* `IdType::Text`; the collation, type and encoding checks; the build's intake; the region's arrays and the loader; `external_ids`; every SQL function. Also the `text_ids` tests, and the soak at 10M nodes with synthetic IRIs (`bench/soak.py --ids text`), recorded in §12. +4. *`pg_warren`: text graphs end to end:* `IdType::Text`; the collation, type and encoding checks; the build's intake; the region's arrays and the loader; `external_ids`; every SQL function. Also the `text_ids` tests, and the soak at 10M nodes with synthetic IRIs (`bench/soak.py --ids text`), recorded in §12. **Done** (see "Text node ids", *Built (step 4)*; the soak at half and full scale in §12, and the rows made as they are returned). 5. *Ownership transfer:* `set_owner`, ownership re-checked after the lock, and `set_owner_race`. **Done** (see "Ownership transfer", *Implemented*). 6. *Dropped roles:* `drop_owned`, `reassign_owned`, `orphans`, `drop_orphans` and `on_drop_owned`, as items 44 and 45 decide; `docs/semantics.md`'s procedure; the trusted-install and dump tests. **Done** (see "When an owner role is dropped", *Implemented*). 7. *Quotas:* the schema (`graphs.reader_vector_quota`, `warren.vector_quotas`, the index), `set_vector_quota`, `vector_usage` and enforcement, as item 46 decides; the race tests. **Done** (see "Quotas on readers' vectors", *Implemented*). 8. *The activity table and the cache counters:* `warren.cache()`, `warren.backends()` and the wait event. **Done** (see "Observability", *Implemented (step 8)*). 9. *Progress:* the `warren-core` port, reporting from every call in the table above, `warren.progress()` and `warren.report_progress()`, and the library calling it; the cost measured. **Done** (see "Observability", *Implemented (step 9)*). -10. *Release 0.6.0:* the install script, `sql/pg_warren--0.5.0--0.6.0.sql`, `sql/released.sha256`, and `test/upgrade.py` from every release. +10. *Release 0.6.0:* the install script, `sql/pg_warren--0.5.0--0.6.0.sql`, `sql/released.sha256`, and `test/upgrade.py` from every release. *Done (2026-10-24):* `sql/pg_warren--0.6.0.sql` is frozen, `sql/pg_warren--0.5.0--0.6.0.sql` creates the twelve functions, the quota's column, table and index, and replaces the library's PL/pgSQL so that it reports progress, and `test/upgrade.py` checks every release's path to the version in development. ## 6. Compute layer @@ -1732,6 +1732,14 @@ Test it by installing as a non-superuser in CI and running the suite. **Make the - Per-role quotas on readers' vectors. - Observability: what each backend, or the shared cache, holds and its bytes; cache hits, misses and evictions; and the progress of a long `refresh`, `eigen` or `louvain`, through a view or function a monitoring query can poll. +**Done (0.6.0, 2026-10-24):** +- `text` node ids as §5.7 designs them (§14 items 41 and 42): the id map's three arrays, bytewise comparison with nondeterministic collations refused, the encoding recorded and checked, and every function taking and returning `text` through the existing `anyelement` dispatch; the build maps edge endpoints on the compute pool for every id type, and calls that map more than 65,536 ids index the id map once per call; +- ownership transfer, `warren.set_owner` (item 43), with ownership re-checked once a writer holds the graph's row lock; +- cleanup after a dropped role (items 44 and 45): `reassign_owned`, `drop_owned`, `orphans`, `drop_orphans`, and `on_drop_owned` for a superuser's opt-in `DROP OWNED` trigger, which the extension never creates; +- quotas on readers' persisted vectors (items 46 and 47): `set_vector_quota`, `vector_usage`, enforced under a lock per (graph, role); +- observability (items 48 and 49): the activity table beside the registry, `warren.cache()`, `warren.backends()` and the `WarrenSharedLoad` wait event, `warren.progress()` and `warren.report_progress()`, reported by every long call and the library; +- measured (§12): the 100M-edge soak with IRIs against `bigint` ids, at half and full scale, every stage run: the text version is 28% larger and builds in 1.39× the time, `pagerank`, `eigen` and the community detections are within 12% and the library within 1.0–1.4×, and calls that return a row per node about double; the `bigint` build takes 79 s, from 173 s in 0.5.0's soak; Louvain and Leiden complete at 10M nodes. `components`, `minimum_spanning_forest`, `vector_topk` and `most_similar` make their rows as they are returned; `walk` and `neighbours` are left to M11. Front coding of text ids is deferred to M11's real IRIs (item 41). + **M11 — 0.7.0: proven on real graphs and real deployments.** - Benchmarks on real data, not only synthetic graphs: - SNAP's LiveJournal, Orkut and road networks (§12); diff --git a/README.md b/README.md index 543c44e1..65c8e4fa 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,7 @@ the server headers (`postgresql-server-dev-NN`), `clang`/`libclang-dev`, and cargo-pgrx at exactly the version the extension uses: ```sh -git checkout v0.5.0 # the latest release; main is the version in development +git checkout v0.6.0 # the latest release; main is the version in development cargo install cargo-pgrx --version 0.19.3 --locked cargo pgrx init --pg16 "$(command -v pg_config)" cd crates/pg_warren @@ -282,20 +282,34 @@ Every SQL example in the README and `docs/` runs in CI, in slower; and Louvain and Leiden at 10M nodes still write several GB of level graphs ([docs/limits.md](docs/limits.md)). -- **0.6.0** (in development, M10): identity, lifecycle and visibility: - `text` node ids, ownership transfer, cleanup when an owner role is - dropped, quotas on readers' vectors, and observability (DESIGN.md §13). +- **0.6.0**: identity, lifecycle and visibility, milestone M10. Nodes + can be keyed on strings: `text` is a fourth id type, for IRIs and slugs, + compared bytewise (docs/semantics.md, "Text ids"). A graph's ownership + can be transferred (`warren.set_owner`); a dropped owner role's graphs, + vectors and reader entries are handled by `warren.reassign_owned` and + `warren.drop_owned`, an optional superuser's `DROP OWNED` trigger, and + `warren.orphans()` and `warren.drop_orphans()`; a graph's owner can + bound the vectors other roles persist on it + (`warren.set_vector_quota`). `warren.cache()`, `warren.backends()` and + `warren.progress()` show what each backend's cache holds and counts and + how far each long call has got (DESIGN.md §5.7). Upgrading needs `ALTER + EXTENSION pg_warren UPDATE` (see the changelog). + + In the 100M-edge soak with IRIs as node ids, the version is 28% larger + than with `bigint` ids and the build takes 1.39× as long; `pagerank`, + `eigen` and the community detections are within 12%, the library within + 1.0–1.4×, and calls that return a row per node (`bfs`, `components`) + take about twice as long, since they build each id's `text`. The build maps edge endpoints on the compute pool for every id + type: the `bigint` graph builds in 79 s, from 173 s in 0.5.0's soak. The + caveats: `walk` and `neighbours` still build their rows before + returning the first; and on PostgreSQL 15 and 16, other backends' rows + in `warren.backends()` and `warren.progress()` need + `shared_preload_libraries` ([docs/limits.md](docs/limits.md)). See [CHANGELOG.md](CHANGELOG.md). This is pre-1.0 software: the SQL surface may still change between minor versions, with an upgrade script each time. -Not there yet: the rest of the road to 1.0 (DESIGN.md §13) is text ids, -and observability (M10; ownership transfer, `warren.set_owner`, cleanup -after a dropped role, `warren.reassign_owned`, `warren.drop_owned`, the -optional `DROP OWNED` trigger and `warren.drop_orphans`, quotas on readers' -vectors, `warren.set_vector_quota`, the cache's views, `warren.cache()` and -`warren.backends()`, and progress, `warren.progress()` and -`warren.report_progress()`, are built for 0.6.0); +Not there yet: the rest of the road to 1.0 (DESIGN.md §13) is benchmarks and robustness on real graphs and deployments, and packages (M11); and a reviewed, frozen SQL surface (M12, 1.0.0). Incremental maintenance, general sparse matrix algebra (contraction's `PᵀAP` is the diff --git a/docs/limits.md b/docs/limits.md index c9bd326c..5b1612f9 100644 --- a/docs/limits.md +++ b/docs/limits.md @@ -11,7 +11,7 @@ is set in the server's configuration and takes effect on reload. | Setting | Default | Range | What it bounds | When exceeded | |---|---|---|---|---| -| `warren.threads` | half the cores, at least 1 | 1–1024 | threads a compute kernel runs on (`spmv`, `spmm`, `vector_map`, reductions, `gather`, `eigen`, `distance`, `triangles`, the sort in `core_number`, the centralities' sources and the similarities) | — | +| `warren.threads` | half the cores, at least 1 | 1–1024 | threads a compute kernel runs on (`spmv`, `spmm`, `vector_map`, reductions, `gather`, `eigen`, `distance`, `triangles`, the sort in `core_number`, the centralities' sources and the similarities), and the mapping of a `refresh`'s edge endpoints | — | | `warren.max_compute_memory` | `4GB` | `64kB` up | memory one call holds: every vector it reads (once, cached or not), every vector and work buffer it allocates, and what its statement holds in the cache past `warren.max_cache_memory` | `configuration_limit_exceeded` (53400), before allocating or loading | | `warren.max_cache_memory` | `4GB` | `64kB` up | what this backend's cache holds *between* statements: cached versions, cached persisted vectors, and every role's transient vectors | versions and persisted vectors no running statement has read are evicted, least recently used first; a transient write that would go over is refused with 53400 | | `warren.shared_cache_memory` | `4GB` | `0` up; `0` turns it off | the cluster's shared versions and persisted vectors, of every database together; set in `postgresql.conf` (or `ALTER SYSTEM`) and reloaded, not per session ([the shared cache](#the-shared-cache)) | unpinned entries no backend maps are evicted, least recently used first; one that still does not fit is loaded into the reading backend's own cache, with a `NOTICE` | diff --git a/docs/semantics.md b/docs/semantics.md index df28de5d..c887589c 100644 --- a/docs/semantics.md +++ b/docs/semantics.md @@ -17,7 +17,8 @@ SELECT warren.create_graph(name, node_query, edge_query, ``` - `node_query` returns one column, the node's **external id**: `integer`, - `bigint` or `uuid`. Its type is the graph's **id type**. + `bigint`, `uuid` or `text` (`varchar` is read as `text`). Its type is + the graph's **id type** ([text ids](#text-ids)). - `edge_query` returns `src, dst` (of the id type), then a `text` **relation** if `has_relations`, then a `double precision` weight if `weighted`. @@ -469,6 +470,10 @@ Functions are polymorphic over the id type and check it at run time. a typed NULL: `warren.components('social', NULL::bigint)`. `gather` takes a second witness for its result type. - `uuid` graphs work the same way (`ARRAY[$1::uuid]`, `NULL::uuid`). +- `text` graphs too (`'https://example.org/a'::text`, `NULL::text`): a + lone string literal needs its `::text`, since Postgres cannot resolve an + untyped literal to `anyelement`, but `ARRAY['…']` is already `text[]`. + A `varchar` argument is widened where the result holds no ids. ```sql SELECT warren.neighbours('social', 3::bigint); @@ -477,6 +482,30 @@ SELECT * FROM warren.components('social', NULL::bigint); SELECT * FROM warren.vector_values('social', 'pr', NULL::bigint); -- after pagerank('social', 'pr') ``` +### Text ids + +- Ids compare **bytewise** in the database encoding, which is `COLLATE "C"` + order, whatever the column's collation. Under a deterministic collation + (every default one) that is SQL's own equality, so an edge joins the + node SQL would join it to. A nondeterministic collation (for example + ICU's case-insensitive ones) makes `'Alice' = 'alice'`, which bytewise + ids cannot follow, so `create_graph` and `refresh` refuse such an id + column (`datatype_mismatch`, 42804): write `id COLLATE "C"` for exact + ids, or normalise in the query (`lower(id)`). `bpchar`, `name` and + `citext` ids are refused too, each with the cast to write. +- The first build numbers dense ids in byte order (`node_order := 'id'`). +- Ids have no length limit beyond memory; the empty string is an id, and a + NULL id is skipped, as for every id type. +- A version records the database encoding, and one restored into a + database of another encoding is refused (55000): create the graph again + there. +- A text graph's id map holds each id's bytes once plus 12 bytes a node + (at 10M nodes, 120 MB plus the ids; `bigint` takes 240 MB). Calls that + map more than 65,536 ids at once (`vector_from_query`, `gather`'s + messages, `similarity`'s pairs) build a hash index of 8 bytes a node for + the call, charged to `warren.max_compute_memory`, and search without it + if it does not fit. + ## Unknown and tombstoned ids An external id the version does not have, or has only as a tombstone, is From c891ee99563e0d944baa866abbb1cbfa2f5e316a Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 24 Oct 2026 10:12:35 +0000 Subject: [PATCH 2/4] release: freeze 0.6.0, develop 0.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit sql/pg_warren--0.6.0.sql is frozen by sql/released.sha256, and sql/released.commits records the commit to tag v0.6.0, 7d12900, for 0.6.0. The workspace version is 0.7.0 (in development) with its generated sql/pg_warren--0.7.0.sql (the same entities as 0.6.0's, in pgrx's order). sql/pg_warren--0.6.0--0.7.0.sql is a header only, since the schema has not changed; cargo pgrx install installs it and ALTER EXTENSION UPDATE runs it. CHANGELOG has 0.7.0, and README's Status names what it is for (M11, proof at scale and in deployment, DESIGN.md §13). test/upgrade.py, which reads the releases from sql/released.sha256 and sql/released.commits, now checks six release paths: 0.1.0 to 0.5.0 chained through 0.6.0, and 0.6.0 -> 0.7.0, each seeded by that release's own build (0.6.0's from 7d12900). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke --- CHANGELOG.md | 2 + Cargo.lock | 4 +- Cargo.toml | 2 +- README.md | 4 + sql/pg_warren--0.6.0--0.7.0.sql | 8 + sql/pg_warren--0.7.0.sql | 2388 +++++++++++++++++++++++++++++++ sql/released.commits | 1 + sql/released.sha256 | 1 + 8 files changed, 2407 insertions(+), 3 deletions(-) create mode 100644 sql/pg_warren--0.6.0--0.7.0.sql create mode 100644 sql/pg_warren--0.7.0.sql diff --git a/CHANGELOG.md b/CHANGELOG.md index 8e1396c8..8401ed2d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,7 @@ # Changelog +## 0.7.0 — in development + ## 0.6.0 — identity, lifecycle and visibility (2026-10-24) Milestone M10 of DESIGN.md §13. Nodes can be keyed on strings: `text` is diff --git a/Cargo.lock b/Cargo.lock index 4c4df854..fbf6fad2 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1286,7 +1286,7 @@ dependencies = [ [[package]] name = "pg_warren" -version = "0.6.0" +version = "0.7.0" dependencies = [ "libc", "pgrx", @@ -2231,7 +2231,7 @@ dependencies = [ [[package]] name = "warren-core" -version = "0.6.0" +version = "0.7.0" dependencies = [ "criterion", "nalgebra", diff --git a/Cargo.toml b/Cargo.toml index 396042c5..1c815f1e 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -8,7 +8,7 @@ exclude = ["fuzz"] default-members = ["crates/warren-core"] [workspace.package] -version = "0.6.0" +version = "0.7.0" edition = "2021" license = "Apache-2.0" repository = "https://github.com/ExaDev/pg_warren" diff --git a/README.md b/README.md index 65c8e4fa..1c060b12 100644 --- a/README.md +++ b/README.md @@ -306,6 +306,10 @@ Every SQL example in the README and `docs/` runs in CI, in in `warren.backends()` and `warren.progress()` need `shared_preload_libraries` ([docs/limits.md](docs/limits.md)). +- **0.7.0** (in development, M11): proof at scale and in deployment: + benchmarks on real graphs, robustness behind connection poolers, on + replicas and through crashes, and packages (DESIGN.md §13). + See [CHANGELOG.md](CHANGELOG.md). This is pre-1.0 software: the SQL surface may still change between minor versions, with an upgrade script each time. diff --git a/sql/pg_warren--0.6.0--0.7.0.sql b/sql/pg_warren--0.6.0--0.7.0.sql new file mode 100644 index 00000000..33b2c33e --- /dev/null +++ b/sql/pg_warren--0.6.0--0.7.0.sql @@ -0,0 +1,8 @@ +-- pg_warren 0.6.0 -> 0.7.0 (DESIGN.md §13; CONTRIBUTING.md "Releases and +-- upgrades"). Hand-maintained: every schema change made after 0.6.0 is +-- mirrored here in the same commit, and test/upgrade.py checks that +-- ALTER EXTENSION pg_warren UPDATE from 0.6.0 yields the catalog of a fresh +-- 0.7.0 install. In a trusted install it runs as the bootstrap superuser +-- (DESIGN.md §7): create only in schema warren, schema-qualify, and use only +-- exact-match pg_catalog operators and functions in what runs here. +\echo Use "ALTER EXTENSION pg_warren UPDATE TO '0.7.0'" to load this file. \quit diff --git a/sql/pg_warren--0.7.0.sql b/sql/pg_warren--0.7.0.sql new file mode 100644 index 00000000..d162c0f5 --- /dev/null +++ b/sql/pg_warren--0.7.0.sql @@ -0,0 +1,2388 @@ +/* */ +/* +This file is auto generated by pgrx. + +The ordering of items is not stable, it is driven by a dependency graph. +*/ +/* */ + +/* */ +-- crates/pg_warren/src/extension.rs:18 +CREATE SCHEMA IF NOT EXISTS warren; /* pg_warren::extension::warren */ +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/library/mod.rs:9 +-- requires: +-- extension::warren + +-- warren.leiden (DESIGN.md §6.7): Leiden community detection (Traag, +-- Waltman and van Eck 2019) as a library over the primitives. Each level +-- moves nodes between communities (warren.local_moving, from the partition +-- the level below found) on the level's graph, refines each community +-- into well-connected parts (warren.leiden_refine), lifts both partitions +-- to `graph` (warren.lift), and contracts `graph` by the refined one +-- (warren.contract) into the next level's graph; the next level starts +-- from the unrefined partition, projected onto it (warren.project, min). +-- As warren.louvain's, every level's graph is a quotient of `graph`, and +-- only one exists at a time. Every community found is connected. As +-- Traag et al.'s, it goes on while there is something to aggregate: it +-- stops when a level's local moving leaves every node alone, when the +-- refinement leaves every node alone (the next level's graph would be this +-- one), or after `max_levels`; in the last two cases it returns the +-- refined partition, so every exit returns a connected one. A level's +-- local moving stops when a sweep raises modularity by no more than +-- `tolerance`. That multilevel pass is one iteration; each later one +-- starts its first level's moving from the partition the one before found, +-- not from each node alone, as igraph's `n_iterations` and leidenalg's +-- `optimise_partition` loop do. The run stops after `iterations`, or +-- after an iteration whose partition is the one it started from: with the +-- same start and seed, the next would find it again. A negative +-- `iterations` runs until then, at most 100 iterations (a WARNING if it +-- is still changing). Arguments and output as warren.louvain's; +-- `randomness` is θ of the refinement. `levels_prefix` keeps every +-- iteration's levels in turn (`_1`, … numbered on across +-- iterations), so the last is `out`. The level's graph, a level graph of +-- warren.contract's (`_warren_level_`), is dropped at the end of +-- each iteration. +CREATE FUNCTION warren.leiden(graph text, "out" text, resolution float8 DEFAULT 1.0, + seed bigint DEFAULT 0, max_levels int DEFAULT 100, + tolerance float8 DEFAULT 1e-7, randomness float8 DEFAULT 0.01, + iterations int DEFAULT 2, weighted boolean DEFAULT NULL, levels_prefix text DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS float8 LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['moved', 'refined', 'initial', 'keys', 'down', 'refined_down', 'prev', 'changed']; + g text := graph; + gv bigint := v; + next_g text; + next_v bigint; + level int; + iteration int := 0; + written int := 0; + nodes bigint; + found bigint; + refined bigint; + q float8; +BEGIN + IF "out" IS NULL OR resolution IS NULL OR seed IS NULL OR max_levels IS NULL OR tolerance IS NULL + OR randomness IS NULL OR iterations IS NULL THEN + RAISE EXCEPTION '% must not be NULL', + CASE WHEN "out" IS NULL THEN 'out' WHEN resolution IS NULL THEN 'resolution' + WHEN seed IS NULL THEN 'seed' WHEN max_levels IS NULL THEN 'max_levels' + WHEN tolerance IS NULL THEN 'tolerance' WHEN randomness IS NULL THEN 'randomness' + ELSE 'iterations' END + USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with("out", '_warren_') OR starts_with(levels_prefix, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF max_levels < 1 THEN + RAISE EXCEPTION 'max_levels must be at least 1, not %', max_levels USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF iterations = 0 THEN + RAISE EXCEPTION 'iterations must be at least 1, or negative to run until the partition is stable, not 0' + USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF tolerance = 'NaN'::float8 OR tolerance < 0 THEN + RAISE EXCEPTION 'tolerance must be a number of at least 0, not %', tolerance + USING ERRCODE = 'invalid_parameter_value'; + END IF; + BEGIN + LOOP + iteration := iteration + 1; + -- A later iteration's first level moves from the last partition. + IF iteration > 1 THEN + PERFORM warren.vector_map(graph, '_warren_leiden_prev', '_warren_leiden_down', false, v); + END IF; + level := 0; + refined := NULL; + LOOP + level := level + 1; + PERFORM warren.report_progress('leiden', graph, format('iteration %s, level %s', iteration, level), + level - 1, max_levels); + nodes := (SELECT (e ->> 'node_count')::bigint FROM jsonb_array_elements(warren.info(g) -> 'versions') e + WHERE (e ->> 'version')::bigint = gv); + IF level > 1 THEN + PERFORM warren.vector_from_query(g, '_warren_leiden_keys', + format('SELECT node_id, node_id FROM warren.id_map(%L, NULL::bigint, %s)', g, gv), 'i64', + persist := false, version := gv); + END IF; + found := warren.local_moving(g, '_warren_leiden_moved', + initial := CASE WHEN level > 1 THEN '_warren_leiden_initial' + WHEN iteration > 1 THEN '_warren_leiden_prev' END, + keys := CASE WHEN level > 1 THEN '_warren_leiden_keys' END, + resolution := resolution, seed := seed, tolerance := tolerance, + weighted := CASE WHEN level = 1 THEN weighted ELSE true END, + persist := false, version := gv); + -- This level's partition, on `graph`. + IF level = 1 THEN + PERFORM warren.vector_map(graph, '_warren_leiden_down', '_warren_leiden_moved', false, v); + ELSE + PERFORM warren.lift(g, '_warren_leiden_moved', graph, '_warren_leiden_down', + persist := false, version := gv); + END IF; + -- Every node alone: the partition is this level's nodes, the refined + -- partition of the level below, which is connected. + IF found < nodes THEN + refined := warren.leiden_refine(g, '_warren_leiden_moved', '_warren_leiden_refined', + keys := CASE WHEN level > 1 THEN '_warren_leiden_keys' END, + resolution := resolution, seed := seed, randomness := randomness, + weighted := CASE WHEN level = 1 THEN weighted ELSE true END, + persist := false, version := gv); + IF level = 1 THEN + PERFORM warren.vector_map(graph, '_warren_leiden_refined_down', '_warren_leiden_refined', false, v); + ELSE + PERFORM warren.lift(g, '_warren_leiden_refined', graph, '_warren_leiden_refined_down', + persist := false, version := gv); + END IF; + -- Stopping here, return the refined partition, which is connected, + -- as Traag et al.'s is; the moved one need not be. + IF level >= max_levels OR refined = nodes THEN + PERFORM warren.vector_map(graph, '_warren_leiden_down', '_warren_leiden_refined_down', false, v); + END IF; + END IF; + IF levels_prefix IS NOT NULL THEN + written := written + 1; + PERFORM warren.vector_map(graph, format('%s_%s', levels_prefix, written), '_warren_leiden_down', true, v); + END IF; + EXIT WHEN found = nodes OR level >= max_levels OR refined = nodes; + -- A level graph of its own, named after its version, a name no + -- graph can have had. + next_v := warren.contract(graph, '_warren_leiden_refined_down', NULL, + weighted := coalesce(weighted, true), version := v); + next_g := '_warren_level_' || next_v; + -- The next level starts from the unrefined partition: every member + -- of a refined community is in the same community, so min is it. + PERFORM warren.project(graph, '_warren_leiden_down', next_g, '_warren_leiden_initial', 'min', + persist := false, version := next_v); + IF g <> graph THEN + PERFORM warren.drop_vector(g, '_warren_leiden_' || part, gv, true) FROM unnest(parts) part; + PERFORM warren.drop_graph(g); + END IF; + g := next_g; + gv := next_v; + END LOOP; + IF g <> graph THEN + PERFORM warren.drop_vector(g, '_warren_leiden_' || part, gv, true) FROM unnest(parts) part; + PERFORM warren.drop_graph(g); + g := graph; + gv := v; + END IF; + EXIT WHEN iteration = iterations; + -- Labels are the least dense id of each community's members, so the + -- partitions are the same exactly when the labels are. + IF iteration > 1 THEN + PERFORM warren.vector_map(graph, '_warren_leiden_changed', + 'CASE WHEN _warren_leiden_down = _warren_leiden_prev THEN 0 ELSE 1 END', false, v); + EXIT WHEN warren.vector_reduce(graph, '_warren_leiden_changed', 'sum', v) = 0; + END IF; + IF iterations < 0 AND iteration = 100 THEN + RAISE WARNING 'leiden did not stabilise in 100 iterations'; + EXIT; + END IF; + END LOOP; + PERFORM warren.vector_map(graph, "out", '_warren_leiden_down', true, v); + q := warren.modularity(graph, "out", resolution, weighted, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_leiden_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_leiden_' || part, v, true) FROM unnest(parts) part; + IF g <> graph THEN + PERFORM warren.drop_vector(g, '_warren_leiden_' || part, gv, true) FROM unnest(parts) part; + PERFORM warren.drop_graph(g); + END IF; + RETURN q; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/step.rs:25 +-- requires: +-- extension::warren + + +CREATE TYPE warren.step AS ( + rels text[], + direction text, + filter text[], + min_hops int, + max_hops int +); + +CREATE FUNCTION warren.steps(steps jsonb) RETURNS warren.step[] +IMMUTABLE STRICT PARALLEL SAFE LANGUAGE plpgsql +SET search_path = pg_catalog, pg_temp +AS $$ +DECLARE + step jsonb; + position int := 0; + key text; + result warren.step[] := '{}'; +BEGIN + IF jsonb_typeof(steps) <> 'array' THEN + RAISE EXCEPTION USING ERRCODE = 'invalid_parameter_value', + MESSAGE = format('steps must be a JSON array of step objects, not %s', + jsonb_typeof(steps)); + END IF; + FOR step IN SELECT e FROM jsonb_array_elements(steps) AS x(e) LOOP + position := position + 1; + IF jsonb_typeof(step) <> 'object' THEN + RAISE EXCEPTION USING ERRCODE = 'invalid_parameter_value', + MESSAGE = format('step %s must be an object, not %s', position, + jsonb_typeof(step)); + END IF; + FOR key IN SELECT jsonb_object_keys(step) LOOP + IF key NOT IN ('rels', 'direction', 'filter', 'min_hops', 'max_hops') THEN + RAISE EXCEPTION USING ERRCODE = 'invalid_parameter_value', + MESSAGE = format('step %s has an unknown key "%s"', position, key); + END IF; + END LOOP; + FOREACH key IN ARRAY ARRAY['rels', 'filter'] LOOP + IF jsonb_typeof(step -> key) NOT IN ('array', 'null') THEN + RAISE EXCEPTION USING ERRCODE = 'invalid_parameter_value', + MESSAGE = format('step %s: %s must be an array or null, not %s', position, + key, jsonb_typeof(step -> key)); + END IF; + END LOOP; + result := result || ROW( + CASE WHEN jsonb_typeof(step -> 'rels') = 'array' THEN + ARRAY(SELECT jsonb_array_elements_text(step -> 'rels')) END, + coalesce(step ->> 'direction', 'out'), + CASE WHEN jsonb_typeof(step -> 'filter') = 'array' THEN + ARRAY(SELECT jsonb_array_elements_text(step -> 'filter')) END, + coalesce((step ->> 'min_hops')::int, 1), + (step ->> 'max_hops')::int)::warren.step; + END LOOP; + RETURN result; +END +$$; + +CREATE CAST (jsonb AS warren.step[]) WITH FUNCTION warren.steps(jsonb); +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:7 +-- requires: +-- extension::warren + +-- warren.pagerank (DESIGN.md §6.7): PageRank by power iteration, as +-- NetworkX computes it. Rank flows along edges, split by out-weight (the +-- edge count when unweighted); dangling nodes' rank, and the 1 - damping +-- teleport, go to the personalisation: uniform, or proportional to vector +-- `seed`. Stops when the L1 change falls below (live nodes) × tol, or after +-- max_iter iterations. Writes vector `out` (f64, summing to 1) and returns +-- the iterations run. Constants enter expressions as constant vectors, +-- never as text, so no client setting (extra_float_digits) rounds them. +CREATE FUNCTION warren.pagerank(graph text, "out" text, damping float8 DEFAULT 0.85, + tol float8 DEFAULT 1e-6, max_iter int DEFAULT 100, + seed text DEFAULT NULL, rels text[] DEFAULT NULL, + weighted boolean DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['one', 'out', 'x', 'p', 'total', 'damping', 'teleport', 'msg', 'dangling_x', + 'dangling', 'y', 'next']; + nodes float8; + dangling float8; + delta float8; + i int := 0; +BEGIN + IF starts_with("out", '_warren_') OR starts_with(seed, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + BEGIN + -- Every live node's out-weight: the row sums of A. + PERFORM warren.vector_const(graph, '_warren_pagerank_one', 1, persist := false, version := v); + PERFORM warren.spmv(graph, '_warren_pagerank_one', '_warren_pagerank_out', 'plus_first', + rels := rels, weighted := weighted, persist := false, version := v); + nodes := warren.vector_reduce(graph, '_warren_pagerank_one', 'sum', v); + PERFORM warren.vector_const(graph, '_warren_pagerank_x', 1.0 / nodes, persist := false, version := v); + IF seed IS NULL THEN + PERFORM warren.vector_map(graph, '_warren_pagerank_p', '_warren_pagerank_x', false, v); + ELSE + -- Times the reciprocal: a constant vector is 0 at tombstoned rows, + -- where vector_map evaluates too, so dividing by one would fail. + PERFORM warren.vector_const(graph, '_warren_pagerank_total', 1.0 / warren.vector_reduce(graph, seed, 'sum', v), + persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_pagerank_p', format('%I * _warren_pagerank_total', seed), false, v); + END IF; + PERFORM warren.vector_const(graph, '_warren_pagerank_damping', damping, persist := false, version := v); + PERFORM warren.vector_const(graph, '_warren_pagerank_teleport', 1 - damping, persist := false, version := v); + WHILE i < max_iter LOOP + i := i + 1; + PERFORM warren.report_progress('pagerank', graph, 'iterating', i - 1, max_iter); + PERFORM warren.vector_map(graph, '_warren_pagerank_msg', + 'CASE WHEN _warren_pagerank_out > 0 THEN _warren_pagerank_x / _warren_pagerank_out ELSE 0 END', false, v); + PERFORM warren.vector_map(graph, '_warren_pagerank_dangling_x', + 'CASE WHEN _warren_pagerank_out > 0 THEN 0 ELSE _warren_pagerank_x END', false, v); + dangling := warren.vector_reduce(graph, '_warren_pagerank_dangling_x', 'sum', v); + PERFORM warren.vector_const(graph, '_warren_pagerank_dangling', dangling, persist := false, version := v); + PERFORM warren.spmv(graph, '_warren_pagerank_msg', '_warren_pagerank_y', transpose := true, + rels := rels, weighted := weighted, persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_pagerank_next', + '_warren_pagerank_damping * (_warren_pagerank_y + _warren_pagerank_dangling * _warren_pagerank_p) + + _warren_pagerank_teleport * _warren_pagerank_p', false, v); + delta := warren.vector_norm_diff(graph, '_warren_pagerank_next', '_warren_pagerank_x', 'l1', v, v); + PERFORM warren.vector_map(graph, '_warren_pagerank_x', '_warren_pagerank_next', false, v); + EXIT WHEN delta < nodes * tol; + END LOOP; + PERFORM warren.vector_map(graph, "out", '_warren_pagerank_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_pagerank_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_pagerank_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:24 +-- requires: +-- extension::warren + +-- warren.aggregate_neighbours (DESIGN.md §6.7, A.5): each node's row of +-- matrix `x` replaced by the mean, sum or max of its neighbours' rows +-- (out-neighbours, or in-neighbours with direction := 'in'; distinct +-- neighbours over `rels`; weights are not read), `hops` times over. A node +-- with no neighbours gets NaN for mean (SQL's AVG of nothing), 0 for sum +-- and -Infinity (the least bigint) for max. mean needs an f32 or f64 +-- matrix; the result keeps x's dtype. Writes matrix `out`. +CREATE FUNCTION warren.aggregate_neighbours(graph text, x text, "out" text, agg text DEFAULT 'mean', + hops int DEFAULT 1, rels text[] DEFAULT NULL, + direction text DEFAULT 'out', version bigint DEFAULT NULL) +RETURNS void LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['x', 'degree']; + semiring text := CASE agg WHEN 'mean' THEN 'plus_second' WHEN 'sum' THEN 'plus_second' + WHEN 'max' THEN 'max_second' END; + dtype text; +BEGIN + IF starts_with("out", '_warren_') OR starts_with(x, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF semiring IS NULL THEN + RAISE EXCEPTION 'agg must be mean, sum or max, not %', agg USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF direction NOT IN ('out', 'in') THEN + RAISE EXCEPTION 'direction must be ''out'' or ''in'', not %', direction USING ERRCODE = 'invalid_parameter_value'; + END IF; + dtype := warren.explain_map(graph, format('%I', x), v) ->> 'dtype'; + IF agg = 'mean' AND dtype NOT IN ('f32', 'f64') THEN + RAISE EXCEPTION 'mean needs an f32 or f64 matrix, not %', dtype USING ERRCODE = 'datatype_mismatch'; + END IF; + BEGIN + PERFORM warren.vector_map(graph, '_warren_aggregate_neighbours_x', format('%I', x), false, v); + IF agg = 'mean' THEN + PERFORM warren.vector_degree(graph, '_warren_aggregate_neighbours_degree', direction, rels, dtype, false, v); + END IF; + FOR hop IN 1..hops LOOP + PERFORM warren.report_progress('aggregate_neighbours', graph, 'hop', hop - 1, hops); + PERFORM warren.spmm(graph, '_warren_aggregate_neighbours_x', '_warren_aggregate_neighbours_x', semiring, + transpose := direction = 'in', rels := rels, weighted := false, persist := false, + version := v); + IF agg = 'mean' THEN + PERFORM warren.vector_map(graph, '_warren_aggregate_neighbours_x', + 'CASE WHEN _warren_aggregate_neighbours_degree > 0 + THEN _warren_aggregate_neighbours_x / _warren_aggregate_neighbours_degree END', false, v); + END IF; + END LOOP; + PERFORM warren.vector_map(graph, "out", '_warren_aggregate_neighbours_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_aggregate_neighbours_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_aggregate_neighbours_' || part, v, true) FROM unnest(parts) part; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:23 +-- requires: +-- extension::warren + +-- warren.sssp_vector (DESIGN.md §6.7): single-source shortest-path +-- distances by Bellman-Ford relaxation (min_plus SpMV along edges, +-- accumulating min) until no distance falls, or after max_iter iterations. +-- By default the bound is the live node count, and distances still falling +-- then mean a negative cycle reachable from the source, an error (22023). +-- Weights as spmv reads them (1 if unweighted); unreachable nodes are +-- Infinity. Writes vector `out` (f64) and returns the iterations run. +CREATE FUNCTION warren.sssp_vector(graph text, "out" text, source anyelement, + rels text[] DEFAULT NULL, weighted boolean DEFAULT NULL, + max_iter int DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['x', 'next', 'fell']; + limit_ int; + falling float8; + i int := 0; +BEGIN + IF starts_with("out", '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + BEGIN + PERFORM warren.vector_from_query(graph, '_warren_sssp_vector_x', + format('SELECT %L::%s, 0', source, pg_typeof(source)), + fill := 'Infinity', persist := false, version := v); + IF warren.vector_reduce(graph, '_warren_sssp_vector_x', 'min', v) <> 0 THEN + RAISE EXCEPTION 'node % is not a node of graph "%"', source, graph USING ERRCODE = 'invalid_parameter_value'; + END IF; + limit_ := coalesce(max_iter, (warren.vector_stats(graph, '_warren_sssp_vector_x', v) ->> 'count')::int); + WHILE i < limit_ LOOP + i := i + 1; + PERFORM warren.report_progress('sssp_vector', graph, 'iterating', i - 1, limit_); + PERFORM warren.vector_map(graph, '_warren_sssp_vector_next', '_warren_sssp_vector_x', false, v); + PERFORM warren.spmv(graph, '_warren_sssp_vector_x', '_warren_sssp_vector_next', 'min_plus', + transpose := true, rels := rels, weighted := weighted, accumulate := 'min', + persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_sssp_vector_fell', + 'CASE WHEN _warren_sssp_vector_next < _warren_sssp_vector_x THEN 1 ELSE 0 END', false, v); + falling := warren.vector_reduce(graph, '_warren_sssp_vector_fell', 'sum', v); + PERFORM warren.vector_map(graph, '_warren_sssp_vector_x', '_warren_sssp_vector_next', false, v); + EXIT WHEN falling = 0; + END LOOP; + IF max_iter IS NULL AND falling > 0 THEN + RAISE EXCEPTION 'a negative cycle is reachable from node % of graph "%"', source, graph + USING ERRCODE = 'invalid_parameter_value'; + END IF; + PERFORM warren.vector_map(graph, "out", '_warren_sssp_vector_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_sssp_vector_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_sssp_vector_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:13 +-- requires: +-- extension::warren + +-- warren.components_vector (DESIGN.md §6.7): weakly connected components by +-- min-label propagation (min_second SpMV both ways, accumulating min) until +-- no label changes. Each component is labelled by its smallest dense id, +-- as warren.components labels it. Writes vector `out` (i64) and returns +-- the iterations run. +CREATE FUNCTION warren.components_vector(graph text, "out" text, rels text[] DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['x', 'next']; + delta float8; + i int := 0; +BEGIN + IF starts_with("out", '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + BEGIN + PERFORM warren.vector_dense_id(graph, '_warren_components_vector_x', 'i64', false, v); + LOOP + i := i + 1; + PERFORM warren.report_progress('components_vector', graph, 'iterating', i - 1, NULL); + PERFORM warren.vector_map(graph, '_warren_components_vector_next', '_warren_components_vector_x', false, v); + PERFORM warren.spmv(graph, '_warren_components_vector_x', '_warren_components_vector_next', 'min_second', + rels := rels, accumulate := 'min', persist := false, version := v); + PERFORM warren.spmv(graph, '_warren_components_vector_x', '_warren_components_vector_next', 'min_second', + transpose := true, rels := rels, accumulate := 'min', persist := false, version := v); + delta := warren.vector_norm_diff(graph, '_warren_components_vector_next', '_warren_components_vector_x', + 'l1', v, v); + PERFORM warren.vector_map(graph, '_warren_components_vector_x', '_warren_components_vector_next', false, v); + EXIT WHEN delta = 0; + END LOOP; + PERFORM warren.vector_map(graph, "out", '_warren_components_vector_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_components_vector_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_components_vector_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/library/mod.rs:8 +-- requires: +-- extension::warren + +-- warren.louvain (DESIGN.md §6.7): Louvain community detection (Blondel +-- et al. 2008) as a library over the primitives. Each level moves nodes +-- between communities (warren.local_moving, from each node alone) on the +-- level's graph, lifts the partition found to `graph` (warren.lift), and +-- contracts `graph` by it (warren.contract) into the next level's graph, +-- whose nodes are the communities: contracting `graph` by the composed +-- partition gives the graph that contracting the level's graph would, so +-- every level's graph is a quotient of `graph`, one lift away, and only +-- one exists at a time. It stops when a level merges no node, when a +-- level raises modularity by no more than `tolerance` (NetworkX's +-- `threshold`), or after `max_levels`. The graph is undirected for this +-- (an edge and its reverse add up; edges weighted if `weighted`, NULL: if +-- the graph has weights). Communities are labelled by the smallest dense +-- id of their members: a level's nodes are keyed by those dense ids. Writes +-- vector `out` (i64, on version `version`, NULL: the current one), with +-- each level's partition as `_` if `levels_prefix` +-- is given, and returns the partition's modularity. The level's graph, a +-- level graph of warren.contract's (`_warren_level_`), and the +-- intermediate vectors are dropped at the end. +CREATE FUNCTION warren.louvain(graph text, "out" text, resolution float8 DEFAULT 1.0, + seed bigint DEFAULT 0, max_levels int DEFAULT 100, + tolerance float8 DEFAULT 1e-7, weighted boolean DEFAULT NULL, + levels_prefix text DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS float8 LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['single', 'labels', 'keys', 'down']; + g text := graph; + gv bigint := v; + next_g text; + next_v bigint; + level int := 0; + nodes bigint; + found bigint; + q float8; + q_next float8; +BEGIN + IF "out" IS NULL OR resolution IS NULL OR seed IS NULL OR max_levels IS NULL OR tolerance IS NULL THEN + RAISE EXCEPTION '% must not be NULL', + CASE WHEN "out" IS NULL THEN 'out' WHEN resolution IS NULL THEN 'resolution' + WHEN seed IS NULL THEN 'seed' WHEN max_levels IS NULL THEN 'max_levels' ELSE 'tolerance' END + USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with("out", '_warren_') OR starts_with(levels_prefix, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF max_levels < 1 THEN + RAISE EXCEPTION 'max_levels must be at least 1, not %', max_levels USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF tolerance = 'NaN'::float8 OR tolerance < 0 THEN + RAISE EXCEPTION 'tolerance must be a number of at least 0, not %', tolerance + USING ERRCODE = 'invalid_parameter_value'; + END IF; + BEGIN + PERFORM warren.vector_dense_id(graph, '_warren_louvain_single', persist := false, version := v); + q := warren.modularity(graph, '_warren_louvain_single', resolution, weighted, v); + LOOP + level := level + 1; + PERFORM warren.report_progress('louvain', graph, format('level %s', level), level - 1, max_levels); + nodes := (SELECT (e ->> 'node_count')::bigint FROM jsonb_array_elements(warren.info(g) -> 'versions') e + WHERE (e ->> 'version')::bigint = gv); + IF level > 1 THEN + -- A level's nodes are the communities found below it, keyed by + -- the least dense id of their members in `graph`. + PERFORM warren.vector_from_query(g, '_warren_louvain_keys', + format('SELECT node_id, node_id FROM warren.id_map(%L, NULL::bigint, %s)', g, gv), 'i64', + persist := false, version := gv); + END IF; + found := warren.local_moving(g, '_warren_louvain_labels', + keys := CASE WHEN level > 1 THEN '_warren_louvain_keys' END, + resolution := resolution, seed := seed, tolerance := 0, + weighted := CASE WHEN level = 1 THEN weighted ELSE true END, + persist := false, version := gv); + EXIT WHEN level > 1 AND found = nodes; + q_next := warren.modularity(g, '_warren_louvain_labels', resolution, + CASE WHEN level = 1 THEN weighted ELSE true END, gv); + -- This level's partition, on `graph`. + IF level = 1 THEN + PERFORM warren.vector_map(graph, '_warren_louvain_down', '_warren_louvain_labels', false, v); + ELSE + PERFORM warren.lift(g, '_warren_louvain_labels', graph, '_warren_louvain_down', + persist := false, version := gv); + END IF; + IF levels_prefix IS NOT NULL THEN + PERFORM warren.vector_map(graph, format('%s_%s', levels_prefix, level), '_warren_louvain_down', true, v); + END IF; + EXIT WHEN found = nodes OR level >= max_levels OR q_next - q <= tolerance; + q := q_next; + -- A level graph of its own, named after its version, a name no + -- graph can have had. + next_v := warren.contract(graph, '_warren_louvain_down', NULL, + weighted := coalesce(weighted, true), version := v); + next_g := '_warren_level_' || next_v; + IF g <> graph THEN + PERFORM warren.drop_vector(g, '_warren_louvain_' || part, gv, true) FROM unnest(parts) part; + PERFORM warren.drop_graph(g); + END IF; + g := next_g; + gv := next_v; + END LOOP; + PERFORM warren.vector_map(graph, "out", '_warren_louvain_down', true, v); + q := warren.modularity(graph, "out", resolution, weighted, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_louvain_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_louvain_' || part, v, true) FROM unnest(parts) part; + IF g <> graph THEN + PERFORM warren.drop_vector(g, '_warren_louvain_' || part, gv, true) FROM unnest(parts) part; + PERFORM warren.drop_graph(g); + END IF; + RETURN q; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:18 +-- requires: +-- extension::warren + +-- warren.label_propagation (DESIGN.md §6.7, A.1): min-label propagation. +-- Labels start as vector `seed` (default: each node's dense id); each +-- iteration every node takes the smallest label among itself and its +-- neighbours (both directions, over `rels`), until no label changes or +-- after max_iter iterations. With `mask`, only the nodes it selects (a +-- non-zero value, GraphBLAS semantics) send or take labels; the others +-- keep theirs. It is deterministic min-propagation, not majority-vote +-- label propagation. Writes vector `out` (f64) and returns the iterations +-- run. +CREATE FUNCTION warren.label_propagation(graph text, "out" text, seed text DEFAULT NULL, + max_iter int DEFAULT 100, mask text DEFAULT NULL, + rels text[] DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['x', 'msg', 'next']; + selected text; + delta float8; + i int := 0; +BEGIN + IF starts_with("out", '_warren_') OR starts_with(seed, '_warren_') OR starts_with(mask, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + BEGIN + IF seed IS NULL THEN + PERFORM warren.vector_dense_id(graph, '_warren_label_propagation_x', 'f64', false, v); + ELSE + PERFORM warren.vector_map(graph, '_warren_label_propagation_x', format('%I::float8', seed), false, v); + END IF; + IF mask IS NOT NULL THEN + selected := CASE warren.explain_map(graph, format('%I', mask), v) ->> 'dtype' + WHEN 'bool' THEN format('%I', mask) ELSE format('%I <> 0', mask) END; + END IF; + WHILE i < max_iter LOOP + i := i + 1; + PERFORM warren.report_progress('label_propagation', graph, 'iterating', i - 1, max_iter); + -- Unselected nodes send NULL (NaN), which min never picks over a label. + PERFORM warren.vector_map(graph, '_warren_label_propagation_msg', + CASE WHEN selected IS NULL THEN '_warren_label_propagation_x' + ELSE format('CASE WHEN %s THEN _warren_label_propagation_x END', selected) END, false, v); + PERFORM warren.vector_map(graph, '_warren_label_propagation_next', '_warren_label_propagation_x', false, v); + PERFORM warren.spmv(graph, '_warren_label_propagation_msg', '_warren_label_propagation_next', 'min_second', + rels := rels, mask := mask, accumulate := 'min', persist := false, version := v); + PERFORM warren.spmv(graph, '_warren_label_propagation_msg', '_warren_label_propagation_next', 'min_second', + transpose := true, rels := rels, mask := mask, accumulate := 'min', persist := false, + version := v); + delta := warren.vector_norm_diff(graph, '_warren_label_propagation_next', '_warren_label_propagation_x', + 'l1', v, v); + PERFORM warren.vector_map(graph, '_warren_label_propagation_x', '_warren_label_propagation_next', false, v); + EXIT WHEN delta = 0; + END LOOP; + PERFORM warren.vector_map(graph, "out", '_warren_label_propagation_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_label_propagation_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_label_propagation_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:34 +-- requires: +-- extension::warren + +-- warren.katz_centrality (DESIGN.md §6.7): Katz centrality by power +-- iteration, as NetworkX computes it. Centrality flows along edges, +-- weighted by edge weight (spmv's rule: weights if the graph has them, +-- unless weighted := false), attenuated by alpha, plus each node's beta: +-- the constant `beta`, or beta × vector `seed` (NetworkX's per-node beta). +-- From 0, each iteration takes alpha · Aᵀx + β; it stops when the L1 change +-- falls below (live nodes) × tol, or after max_iter iterations with a +-- WARNING (NetworkX raises). Alpha at or above 1 / the largest eigenvalue +-- diverges: once a value or the L2 norm is not finite the run is refused +-- (22003), as NetworkX raises, and writes nothing. With normalized, the +-- result is scaled to unit L2 norm. Writes vector `out` (f64) and returns +-- the iterations run. Constants enter expressions as constant vectors; a +-- constant vector is 0 at tombstoned rows, where vector_map evaluates too, +-- so a division by one is guarded. +CREATE FUNCTION warren.katz_centrality(graph text, "out" text, alpha float8 DEFAULT 0.1, + beta float8 DEFAULT 1.0, seed text DEFAULT NULL, + normalized boolean DEFAULT true, tol float8 DEFAULT 1e-6, + max_iter int DEFAULT 1000, rels text[] DEFAULT NULL, + weighted boolean DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['one', 'b', 'alpha', 'x', 'y', 'next', 'scale']; + nodes float8; + norm float8; + delta float8; + i int := 0; +BEGIN + IF "out" IS NULL OR alpha IS NULL OR beta IS NULL OR normalized IS NULL OR tol IS NULL OR max_iter IS NULL THEN + RAISE EXCEPTION '% must not be NULL', + CASE WHEN "out" IS NULL THEN 'out' WHEN alpha IS NULL THEN 'alpha' WHEN beta IS NULL THEN 'beta' + WHEN normalized IS NULL THEN 'normalized' WHEN tol IS NULL THEN 'tol' ELSE 'max_iter' END + USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with("out", '_warren_') OR starts_with(seed, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF tol < 0 THEN + RAISE EXCEPTION 'tol must be at least 0, not %', tol USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF max_iter < 1 THEN + RAISE EXCEPTION 'max_iter must be at least 1, not %', max_iter USING ERRCODE = 'invalid_parameter_value'; + END IF; + BEGIN + PERFORM warren.vector_const(graph, '_warren_katz_centrality_one', 1, persist := false, version := v); + nodes := warren.vector_reduce(graph, '_warren_katz_centrality_one', 'sum', v); + IF nodes = 0 THEN + RAISE EXCEPTION 'graph "%" has no live nodes', graph USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF seed IS NULL THEN + PERFORM warren.vector_const(graph, '_warren_katz_centrality_b', beta, persist := false, version := v); + ELSE + -- vector_reduce names a missing seed vector (42704); vector_map would + -- call it a missing column. + PERFORM warren.vector_reduce(graph, seed, 'nnz', v); + PERFORM warren.vector_map(graph, '_warren_katz_centrality_b', format('%I::float8', seed), false, v); + PERFORM warren.vector_const(graph, '_warren_katz_centrality_scale', beta, persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_katz_centrality_b', + '_warren_katz_centrality_scale * _warren_katz_centrality_b', false, v); + END IF; + PERFORM warren.vector_const(graph, '_warren_katz_centrality_alpha', alpha, persist := false, version := v); + PERFORM warren.vector_const(graph, '_warren_katz_centrality_x', 0, persist := false, version := v); + WHILE i < max_iter LOOP + i := i + 1; + PERFORM warren.report_progress('katz_centrality', graph, 'iterating', i - 1, max_iter); + PERFORM warren.spmv(graph, '_warren_katz_centrality_x', '_warren_katz_centrality_y', + transpose := true, rels := rels, weighted := weighted, persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_katz_centrality_next', + '_warren_katz_centrality_alpha * _warren_katz_centrality_y + _warren_katz_centrality_b', false, v); + delta := warren.vector_norm_diff(graph, '_warren_katz_centrality_next', '_warren_katz_centrality_x', + 'l1', v, v); + IF NOT delta < 'Infinity' THEN + RAISE EXCEPTION 'katz_centrality diverged: its values are not finite after % iterations', i + USING ERRCODE = 'numeric_value_out_of_range', + HINT = 'alpha must be below 1 / the largest eigenvalue of the adjacency matrix.'; + END IF; + PERFORM warren.vector_map(graph, '_warren_katz_centrality_x', '_warren_katz_centrality_next', false, v); + EXIT WHEN delta < nodes * tol; + END LOOP; + IF NOT delta < nodes * tol THEN + RAISE WARNING 'katz_centrality did not converge in % iterations', i; + END IF; + IF normalized THEN + norm := warren.vector_reduce(graph, '_warren_katz_centrality_x', 'l2', v); + IF NOT norm < 'Infinity' THEN + RAISE EXCEPTION 'katz_centrality diverged: its L2 norm overflows after % iterations', i + USING ERRCODE = 'numeric_value_out_of_range', + HINT = 'alpha must be below 1 / the largest eigenvalue of the adjacency matrix.'; + END IF; + PERFORM warren.vector_const(graph, '_warren_katz_centrality_scale', CASE WHEN norm = 0 THEN 1 ELSE norm END, + persist := false, version := v); + PERFORM warren.vector_map(graph, "out", + 'CASE WHEN _warren_katz_centrality_scale <> 0 + THEN _warren_katz_centrality_x / _warren_katz_centrality_scale ELSE 0 END', true, v); + ELSE + PERFORM warren.vector_map(graph, "out", '_warren_katz_centrality_x', true, v); + END IF; + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_katz_centrality_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_katz_centrality_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:41 +-- requires: +-- extension::warren + +-- warren.label_propagation_majority (DESIGN.md §6.7): majority-vote label +-- propagation, synchronous and deterministic. Labels start as vector `seed` +-- (default: each node's dense id); each iteration every node takes the most +-- frequent label among its own and its neighbours' (direction ignored, each +-- neighbour once, over `rels`; a self-loop counts the node's own label once +-- more; weights are not read), the smallest on a tie, until no label +-- changes, or after max_iter iterations with a WARNING (a synchronous +-- update can oscillate). NetworkX's label propagation is asynchronous +-- (random order and ties) or semi-synchronous (by colouring), so neither +-- matches step for step. Tier 2: each iteration is one `gather` (mode, with +-- include_self), which counts the labels in-engine, N rows through the +-- executor. Writes vector `out` (f64) and returns the iterations run. +CREATE FUNCTION warren.label_propagation_majority(graph text, "out" text, seed text DEFAULT NULL, + max_iter int DEFAULT 100, rels text[] DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + info jsonb := warren.info(graph); + v bigint := coalesce(version, (info ->> 'current_version')::bigint); + parts text[] := ARRAY['x', 'next']; + -- Each node's own label and its neighbours', counted in-engine; the most + -- votes win, then the smallest label. + votes text := format( + 'SELECT node_id, value + FROM warren.gather(%1$L, %2$L, NULL::%3$s, NULL::float8, ''mode'', ''both'', %4$L::text[], + version := %5$s, include_self := true)', + graph, '_warren_label_propagation_majority_x', info ->> 'id_type', rels, v); + nodes bigint; + delta float8; + i int := 0; +BEGIN + IF "out" IS NULL OR max_iter IS NULL THEN + RAISE EXCEPTION '% must not be NULL', CASE WHEN "out" IS NULL THEN 'out' ELSE 'max_iter' END + USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with("out", '_warren_') OR starts_with(seed, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF max_iter < 1 THEN + RAISE EXCEPTION 'max_iter must be at least 1, not %', max_iter USING ERRCODE = 'invalid_parameter_value'; + END IF; + BEGIN + IF seed IS NULL THEN + PERFORM warren.vector_dense_id(graph, '_warren_label_propagation_majority_x', 'f64', false, v); + ELSE + -- vector_reduce names a missing seed vector (42704); vector_map would + -- call it a missing column. + PERFORM warren.vector_reduce(graph, seed, 'nnz', v); + PERFORM warren.vector_map(graph, '_warren_label_propagation_majority_x', format('%I::float8', seed), false, v); + END IF; + -- No live nodes, no labels: no iterations (NetworkX finds no communities). + nodes := (warren.vector_stats(graph, '_warren_label_propagation_majority_x', v) ->> 'count')::bigint; + WHILE i < max_iter AND nodes > 0 LOOP + i := i + 1; + PERFORM warren.report_progress('label_propagation_majority', graph, 'iterating', i - 1, max_iter); + PERFORM warren.vector_from_query(graph, '_warren_label_propagation_majority_next', votes, 'f64', + persist := false, version := v); + delta := warren.vector_norm_diff(graph, '_warren_label_propagation_majority_next', + '_warren_label_propagation_majority_x', 'l1', v, v); + PERFORM warren.vector_map(graph, '_warren_label_propagation_majority_x', + '_warren_label_propagation_majority_next', false, v); + EXIT WHEN delta = 0; + END LOOP; + IF delta <> 0 THEN + RAISE WARNING 'label_propagation_majority did not converge in % iterations', i; + END IF; + PERFORM warren.vector_map(graph, "out", '_warren_label_propagation_majority_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_label_propagation_majority_' || part, v, true) + FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_label_propagation_majority_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:40 +-- requires: +-- extension::warren + +-- warren.clustering (DESIGN.md §6.7): the local clustering coefficient, +-- as NetworkX's clustering computes it for an undirected, unweighted +-- graph: 2T / (d(d - 1)) from each node's triangles T (warren.triangles: +-- directions and self-loops ignored) and its distinct neighbours d in +-- either direction over `rels`, and 0 where T is 0. Unlike NetworkX, a +-- self-loop counts its node among its own neighbours in d (vector_degree +-- does; no primitive tells which nodes have one). Edge weights are not +-- read. Writes vector `out` (f64). +CREATE FUNCTION warren.clustering(graph text, "out" text, rels text[] DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS void LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['triangles', 'degree']; +BEGIN + IF "out" IS NULL THEN + RAISE EXCEPTION 'out must not be NULL' USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with("out", '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + BEGIN + PERFORM warren.triangles(graph, '_warren_clustering_triangles', rels, false, v); + PERFORM warren.vector_degree(graph, '_warren_clustering_degree', 'both', rels, 'f64', false, v); + PERFORM warren.vector_map(graph, "out", + 'CASE WHEN _warren_clustering_triangles = 0 THEN 0 + ELSE 2 * _warren_clustering_triangles::float8 + / (_warren_clustering_degree * (_warren_clustering_degree - 1)) END', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_clustering_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_clustering_' || part, v, true) FROM unnest(parts) part; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:8 +-- requires: +-- extension::warren + +-- warren.degree_vector (DESIGN.md §6.7): each node's degree in `direction` +-- over `rels`, as vector `out` (f64); returns its vector id. +CREATE FUNCTION warren.degree_vector(graph text, "out" text, direction text DEFAULT 'out', + rels text[] DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS bigint LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +BEGIN + IF starts_with("out", '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + RETURN warren.vector_degree(graph, "out", direction, rels, 'f64', true, version); +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:29 +-- requires: +-- extension::warren + +-- warren.eigenvector_centrality (DESIGN.md §6.7): eigenvector centrality +-- by power iteration, as NetworkX computes it. Centrality flows along edges +-- (the left eigenvector of A: a node is central when central nodes point at +-- it), weighted by edge weight (spmv's rule: weights if the graph has them, +-- unless weighted := false). From the uniform 1/N, each iteration takes +-- x + Aᵀx (the shift by I keeps a periodic graph from oscillating) and +-- scales it to unit L2 norm; it stops when the L1 change falls below +-- (live nodes) × tol, or after max_iter iterations with a WARNING (NetworkX +-- raises). Values too large for double precision (an L2 norm past about +-- 1e154) are refused (22003) and write nothing. Writes vector `out` (f64, +-- unit L2 norm) and returns the iterations run. Constants enter +-- expressions as constant vectors; a constant vector is 0 at tombstoned +-- rows, where vector_map evaluates too, so a division by one is guarded. +CREATE FUNCTION warren.eigenvector_centrality(graph text, "out" text, tol float8 DEFAULT 1e-6, + max_iter int DEFAULT 100, rels text[] DEFAULT NULL, + weighted boolean DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['one', 'x', 'y', 'next', 'scale']; + nodes float8; + norm float8; + delta float8; + i int := 0; +BEGIN + IF "out" IS NULL OR tol IS NULL OR max_iter IS NULL THEN + RAISE EXCEPTION '% must not be NULL', CASE WHEN "out" IS NULL THEN 'out' WHEN tol IS NULL THEN 'tol' ELSE 'max_iter' END + USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with("out", '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF tol < 0 THEN + RAISE EXCEPTION 'tol must be at least 0, not %', tol USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF max_iter < 1 THEN + RAISE EXCEPTION 'max_iter must be at least 1, not %', max_iter USING ERRCODE = 'invalid_parameter_value'; + END IF; + BEGIN + PERFORM warren.vector_const(graph, '_warren_eigenvector_centrality_one', 1, persist := false, version := v); + nodes := warren.vector_reduce(graph, '_warren_eigenvector_centrality_one', 'sum', v); + IF nodes = 0 THEN + RAISE EXCEPTION 'graph "%" has no live nodes', graph USING ERRCODE = 'invalid_parameter_value'; + END IF; + PERFORM warren.vector_const(graph, '_warren_eigenvector_centrality_x', 1.0 / nodes, persist := false, + version := v); + WHILE i < max_iter LOOP + i := i + 1; + PERFORM warren.report_progress('eigenvector_centrality', graph, 'iterating', i - 1, max_iter); + PERFORM warren.spmv(graph, '_warren_eigenvector_centrality_x', '_warren_eigenvector_centrality_y', + transpose := true, rels := rels, weighted := weighted, persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_eigenvector_centrality_next', + '_warren_eigenvector_centrality_x + _warren_eigenvector_centrality_y', false, v); + norm := warren.vector_reduce(graph, '_warren_eigenvector_centrality_next', 'l2', v); + IF NOT norm < 'Infinity' THEN + RAISE EXCEPTION 'eigenvector_centrality diverged: its values are not finite after % iterations', i + USING ERRCODE = 'numeric_value_out_of_range', + HINT = 'The edge weights are too large: their squares overflow double precision.'; + END IF; + PERFORM warren.vector_const(graph, '_warren_eigenvector_centrality_scale', + CASE WHEN norm = 0 THEN 1 ELSE norm END, persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_eigenvector_centrality_next', + 'CASE WHEN _warren_eigenvector_centrality_scale <> 0 + THEN _warren_eigenvector_centrality_next / _warren_eigenvector_centrality_scale ELSE 0 END', false, v); + delta := warren.vector_norm_diff(graph, '_warren_eigenvector_centrality_next', + '_warren_eigenvector_centrality_x', 'l1', v, v); + PERFORM warren.vector_map(graph, '_warren_eigenvector_centrality_x', '_warren_eigenvector_centrality_next', + false, v); + EXIT WHEN delta < nodes * tol; + END LOOP; + IF NOT delta < nodes * tol THEN + RAISE WARNING 'eigenvector_centrality did not converge in % iterations', i; + END IF; + PERFORM warren.vector_map(graph, "out", '_warren_eigenvector_centrality_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_eigenvector_centrality_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_eigenvector_centrality_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:39 +-- requires: +-- extension::warren + +-- warren.hits (DESIGN.md §6.7): HITS hubs and authorities by power +-- iteration, as NetworkX computes them (its default, method := +-- 'power_iteration'). From hubs uniform at 1/N, each iteration takes the +-- authorities a = Aᵀh (a node pointed at by good hubs) and the hubs h = A·a +-- (a node pointing at good authorities), weighted by edge weight (spmv's +-- rule: weights if the graph has them, unless weighted := false), and +-- scales h to a largest value of 1; it stops when the L1 change of h falls +-- below tol (not scaled by N, as NetworkX), or after max_iter iterations +-- with a WARNING (NetworkX raises). With normalized, each result sums to +-- 1; otherwise each has a largest value of 1. A zero sum or maximum (no +-- edges) leaves the zeros as they are. Values too large for double +-- precision are refused (22003) and write nothing. Writes vectors `hubs` +-- and `authorities` (f64) and returns the iterations run. A constant +-- vector is 0 at tombstoned rows, where vector_map evaluates too, so a +-- division by one is guarded. +CREATE FUNCTION warren.hits(graph text, hubs text, authorities text, tol float8 DEFAULT 1e-8, + max_iter int DEFAULT 100, normalized boolean DEFAULT true, + rels text[] DEFAULT NULL, weighted boolean DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['one', 'x', 'a', 'next', 'scale']; + nodes float8; + scale float8; + hub_scale float8; + delta float8; + i int := 0; +BEGIN + IF hubs IS NULL OR authorities IS NULL OR normalized IS NULL OR tol IS NULL OR max_iter IS NULL THEN + RAISE EXCEPTION '% must not be NULL', + CASE WHEN hubs IS NULL THEN 'hubs' WHEN authorities IS NULL THEN 'authorities' + WHEN normalized IS NULL THEN 'normalized' WHEN tol IS NULL THEN 'tol' ELSE 'max_iter' END + USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with(hubs, '_warren_') OR starts_with(authorities, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF hubs = authorities THEN + RAISE EXCEPTION 'hubs and authorities must be different vectors' USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF tol < 0 THEN + RAISE EXCEPTION 'tol must be at least 0, not %', tol USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF max_iter < 1 THEN + RAISE EXCEPTION 'max_iter must be at least 1, not %', max_iter USING ERRCODE = 'invalid_parameter_value'; + END IF; + BEGIN + PERFORM warren.vector_const(graph, '_warren_hits_one', 1, persist := false, version := v); + nodes := warren.vector_reduce(graph, '_warren_hits_one', 'sum', v); + IF nodes = 0 THEN + RAISE EXCEPTION 'graph "%" has no live nodes', graph USING ERRCODE = 'invalid_parameter_value'; + END IF; + PERFORM warren.vector_const(graph, '_warren_hits_x', 1.0 / nodes, persist := false, version := v); + WHILE i < max_iter LOOP + i := i + 1; + PERFORM warren.report_progress('hits', graph, 'iterating', i - 1, max_iter); + PERFORM warren.spmv(graph, '_warren_hits_x', '_warren_hits_a', transpose := true, rels := rels, + weighted := weighted, persist := false, version := v); + PERFORM warren.spmv(graph, '_warren_hits_a', '_warren_hits_next', rels := rels, weighted := weighted, + persist := false, version := v); + scale := warren.vector_reduce(graph, '_warren_hits_next', 'max', v); + IF NOT abs(scale) < 'Infinity' THEN + RAISE EXCEPTION 'hits diverged: its values are not finite after % iterations', i + USING ERRCODE = 'numeric_value_out_of_range', + HINT = 'The edge weights are too large: their products overflow double precision.'; + END IF; + PERFORM warren.vector_const(graph, '_warren_hits_scale', CASE WHEN scale = 0 THEN 1 ELSE scale END, + persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_hits_next', + 'CASE WHEN _warren_hits_scale <> 0 THEN _warren_hits_next / _warren_hits_scale ELSE 0 END', false, v); + delta := warren.vector_norm_diff(graph, '_warren_hits_next', '_warren_hits_x', 'l1', v, v); + PERFORM warren.vector_map(graph, '_warren_hits_x', '_warren_hits_next', false, v); + EXIT WHEN delta < tol; + END LOOP; + IF NOT delta < tol THEN + RAISE WARNING 'hits did not converge in % iterations', i; + END IF; + IF normalized THEN + hub_scale := warren.vector_reduce(graph, '_warren_hits_x', 'sum', v); + scale := warren.vector_reduce(graph, '_warren_hits_a', 'sum', v); + ELSE + hub_scale := 1; + scale := warren.vector_reduce(graph, '_warren_hits_a', 'max', v); + END IF; + IF NOT abs(hub_scale) < 'Infinity' OR NOT abs(scale) < 'Infinity' THEN + RAISE EXCEPTION 'hits diverged: its values are not finite after % iterations', i + USING ERRCODE = 'numeric_value_out_of_range', + HINT = 'The edge weights are too large: their products overflow double precision.'; + END IF; + PERFORM warren.vector_const(graph, '_warren_hits_scale', CASE WHEN hub_scale = 0 THEN 1 ELSE hub_scale END, + persist := false, version := v); + PERFORM warren.vector_map(graph, hubs, + 'CASE WHEN _warren_hits_scale <> 0 THEN _warren_hits_x / _warren_hits_scale ELSE 0 END', true, v); + PERFORM warren.vector_const(graph, '_warren_hits_scale', CASE WHEN scale = 0 THEN 1 ELSE scale END, + persist := false, version := v); + PERFORM warren.vector_map(graph, authorities, + 'CASE WHEN _warren_hits_scale <> 0 THEN _warren_hits_a / _warren_hits_scale ELSE 0 END', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_hits_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_hits_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/readers.rs:12 +-- pg_warren::catalog::readers::grant + +CREATE FUNCTION warren."grant"(name text, role regrole) RETURNS void +STRICT VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'grant_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/cache/warm.rs:37 +-- pg_warren::cache::warm::evict +CREATE FUNCTION warren."evict"( + "name" TEXT /* & str */ +) RETURNS bigint /* i64 */ +STRICT VOLATILE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'evict_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/components.rs:23 +-- pg_warren::traversal_sql::components::components + +CREATE FUNCTION warren.components(graph text, id_type anyelement, version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, component_id bigint) +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'components_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/bfs.rs:87 +-- pg_warren::traversal_sql::bfs::reachable_set + +CREATE FUNCTION warren.reachable_set(graph text, start anyarray, max_depth int DEFAULT NULL, + rels text[] DEFAULT NULL, direction text DEFAULT 'out', + version bigint DEFAULT NULL) +RETURNS SETOF anyelement +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'reachable_set_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/product.rs:172 +-- pg_warren::compute_sql::product::spmv +CREATE FUNCTION warren."spmv"( + "graph" TEXT, /* Option < & str > */ + "x" TEXT, /* Option < & str > */ + "y" TEXT, /* Option < & str > */ + "semiring" TEXT DEFAULT 'plus_times', /* Option < & str > */ + "transpose" bool DEFAULT false, /* Option < bool > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "mask" TEXT DEFAULT NULL, /* Option < & str > */ + "mask_complement" bool DEFAULT false, /* Option < bool > */ + "replace" bool DEFAULT false, /* Option < bool > */ + "accumulate" TEXT DEFAULT NULL, /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS void +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'spmv_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/drop.rs:16 +-- pg_warren::compute_sql::drop::drop_vector +CREATE FUNCTION warren."drop_vector"( + "graph" TEXT, /* Option < & str > */ + "name" TEXT, /* Option < & str > */ + "version" bigint DEFAULT NULL, /* Option < i64 > */ + "if_exists" bool DEFAULT false /* Option < bool > */ +) RETURNS void +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'drop_vector_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/constructors.rs:93 +-- pg_warren::compute_sql::constructors::vector_degree +CREATE FUNCTION warren."vector_degree"( + "graph" TEXT, /* Option < & str > */ + "name" TEXT, /* Option < & str > */ + "direction" TEXT DEFAULT 'out', /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "dtype" TEXT DEFAULT 'f64', /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_degree_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/structure.rs:137 +-- pg_warren::compute_sql::structure::dag_longest_path +CREATE FUNCTION warren."dag_longest_path"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS double precision /* Option < f64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'dag_longest_path_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/structure.rs:61 +-- pg_warren::compute_sql::structure::strong_components_vector +CREATE FUNCTION warren."strong_components_vector"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'strong_components_vector_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/dropped_roles/orphans.rs:34 +-- pg_warren::catalog::dropped_roles::orphans::orphans +CREATE FUNCTION warren."orphans"() RETURNS TABLE ( + "kind" TEXT, /* String */ + "graph" TEXT, /* String */ + "role" oid /* pg_sys :: Oid */ +) +STRICT VOLATILE PARALLEL SAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'orphans_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/projection.rs:26 +-- pg_warren::community_sql::projection::project +CREATE FUNCTION warren."project"( + "graph" TEXT, /* Option < & str > */ + "x" TEXT, /* Option < & str > */ + "out_graph" TEXT, /* Option < & str > */ + "y" TEXT, /* Option < & str > */ + "agg" TEXT DEFAULT 'sum', /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS void +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'project_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/paths.rs:116 +-- pg_warren::traversal_sql::paths::dijkstra + +CREATE FUNCTION warren.dijkstra(graph text, start anyelement, target anyelement, + rels text[] DEFAULT NULL, direction text DEFAULT 'out', + version bigint DEFAULT NULL) +RETURNS TABLE (path anyarray, cost float8) +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'dijkstra_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/similarity.rs:245 +-- pg_warren::compute_sql::similarity::similar_pairs + +CREATE FUNCTION warren.similar_pairs(graph text, id_type anyelement, k bigint DEFAULT 10, + measure text DEFAULT 'jaccard', nodes text DEFAULT NULL, + rels text[] DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS TABLE (source anyelement, target anyelement, score float8) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'similar_pairs_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/centrality.rs:89 +-- pg_warren::compute_sql::centrality::betweenness_centrality +CREATE FUNCTION warren."betweenness_centrality"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "k" bigint DEFAULT NULL, /* Option < i64 > */ + "seed" bigint DEFAULT 0, /* Option < i64 > */ + "normalized" bool DEFAULT true, /* Option < bool > */ + "endpoints" bool DEFAULT false, /* Option < bool > */ + "directed" bool DEFAULT true, /* Option < bool > */ + "sources" TEXT DEFAULT NULL, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'betweenness_centrality_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/summary.rs:22 +-- pg_warren::compute_sql::summary::vector_topk + +CREATE FUNCTION warren.vector_topk(graph text, name text, k int, id_type anyelement, + descending bool DEFAULT true, version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, value float8) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'vector_topk_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/constructors.rs:132 +-- pg_warren::compute_sql::constructors::vector_dense_id +CREATE FUNCTION warren."vector_dense_id"( + "graph" TEXT, /* Option < & str > */ + "name" TEXT, /* Option < & str > */ + "dtype" TEXT DEFAULT 'i64', /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_dense_id_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/activity/progress.rs:145 +-- pg_warren::activity::progress::progress + +CREATE FUNCTION warren."progress"() RETURNS TABLE ( + "pid" int, + "database" oid, + "role" regrole, + "command" text, + "graph" text, + "version" bigint, + "phase" text, + "done" bigint, + "total" bigint, + "operation" text, + "operation_phase" text, + "operation_done" bigint, + "operation_total" bigint, + "detail" jsonb, + "command_start" timestamptz, + "operation_start" timestamptz +) +VOLATILE PARALLEL SAFE +LANGUAGE c AS 'MODULE_PATHNAME', 'progress_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/structure.rs:26 +-- pg_warren::compute_sql::structure::core_number +CREATE FUNCTION warren."core_number"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'core_number_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/structure.rs:176 +-- pg_warren::compute_sql::structure::minimum_spanning_forest + +CREATE FUNCTION warren.minimum_spanning_forest(graph text, id_type anyelement, rels text[] DEFAULT NULL, + weighted bool DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS TABLE (source anyelement, target anyelement, weight float8) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'minimum_spanning_forest_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/constructors.rs:38 +-- pg_warren::compute_sql::constructors::vector_const +CREATE FUNCTION warren."vector_const"( + "graph" TEXT, /* Option < & str > */ + "name" TEXT, /* Option < & str > */ + "value" double precision, /* Option < f64 > */ + "dtype" TEXT DEFAULT 'f64', /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_const_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/vector/quota/sql.rs:22 +-- pg_warren::vector::quota::sql::set_vector_quota + +CREATE FUNCTION warren."set_vector_quota"(name text, bytes bigint, role regrole DEFAULT NULL) +RETURNS void VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'set_vector_quota_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/promote.rs:14 +-- pg_warren::catalog::promote::promote +CREATE FUNCTION warren."promote"( + "name" TEXT, /* & str */ + "version" bigint /* i64 */ +) RETURNS void +STRICT +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'promote_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/paths.rs:71 +-- pg_warren::traversal_sql::paths::shortest_path + +CREATE FUNCTION warren.shortest_path(graph text, start anyelement, target anyelement, + max_depth int DEFAULT NULL, rels text[] DEFAULT NULL, + direction text DEFAULT 'out', version bigint DEFAULT NULL) +RETURNS anyarray +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'shortest_path_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/build/refresh.rs:22 +-- pg_warren::build::refresh::refresh +CREATE FUNCTION warren."refresh"( + "name" TEXT, /* & str */ + "promote" bool DEFAULT true, /* bool */ + "label" TEXT DEFAULT NULL /* Option < & str > */ +) RETURNS bigint /* i64 */ + +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'refresh_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/structure.rs:101 +-- pg_warren::compute_sql::structure::topological_order +CREATE FUNCTION warren."topological_order"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'topological_order_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/neighbourhood.rs:54 +-- pg_warren::traversal_sql::neighbourhood::degree + +CREATE FUNCTION warren.degree(graph text, node anyelement, direction text DEFAULT 'out', + rels text[] DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS bigint +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'degree_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/spectral_sql/distance/mod.rs:97 +-- pg_warren::spectral_sql::distance::distance +CREATE FUNCTION warren."distance"( + "graph" TEXT, /* Option < & str > */ + "v1" bigint, /* Option < i64 > */ + "v2" bigint, /* Option < i64 > */ + "method" TEXT, /* Option < & str > */ + "vectors_prefix" TEXT DEFAULT NULL, /* Option < & str > */ + "k" INT DEFAULT NULL, /* Option < i32 > */ + "x" TEXT DEFAULT NULL, /* Option < & str > */ + "y" TEXT DEFAULT NULL, /* Option < & str > */ + "compared_rows" TEXT DEFAULT 'shared', /* Option < & str > */ + "matrix" TEXT DEFAULT NULL, /* Option < & str > */ + "which" TEXT DEFAULT 'largest', /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "symmetrize" bool DEFAULT true, /* Option < bool > */ + "tol" double precision DEFAULT 1e-6, /* Option < f64 > */ + "max_iter" INT DEFAULT NULL, /* Option < i32 > */ + "dtype" TEXT DEFAULT 'f32', /* Option < & str > */ + "graph2" TEXT DEFAULT NULL /* Option < & str > */ +) RETURNS jsonb /* JsonB */ +STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'distance_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/owner/reassign.rs:22 +-- pg_warren::catalog::owner::reassign::reassign_owned + +CREATE FUNCTION warren."reassign_owned"("old" regrole, "new" regrole) RETURNS bigint +STRICT VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'reassign_owned_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/drop.rs:29 +-- pg_warren::catalog::drop::drop_graph +CREATE FUNCTION warren."drop_graph"( + "name" TEXT /* & str */ +) RETURNS void +STRICT +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'drop_graph_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/dropped_roles/event_trigger.rs:21 +-- pg_warren::catalog::dropped_roles::event_trigger::on_drop_owned + +CREATE FUNCTION warren."on_drop_owned"() RETURNS event_trigger +VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'on_drop_owned_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/map/mod.rs:91 +-- pg_warren::compute_sql::map::vector_map +CREATE FUNCTION warren."vector_map"( + "graph" TEXT, /* Option < & str > */ + "y" TEXT, /* Option < & str > */ + "expr" TEXT, /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS void +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_map_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/drop.rs:13 +-- pg_warren::catalog::drop::drop_version +CREATE FUNCTION warren."drop_version"( + "name" TEXT, /* & str */ + "version" bigint /* i64 */ +) RETURNS void +STRICT +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'drop_version_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/cache/shared/introspection.rs:136 +-- pg_warren::cache::shared::introspection::shared_cache +CREATE FUNCTION warren."shared_cache"() RETURNS TABLE ( + "kind" TEXT, /* String */ + "graph" TEXT, /* Option < String > */ + "version" bigint, /* Option < i64 > */ + "vector" TEXT, /* Option < String > */ + "vector_id" bigint, /* Option < i64 > */ + "state" TEXT, /* Option < String > */ + "bytes" bigint, /* i64 */ + "pins" INT, /* i32 */ + "mappers" INT, /* i32 */ + "last_use" bigint, /* i64 */ + "cap" bigint /* Option < i64 > */ +) +STRICT VOLATILE PARALLEL SAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'shared_cache_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/activity/backends.rs:120 +-- pg_warren::activity::backends::backends + +CREATE FUNCTION warren."backends"() RETURNS TABLE ( + "kind" text, + "pid" int, + "database" oid, + "role" regrole, + "private_bytes" bigint, + "transient_bytes" bigint, + "mapped_bytes" bigint, + "lookups" bigint, + "private_hits" bigint, + "shared_hits" bigint, + "waits" bigint, + "private_loads" bigint, + "shared_loads" bigint, + "loaded_bytes" bigint, + "load_seconds" float8, + "private_evictions" bigint, + "shared_evictions" bigint, + "refusals" jsonb, + "since" timestamptz +) +VOLATILE PARALLEL SAFE +LANGUAGE c AS 'MODULE_PATHNAME', 'backends_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/reduce.rs:47 +-- pg_warren::compute_sql::reduce::vector_norm_diff +CREATE FUNCTION warren."vector_norm_diff"( + "graph" TEXT, /* Option < & str > */ + "a" TEXT, /* Option < & str > */ + "b" TEXT, /* Option < & str > */ + "norm" TEXT DEFAULT 'l2', /* Option < & str > */ + "a_version" bigint DEFAULT NULL, /* Option < i64 > */ + "b_version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS double precision /* f64 */ +STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_norm_diff_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/paths.rs:27 +-- pg_warren::traversal_sql::paths::reachable + +CREATE FUNCTION warren.reachable(graph text, start anyelement, target anyelement, + max_depth int DEFAULT NULL, rels text[] DEFAULT NULL, + direction text DEFAULT 'out', version bigint DEFAULT NULL) +RETURNS boolean +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'reachable_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/similarity.rs:48 +-- pg_warren::compute_sql::similarity::similarity + +CREATE FUNCTION warren.similarity(graph text, pairs text, id_type anyelement, + measure text DEFAULT 'jaccard', rels text[] DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS TABLE (source anyelement, target anyelement, score float8) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'similarity_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/read.rs:134 +-- pg_warren::compute_sql::read::id_map + +CREATE FUNCTION warren.id_map(graph text, id_type anyelement, version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, dense_id bigint, live boolean) +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'id_map_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/gather/mod.rs:354 +-- pg_warren::compute_sql::gather::gather + +CREATE FUNCTION warren.gather(graph text, messages text, id_type anyelement, + value_type anycompatible, combine text DEFAULT 'collect', + direction text DEFAULT 'in', rels text[] DEFAULT NULL, + mask text DEFAULT NULL, mask_complement boolean DEFAULT false, + version bigint DEFAULT NULL, include_self boolean DEFAULT false) +RETURNS TABLE (node_id anyelement, value anycompatible) +VOLATILE PARALLEL UNSAFE LANGUAGE c AS 'MODULE_PATHNAME', 'gather_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/stats.rs:30 +-- pg_warren::traversal_sql::stats::last_stats +CREATE FUNCTION warren."last_stats"() RETURNS jsonb /* Option < JsonB > */ +STRICT VOLATILE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'last_stats_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/map/mod.rs:156 +-- pg_warren::compute_sql::map::explain_map +CREATE FUNCTION warren."explain_map"( + "graph" TEXT, /* Option < & str > */ + "expr" TEXT, /* Option < & str > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS jsonb /* JsonB */ +STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'explain_map_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/constructors.rs:163 +-- pg_warren::compute_sql::constructors::triangles +CREATE FUNCTION warren."triangles"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'triangles_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/contract.rs:41 +-- pg_warren::community_sql::contract::contract +CREATE FUNCTION warren."contract"( + "graph" TEXT, /* Option < & str > */ + "label_vector" TEXT, /* Option < & str > */ + "out_graph" TEXT, /* Option < & str > */ + "weighted" bool DEFAULT true, /* Option < bool > */ + "drop_self_loops" bool DEFAULT false, /* Option < bool > */ + "promote" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'contract_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/similarity.rs:182 +-- pg_warren::compute_sql::similarity::most_similar + +CREATE FUNCTION warren.most_similar(graph text, node anyelement, k bigint DEFAULT 10, + measure text DEFAULT 'jaccard', rels text[] DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, score float8) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'most_similar_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/bfs.rs:37 +-- pg_warren::traversal_sql::bfs::bfs + +CREATE FUNCTION warren.bfs(graph text, start anyarray, max_depth int, + direction text DEFAULT 'out', rels text[] DEFAULT NULL, + node_filter "any" DEFAULT NULL, max_results bigint DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, depth int, via anyelement) +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'bfs_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/summary.rs:89 +-- pg_warren::compute_sql::summary::vector_stats +CREATE FUNCTION warren."vector_stats"( + "graph" TEXT, /* Option < & str > */ + "name" TEXT, /* Option < & str > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS jsonb /* JsonB */ +STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_stats_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/local_moving.rs:31 +-- pg_warren::community_sql::local_moving::local_moving +CREATE FUNCTION warren."local_moving"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "initial" TEXT DEFAULT NULL, /* Option < & str > */ + "keys" TEXT DEFAULT NULL, /* Option < & str > */ + "resolution" double precision DEFAULT 1.0, /* Option < f64 > */ + "seed" bigint DEFAULT 0, /* Option < i64 > */ + "max_sweeps" INT DEFAULT 100, /* Option < i32 > */ + "tolerance" double precision DEFAULT 1e-7, /* Option < f64 > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'local_moving_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/vector/quota/sql.rs:97 +-- pg_warren::vector::quota::sql::vector_usage + +CREATE FUNCTION warren."vector_usage"(name text) +RETURNS TABLE (role regrole, vectors bigint, bytes bigint, quota bigint) +STRICT STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'vector_usage_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/read.rs:100 +-- pg_warren::compute_sql::read::matrix_rows + +CREATE FUNCTION warren.matrix_rows(graph text, name text, id_type anyelement, + version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, "row" float8[]) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'matrix_rows_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/neighbourhood.rs:83 +-- pg_warren::traversal_sql::neighbourhood::has_edge + +CREATE FUNCTION warren.has_edge(graph text, src anyelement, dst anyelement, + rel text DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS boolean +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'has_edge_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/extension.rs:91 +-- pg_warren::extension::version +CREATE FUNCTION warren."version"() RETURNS TEXT /* & '_ str */ +STRICT STABLE PARALLEL SAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'version_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/local_moving.rs:100 +-- pg_warren::community_sql::local_moving::leiden_refine +CREATE FUNCTION warren."leiden_refine"( + "graph" TEXT, /* Option < & str > */ + "communities" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "keys" TEXT DEFAULT NULL, /* Option < & str > */ + "resolution" double precision DEFAULT 1.0, /* Option < f64 > */ + "seed" bigint DEFAULT 0, /* Option < i64 > */ + "randomness" double precision DEFAULT 0.01, /* Option < f64 > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'leiden_refine_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/modularity.rs:26 +-- pg_warren::community_sql::modularity::modularity +CREATE FUNCTION warren."modularity"( + "graph" TEXT, /* Option < & str > */ + "labels" TEXT, /* Option < & str > */ + "resolution" double precision DEFAULT 1.0, /* Option < f64 > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS double precision /* Option < f64 > */ +STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'modularity_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/cache/warm.rs:18 +-- pg_warren::cache::warm::prewarm +CREATE FUNCTION warren."prewarm"( + "name" TEXT, /* & str */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* i64 */ +VOLATILE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'prewarm_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/activity/command.rs:164 +-- pg_warren::activity::command::report_progress +CREATE FUNCTION warren."report_progress"( + "command" TEXT, /* Option < & str > */ + "graph" TEXT, /* Option < & str > */ + "phase" TEXT, /* Option < & str > */ + "done" bigint DEFAULT NULL, /* Option < i64 > */ + "total" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS void +VOLATILE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'report_progress_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/cache/introspection.rs:282 +-- pg_warren::cache::introspection::cache +CREATE FUNCTION warren."cache"() RETURNS TABLE ( + "kind" TEXT, /* String */ + "graph" TEXT, /* Option < String > */ + "version" bigint, /* Option < i64 > */ + "vector" TEXT, /* Option < String > */ + "vector_id" bigint, /* Option < i64 > */ + "bytes" bigint, /* i64 */ + "state" TEXT, /* Option < String > */ + "last_use" bigint, /* Option < i64 > */ + "cap" bigint /* Option < i64 > */ +) +STRICT VOLATILE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'cache_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/spectral_sql/eigen.rs:93 +-- pg_warren::spectral_sql::eigen::eigen +CREATE FUNCTION warren."eigen"( + "graph" TEXT, /* Option < & str > */ + "k" INT, /* Option < i32 > */ + "matrix" TEXT DEFAULT 'adjacency', /* Option < & str > */ + "which" TEXT DEFAULT 'largest', /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "symmetrize" bool DEFAULT true, /* Option < bool > */ + "tol" double precision DEFAULT 1e-6, /* Option < f64 > */ + "max_iter" INT DEFAULT NULL, /* Option < i32 > */ + "vectors_prefix" TEXT DEFAULT NULL, /* Option < & str > */ + "version" bigint DEFAULT NULL, /* Option < i64 > */ + "dtype" TEXT DEFAULT 'f32', /* Option < & str > */ + "persist" bool DEFAULT true /* Option < bool > */ +) RETURNS TABLE ( + "i" INT, /* i32 */ + "eigenvalue" double precision, /* f64 */ + "residual" double precision /* f64 */ +) +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'eigen_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/create_graph.rs:42 +-- pg_warren::catalog::create_graph::create_graph +CREATE FUNCTION warren."create_graph"( + "name" TEXT, /* & str */ + "node_query" TEXT, /* & str */ + "edge_query" TEXT, /* & str */ + "directed" bool DEFAULT true, /* bool */ + "weighted" bool DEFAULT false, /* bool */ + "has_relations" bool DEFAULT false, /* bool */ + "build_reverse" bool DEFAULT true, /* bool */ + "retain_versions" INT DEFAULT 2, /* i32 */ + "store_merged" bool DEFAULT true, /* bool */ + "node_order" TEXT DEFAULT 'id' /* & str */ +) RETURNS void +STRICT +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'create_graph_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/step.rs:94 +-- pg_warren::traversal_sql::step::s + +CREATE FUNCTION warren.s(rels text[], direction text DEFAULT 'out', filter "any" DEFAULT NULL, + min_hops int DEFAULT 1, max_hops int DEFAULT NULL) +RETURNS warren.step +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 's_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/version_sql/compact.rs:26 +-- pg_warren::version_sql::compact::compact +CREATE FUNCTION warren."compact"( + "name" TEXT, /* Option < & str > */ + "node_order" TEXT DEFAULT NULL /* Option < & str > */ +) RETURNS bigint /* i64 */ + +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'compact_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/product.rs:210 +-- pg_warren::compute_sql::product::spmm +CREATE FUNCTION warren."spmm"( + "graph" TEXT, /* Option < & str > */ + "x" TEXT, /* Option < & str > */ + "y" TEXT, /* Option < & str > */ + "semiring" TEXT DEFAULT 'plus_times', /* Option < & str > */ + "transpose" bool DEFAULT false, /* Option < bool > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "mask" TEXT DEFAULT NULL, /* Option < & str > */ + "mask_complement" bool DEFAULT false, /* Option < bool > */ + "replace" bool DEFAULT false, /* Option < bool > */ + "accumulate" TEXT DEFAULT NULL, /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS void +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'spmm_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/centrality.rs:157 +-- pg_warren::compute_sql::centrality::harmonic_centrality +CREATE FUNCTION warren."harmonic_centrality"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "k" bigint DEFAULT NULL, /* Option < i64 > */ + "seed" bigint DEFAULT 0, /* Option < i64 > */ + "directed" bool DEFAULT true, /* Option < bool > */ + "sources" TEXT DEFAULT NULL, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'harmonic_centrality_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/info.rs:16 +-- pg_warren::catalog::info::info +CREATE FUNCTION warren."info"( + "name" TEXT /* & str */ +) RETURNS jsonb /* JsonB */ +STRICT STABLE PARALLEL SAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'info_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/dropped_roles/orphans.rs:70 +-- pg_warren::catalog::dropped_roles::orphans::drop_orphans +CREATE FUNCTION warren."drop_orphans"() RETURNS bigint /* i64 */ +STRICT +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'drop_orphans_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/reduce.rs:19 +-- pg_warren::compute_sql::reduce::vector_reduce +CREATE FUNCTION warren."vector_reduce"( + "graph" TEXT, /* Option < & str > */ + "x" TEXT, /* Option < & str > */ + "op" TEXT, /* Option < & str > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS double precision /* Option < f64 > */ +STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_reduce_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/walk.rs:38 +-- pg_warren::traversal_sql::walk::walk + +CREATE FUNCTION warren.walk(graph text, start anyarray, steps warren.step[], + mode text DEFAULT 'ends', distinct_ends boolean DEFAULT true, + max_results bigint DEFAULT NULL, reverse boolean DEFAULT false, + version bigint DEFAULT NULL) +RETURNS TABLE (end_id anyelement, path anyarray, rels text[], count bigint) +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'walk_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/neighbourhood.rs:25 +-- pg_warren::traversal_sql::neighbourhood::neighbours + +CREATE FUNCTION warren.neighbours(graph text, node anyelement, direction text DEFAULT 'out', + rels text[] DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS SETOF anyelement +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'neighbours_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/version_sql/diff.rs:23 +-- pg_warren::version_sql::diff::diff +CREATE FUNCTION warren."diff"( + "name" TEXT, /* & str */ + "v1" bigint, /* i64 */ + "v2" bigint /* i64 */ +) RETURNS jsonb /* JsonB */ +STRICT STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'diff_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/from_query.rs:56 +-- pg_warren::compute_sql::from_query::vector_from_query +CREATE FUNCTION warren."vector_from_query"( + "graph" TEXT, /* Option < & str > */ + "name" TEXT, /* Option < & str > */ + "query" TEXT, /* Option < & str > */ + "dtype" TEXT DEFAULT 'f64', /* Option < & str > */ + "fill" double precision DEFAULT 0, /* Option < f64 > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_from_query_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/dropped_roles/drop_owned.rs:16 +-- pg_warren::catalog::dropped_roles::drop_owned::drop_owned + +CREATE FUNCTION warren."drop_owned"(role regrole) RETURNS bigint +STRICT VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'drop_owned_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/owner/mod.rs:27 +-- pg_warren::catalog::owner::set_owner + +CREATE FUNCTION warren."set_owner"(name text, owner regrole) RETURNS void +STRICT VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'set_owner_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/centrality.rs:125 +-- pg_warren::compute_sql::centrality::closeness_centrality +CREATE FUNCTION warren."closeness_centrality"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "k" bigint DEFAULT NULL, /* Option < i64 > */ + "seed" bigint DEFAULT 0, /* Option < i64 > */ + "wf_improved" bool DEFAULT true, /* Option < bool > */ + "directed" bool DEFAULT true, /* Option < bool > */ + "sources" TEXT DEFAULT NULL, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'closeness_centrality_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/readers.rs:25 +-- pg_warren::catalog::readers::revoke + +CREATE FUNCTION warren."revoke"(name text, role regrole) RETURNS void +STRICT VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'revoke_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/read.rs:64 +-- pg_warren::compute_sql::read::vector_values + +CREATE FUNCTION warren.vector_values(graph text, name text, id_type anyelement, + version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, value float8) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'vector_values_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/projection.rs:60 +-- pg_warren::community_sql::projection::lift +CREATE FUNCTION warren."lift"( + "out_graph" TEXT, /* Option < & str > */ + "y" TEXT, /* Option < & str > */ + "graph" TEXT, /* Option < & str > */ + "x" TEXT, /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS void +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'lift_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/bootstrap.rs:8 +-- finalize + +CREATE TABLE warren.graphs ( + graph_id serial PRIMARY KEY, + name text UNIQUE NOT NULL, + owner regrole NOT NULL, + readers regrole[] NOT NULL DEFAULT '{}', + source text NOT NULL DEFAULT 'query' + CHECK (source IN ('query'::pg_catalog.text, 'contraction'::pg_catalog.text)), + node_query text, + edge_query text, + search_path text NOT NULL, + id_type regtype NOT NULL, + directed boolean NOT NULL DEFAULT true, + build_reverse boolean NOT NULL DEFAULT true, + weighted boolean NOT NULL DEFAULT false, + has_relations boolean NOT NULL DEFAULT false, + retain_versions int NOT NULL DEFAULT 2 CHECK (retain_versions >= 1), + current_version bigint, + store_merged boolean NOT NULL DEFAULT true, + node_order text NOT NULL DEFAULT 'id' + CHECK (node_order IN ('id'::pg_catalog.text, 'locality'::pg_catalog.text)), + reader_vector_quota bigint CHECK (reader_vector_quota >= 0) +); + +CREATE TABLE warren.versions ( + version_id bigserial PRIMARY KEY, + graph_id int NOT NULL REFERENCES warren.graphs ON DELETE CASCADE, + seq bigint NOT NULL, + staged boolean NOT NULL DEFAULT true, + built_at timestamptz NOT NULL, + built_lsn pg_lsn NOT NULL, + node_count bigint NOT NULL, + id_space bigint NOT NULL, + edge_count bigint NOT NULL, + dropped_edges bigint NOT NULL, + deduplicated_edges bigint NOT NULL, + target_width smallint NOT NULL CHECK (target_width IN (4, 8)), + compacted_from bigint REFERENCES warren.versions ON DELETE SET NULL, + bytes bigint NOT NULL, + label text, + parent_version bigint REFERENCES warren.versions ON DELETE SET NULL, + label_vector bigint, + UNIQUE (graph_id, seq) +); + +CREATE TABLE warren.segments ( + version_id bigint NOT NULL REFERENCES warren.versions ON DELETE CASCADE, + segment text NOT NULL, + chunk_no int NOT NULL, + data bytea NOT NULL, + PRIMARY KEY (version_id, segment, chunk_no) +); +ALTER TABLE warren.segments ALTER COLUMN data SET STORAGE EXTERNAL; + +CREATE TABLE warren.vectors ( + vector_id bigserial PRIMARY KEY, + graph_id int NOT NULL REFERENCES warren.graphs ON DELETE CASCADE, + version_id bigint NOT NULL REFERENCES warren.versions ON DELETE CASCADE, + name text NOT NULL, + dtype text NOT NULL, + width int NOT NULL DEFAULT 1, + length bigint NOT NULL, + created_at timestamptz NOT NULL DEFAULT now(), + created_by regrole NOT NULL, + UNIQUE (graph_id, version_id, name) +); + +CREATE TABLE warren.vector_chunks ( + vector_id bigint REFERENCES warren.vectors ON DELETE CASCADE, + chunk_no int, + data bytea NOT NULL, + PRIMARY KEY (vector_id, chunk_no) +); +-- IEEE floats rarely compress, so skip pglz's attempt on every write. +ALTER TABLE warren.vector_chunks ALTER COLUMN data SET STORAGE EXTERNAL; +-- A quota sums one role's vectors on one graph (DESIGN.md §5.7). +CREATE INDEX vectors_created_by ON warren.vectors (graph_id, created_by); + +-- Per-role quotas on readers' persisted vectors, overriding +-- graphs.reader_vector_quota (DESIGN.md §5.7, §14 item 46). +CREATE TABLE warren.vector_quotas ( + graph_id int NOT NULL REFERENCES warren.graphs ON DELETE CASCADE, + role regrole NOT NULL, + bytes bigint NOT NULL CHECK (bytes >= 0), + PRIMARY KEY (graph_id, role) +); + +-- graphs.current_version is not a foreign key: with versions.graph_id it +-- would make a cycle, and pg_dump cannot order the data of tables in a +-- foreign-key cycle, so a dump would not restore (DESIGN.md §5.1). These +-- triggers check what that key checked on updates and deletes, and more: +-- the current version is one of the graph's own versions, and cannot be +-- deleted while it is current. They are not equivalent to a key: inserts +-- are not checked (a restore only inserts; cache::open checks what it +-- reads), and they lock only the rows their queries lock. (versions. +-- label_vector has no foreign key for the same reason; a trigger below +-- clears it instead.) +CREATE FUNCTION warren.current_version_is_the_graphs() RETURNS trigger +LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +BEGIN + -- FOR KEY SHARE, as a foreign key's check locks: a concurrent delete of + -- the version waits for this transaction, then fails in its trigger. + IF NEW.current_version IS NOT NULL AND NOT EXISTS ( + SELECT FROM warren.versions v + WHERE v.version_id = NEW.current_version AND v.graph_id = NEW.graph_id + FOR KEY SHARE) THEN + RAISE EXCEPTION USING ERRCODE = 'foreign_key_violation', + MESSAGE = format('version %s is not a version of graph "%s"', NEW.current_version, NEW.name); + END IF; + RETURN NEW; +END $$; +CREATE TRIGGER current_version_is_the_graphs BEFORE UPDATE OF current_version ON warren.graphs + FOR EACH ROW EXECUTE FUNCTION warren.current_version_is_the_graphs(); + +CREATE FUNCTION warren.current_version_is_kept() RETURNS trigger +LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + current_id bigint; +BEGIN + -- FOR SHARE reads the graph's latest row, not the snapshot's: a + -- promotion committed after this transaction's snapshot fails the + -- delete (serialization_failure under REPEATABLE READ, 23503 under READ + -- COMMITTED), and one in progress makes it wait. Deleted with its + -- graph, the version finds no row. + SELECT g.current_version INTO current_id FROM warren.graphs g + WHERE g.graph_id = OLD.graph_id FOR SHARE; + IF current_id = OLD.version_id THEN + RAISE EXCEPTION USING ERRCODE = 'foreign_key_violation', + MESSAGE = format('version %s is the current version of its graph', OLD.version_id); + END IF; + RETURN OLD; +END $$; +CREATE TRIGGER current_version_is_kept BEFORE DELETE ON warren.versions + FOR EACH ROW EXECUTE FUNCTION warren.current_version_is_kept(); +-- versions.label_vector names the vector a contraction was contracted on +-- (DESIGN.md §6.6). It cannot be a foreign key: with vectors.version_id it +-- would make a cycle. When that vector is deleted, the versions contracted +-- on it forget it, so the link never dangles. +CREATE INDEX versions_label_vector ON warren.versions (label_vector) + WHERE label_vector IS NOT NULL; +-- Retention counts the versions contracted from each version, and an +-- ON DELETE SET NULL finds them: both look versions up by parent. +CREATE INDEX versions_parent_version ON warren.versions (parent_version) + WHERE parent_version IS NOT NULL; +CREATE FUNCTION warren.label_vector_is_forgotten() RETURNS trigger +LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +BEGIN + UPDATE warren.versions SET label_vector = NULL + WHERE label_vector OPERATOR(pg_catalog.=) OLD.vector_id; + RETURN OLD; +END $$; +CREATE TRIGGER label_vector_is_forgotten AFTER DELETE ON warren.vectors + FOR EACH ROW EXECUTE FUNCTION warren.label_vector_is_forgotten(); +-- Trigger functions cannot be called directly; nobody needs EXECUTE on them. +REVOKE EXECUTE ON FUNCTION warren.current_version_is_the_graphs(), warren.current_version_is_kept(), + warren.label_vector_is_forgotten() + FROM PUBLIC; + +-- Only warren.* functions touch these, as the catalog owner (§7). +REVOKE ALL ON ALL TABLES IN SCHEMA warren FROM PUBLIC; +REVOKE ALL ON ALL SEQUENCES IN SCHEMA warren FROM PUBLIC; +GRANT USAGE ON SCHEMA warren TO PUBLIC; + +SELECT pg_catalog.pg_extension_config_dump('warren.graphs', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.graphs_graph_id_seq', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.versions', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.versions_version_id_seq', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.segments', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.vectors', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.vectors_vector_id_seq', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.vector_chunks', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.vector_quotas', ''); +/* */ + diff --git a/sql/released.commits b/sql/released.commits index 78eac1f8..6d0aec79 100644 --- a/sql/released.commits +++ b/sql/released.commits @@ -3,3 +3,4 @@ b711d515aeb3d201c11bf867c88825bc39e97d2d 0.2.0 0f1c1de03f4bc357db0117ef1e8d5e4b7d4eff4f 0.3.0 ee856402bde0f9602e05a791db2a1c23439fde4b 0.4.0 5f4176f4e4c8980acf751a342d8622fdb93838c0 0.5.0 +7d129002bbb621056cf72796bc63a2cad8e83241 0.6.0 diff --git a/sql/released.sha256 b/sql/released.sha256 index c27fd88c..c0e1e720 100644 --- a/sql/released.sha256 +++ b/sql/released.sha256 @@ -3,3 +3,4 @@ 5606cfc0b8059fbb1b1046bdb3d671a2fe7c7ad6451993dbbf91115ee28340a3 pg_warren--0.3.0.sql bd2be6fee5169c5566e250c3db4724380ad5f02c40d7c48be68f4610edce03e3 pg_warren--0.4.0.sql 3b4a6793ae6ae2126c5a1031f0f958083dbb3f01c3d6b5def9cc81eb20ca6b9c pg_warren--0.5.0.sql +3cf77ecc93e3e6f973eb3dc66c89125ec906329ec4b2f7d247cb48dc8094604f pg_warren--0.6.0.sql From 31e031434a2c6d9dee3153fb512781068073bf31 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 24 Oct 2026 10:21:31 +0000 Subject: [PATCH 3/4] =?UTF-8?q?DESIGN=20=C2=A713:=20M11's=20real=20graphs?= =?UTF-8?q?=20are=20a=20keepset-sourced=20suite?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The owner's decision: M11's first objective is a benchmark and test suite on a range of real graph datasets, sourced through keepset (https://mearman.github.io/keepset/), whose per-dataset recipes fetch each dataset and convert it to a common shape. The suite lives in bench/corpus, fetches by recipe and pins what it fetched, and covers social graphs, road networks and knowledge graphs, integer and IRI ids, directed and undirected, weighted and unweighted, from small graphs to past 10M nodes. Each dataset runs oracle checks (NetworkX or igraph) where feasible, the §12 matrix and Appendix A's use cases at 10M nodes. The head-to-head with igraph and Neo4j stays, on graphs from the same suite. It replaces the SNAP and Appendix A bullets. Cross-references follow: §12's SNAP sentence points at the suite (its recipes can include those SNAP graphs), §5.7's front-coding candidate is the suite's IRI-keyed knowledge graphs rather than a slice of Wikidata or DBpedia, and README's 0.7.0 Status and "Not there yet" name the suite. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke --- DESIGN.md | 20 ++++++++++++++------ README.md | 14 ++++++++------ 2 files changed, 22 insertions(+), 12 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 380f45e7..b3e74db7 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -444,7 +444,7 @@ Knowledge graphs key on strings (IRIs, slugs), and today they must be mapped to Dense → external is O(1): two adjacent offsets and a slice. External → dense is a binary search of `id_order`. Each probe makes about three random reads (the order entry, the offsets, the bytes), where a fixed type's probe makes one, into its pairs. Both directions are stored, so a load builds nothing: the fixed types build their inverse at load (§5.5), but a text version's region holds exactly its segments. *Memory at 10M nodes,* with a mean id length of *L* bytes: 80 MB of offsets, 40 MB of order and 10M·*L* bytes of ids. That is 360 MB at *L* = 24 (slugs), 600 MB at 48 and 760 MB at 64 (typical IRIs). By comparison, `int8` takes 240 MB (16-byte pairs and an 8-byte inverse) and `uuid` 400 MB. The shared cache holds it once per cluster. On disk it is the same size: segments are `STORAGE EXTERNAL`, uncompressed (§5.1). Considered and not taken: - *Pairs of (offset, length, dense id) sorted by id*, the closest shape to the fixed layout. They cost 16–20 bytes a node on top of the bytes, and an inverse is still needed. - *A 128-bit fingerprint per id, stored as a fixed-width id*, which would reuse every `uuid` path. Dense ids would then follow hash order, not text order, so a first build's ids would carry no locality: the random order §14 item 22 measured as the slow case. Also, a collision, which is rare but can be constructed for any unkeyed hash, would merge two nodes unless every lookup also compared bytes, and comparing bytes needs the arena anyway. -- *Front coding*: sorted blocks of 16 ids, each storing only what differs from its predecessor, as RDF dictionaries such as HDT do. IRIs share long prefixes, so it might cut `id_bytes` 2–4×. The cost is a block decode per lookup and per id returned, and a second permutation (dense id → sorted rank). **Deferred** until it can be measured on real IRIs in M11; LiveJournal, Orkut and the road networks have integer ids, so the candidate is a slice of Wikidata or DBpedia (§13). A front-coded `id_bytes` would be a new segment format, read beside this one. +- *Front coding*: sorted blocks of 16 ids, each storing only what differs from its predecessor, as RDF dictionaries such as HDT do. IRIs share long prefixes, so it might cut `id_bytes` 2–4×. The cost is a block decode per lookup and per id returned, and a second permutation (dense id → sorted rank). **Deferred** until it can be measured on real IRIs in M11; the candidates are the IRI-keyed knowledge graphs of M11's suite (§13, `bench/corpus`). A front-coded `id_bytes` would be a new segment format, read beside this one. **Order and equality (item 42): bytewise.** Text ids compare as their bytes in the database encoding (`memcmp`, then length), which is `COLLATE "C"` order, whatever the column's collation. Under any *deterministic* collation, Postgres's own `text` equality is already bytewise: `texteq` compares bytes, and deterministic ICU and libc collations break ties bytewise. So a node and an edge endpoint that are equal in SQL are equal in pg_warren. The collation's order would only have decided which dense ids a first build hands out. A *nondeterministic* collation (for example ICU `und-u-ks-level2`) makes `'Alice' = 'alice'` in SQL. Bytewise equality cannot follow that, so an edge to `'alice'` would be dropped where a SQL join keeps it. **So `create_graph` and every `refresh` refuse an id column (the node query's, and the edge query's source and target) whose collation is nondeterministic** (`get_collation_isdeterministic` of the cursor's `attcollation`), with `datatype_mismatch` and a hint: write `id COLLATE "C"` for exact ids, or normalise in the query (`lower(id)`). Also: - *Types.* `text` and `varchar` are accepted, since they have the same representation. `varchar` ids are read as `text`, and `id_type` records `text`, so node and edge columns may mix the two. Refused, each with the cast to write: `bpchar`, whose equality ignores trailing spaces; `name`; and `citext`, whose case-insensitive operators set the same trap as a nondeterministic collation. @@ -1640,7 +1640,7 @@ At full scale every §12 target was met for both id types except `reachable_set` The label changes nothing for a serial plan, and with the label `RESTRICTED` no plan ran the call in a worker (a parallel scan's workers only fed the leader, which made the calls). Marked `SAFE`, a per-row read over 1M rows scales with the workers: against the same read with 0 workers, with the shared cache 2.6–2.8× faster with 2 and 3.2–3.6× with 4 on 4 vCPUs; without it 2.7× and 3.2× at 1M nodes, but 1.6× and 2.0× at 10M, where every worker first loads 1.12 GB. The planner picks a parallel plan by itself only for the 1M-row table (the 100k-row table, 4.4 MB, is under `min_parallel_table_scan_size`), where it is 2.5–2.8× faster with the shared cache and 2.1–2.6× without. What a worker pays first is its own read of the version: attaching and validating a shared one (about 20 ms for 112 MB, so at 10k rows 4 workers gain nothing), or loading a private one (about 0.1 s for 112 MB, 1.5–3 s for 1.12 GB, all workers at once): without the preload, a forced parallel plan over 10k rows is 1.8–3× slower than the leader alone, the regression §14 item 33 foresaw, which the owner accepted there; docs/semantics.md documents it with its remedies (the preload, `ALTER FUNCTION … PARALLEL RESTRICTED`, or `SET max_parallel_workers_per_gather = 0`). Neither the planner's own plans nor any plan with the shared cache was slower. Not measured: the private copies' memory (each worker holds one until the query ends, within `warren.max_cache_memory` and `warren.max_compute_memory`, so a parallel plan that reads a version privately holds up to one per worker plus the leader's, §5.6). -Benchmark harness: generators for random, power-law (preferential attachment), grid, and *evolving* graphs (a base graph plus a parameterised perturbation, so `distance` methods can be evaluated for sensitivity and monotonicity); loads into `nodes`/`edges`; builds; runs the matrix against baselines (recursive CTE, NetworkX, igraph). Commit results per version to `bench/results/`. *Implemented (M2):* `bench/run.sh` runs `bench/matrix.json`; a workload names the functions it `requires` and is recorded as skipped until they exist. SNAP datasets (LiveJournal, Orkut, road networks) as secondary benchmarks. +Benchmark harness: generators for random, power-law (preferential attachment), grid, and *evolving* graphs (a base graph plus a parameterised perturbation, so `distance` methods can be evaluated for sensitivity and monotonicity); loads into `nodes`/`edges`; builds; runs the matrix against baselines (recursive CTE, NetworkX, igraph). Commit results per version to `bench/results/`. *Implemented (M2):* `bench/run.sh` runs `bench/matrix.json`; a workload names the functions it `requires` and is recorded as skipped until they exist. Real graphs are M11's suite (§13, `bench/corpus`), sourced through keepset recipes, which can include SNAP's LiveJournal, Orkut and road networks. ## 13. Repository layout and milestones @@ -1741,10 +1741,18 @@ Test it by installing as a non-superuser in CI and running the suite. **Make the - measured (§12): the 100M-edge soak with IRIs against `bigint` ids, at half and full scale, every stage run: the text version is 28% larger and builds in 1.39× the time, `pagerank`, `eigen` and the community detections are within 12% and the library within 1.0–1.4×, and calls that return a row per node about double; the `bigint` build takes 79 s, from 173 s in 0.5.0's soak; Louvain and Leiden complete at 10M nodes. `components`, `minimum_spanning_forest`, `vector_topk` and `most_similar` make their rows as they are returned; `walk` and `neighbours` are left to M11. Front coding of text ids is deferred to M11's real IRIs (item 41). **M11 — 0.7.0: proven on real graphs and real deployments.** -- Benchmarks on real data, not only synthetic graphs: - - SNAP's LiveJournal, Orkut and road networks (§12); - - Appendix A's scale targets at 10M, which have not been run; - - a head-to-head with igraph and Neo4j (with and without GDS) on the same machine and graph, published with its caveats. +- **A benchmark and test suite on a range of real graph datasets, sourced through keepset** ([mearman.github.io/keepset](https://mearman.github.io/keepset/), `github.com/Mearman/keepset`). Keepset is a dataset catalogue whose per-dataset recipes fetch each dataset and convert it to a common shape. + - The suite lives in `bench/corpus`. It fetches by recipe and pins what it fetched. + - It covers a spread of domains, sizes and shapes: + - social graphs, road networks and knowledge graphs; + - integer and IRI (`text`) ids; + - directed and undirected, weighted and unweighted; + - from small graphs up to and past 10M nodes. + - Each dataset runs: + - correctness checks against an oracle (NetworkX or igraph) where one is feasible; + - the §12 performance matrix; + - Appendix A's use cases at 10M nodes, which have not been run. + - A head-to-head with igraph and Neo4j (with and without GDS) on the same machine, on graphs from the same suite, published with its caveats. - Robustness: - a soak of several days behind a connection pooler, watching for memory growth; - many backends refreshing and querying one graph at once; diff --git a/README.md b/README.md index 1c060b12..32a35946 100644 --- a/README.md +++ b/README.md @@ -306,16 +306,18 @@ Every SQL example in the README and `docs/` runs in CI, in in `warren.backends()` and `warren.progress()` need `shared_preload_libraries` ([docs/limits.md](docs/limits.md)). -- **0.7.0** (in development, M11): proof at scale and in deployment: - benchmarks on real graphs, robustness behind connection poolers, on - replicas and through crashes, and packages (DESIGN.md §13). +- **0.7.0** (in development, M11): proof at scale and in deployment: a + benchmark and test suite on a range of real graph datasets sourced + through [keepset](https://mearman.github.io/keepset/) (`bench/corpus`), + robustness behind connection poolers, on replicas and through crashes, + and packages (DESIGN.md §13). See [CHANGELOG.md](CHANGELOG.md). This is pre-1.0 software: the SQL surface may still change between minor versions, with an upgrade script each time. -Not there yet: the rest of the road to 1.0 (DESIGN.md §13) is -benchmarks and robustness on real graphs and deployments, and packages -(M11); and a reviewed, frozen SQL surface (M12, 1.0.0). Incremental +Not there yet: the rest of the road to 1.0 (DESIGN.md §13) is a +benchmark and test suite on real graph datasets, robustness in real +deployments, and packages (M11); and a reviewed, frozen SQL surface (M12, 1.0.0). Incremental maintenance, general sparse matrix algebra (contraction's `PᵀAP` is the one sparse product) and a GPU backend are left to 1.x ([docs/limits.md](docs/limits.md)). From e473f45d9636a322e247feb4f7baec2abf01e5ff Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 24 Oct 2026 10:40:00 +0000 Subject: [PATCH 4/4] CHANGELOG 0.6.0: restart after installing when the library is preloaded Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke --- CHANGELOG.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8401ed2d..28a753f6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,11 @@ builds in 79 s where 0.5.0's soak took 173 s (under Performance). ### Upgrading +- **If `pg_warren` is in `shared_preload_libraries`, restart the server** + after installing the new build. The postmaster keeps the library it loaded + at start, and every backend inherits it, so until a restart the server runs + the old code against the updated catalog. Without the preload, new sessions + load the new library. - **Run `ALTER EXTENSION pg_warren UPDATE;` in every database that has the extension**, after installing the new build and before using the new version. The new library is loaded at once, but the catalog changes only