diff --git a/CHANGELOG.md b/CHANGELOG.md index 83a8e81c..28a753f6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,9 +1,101 @@ # Changelog -## 0.6.0 — in development +## 0.7.0 — in development -### Added +## 0.6.0 — identity, lifecycle and visibility (2026-10-24) +Milestone M10 of DESIGN.md §13. Nodes can be keyed on strings: `text` is +a fourth id type, for IRIs and slugs, and every function takes and +returns `text` ids with no new signature. A graph's ownership can be +transferred (`warren.set_owner`), and the graphs, vectors and reader +entries of a dropped role no longer linger unseen: `reassign_owned` and +`drop_owned` run before `DROP OWNED`, a superuser may opt into a `DROP +OWNED` trigger, and `orphans()` and `drop_orphans()` find what is left. A +graph's owner can bound the vectors other roles persist on it. A +monitoring query can see what each backend's cache holds and counts +(`warren.cache()`, `warren.backends()`), and how far each long call has +got (`warren.progress()`). In the 100M-edge soak with IRIs as node ids, a +text graph's version is 28% larger than with `bigint` ids and builds in +1.39× the time; `pagerank` and `eigen` are within 12%, and the +community detections within 3%. The build itself, for every id type, +now maps edge endpoints on the compute pool, and the `bigint` graph +builds in 79 s where 0.5.0's soak took 173 s (under Performance). + +### Upgrading + +- **If `pg_warren` is in `shared_preload_libraries`, restart the server** + after installing the new build. The postmaster keeps the library it loaded + at start, and every backend inherits it, so until a restart the server runs + the old code against the updated catalog. Without the preload, new sessions + load the new library. +- **Run `ALTER EXTENSION pg_warren UPDATE;` in every database that has the + extension**, after installing the new build and before using the new + version. The new library is loaded at once, but the catalog changes only + with the update, so calls in between can fail. The update: + - creates twelve functions: `set_owner`; `reassign_owned`, + `drop_owned`, `on_drop_owned`, `orphans` and `drop_orphans`; + `set_vector_quota` and `vector_usage`; `cache` and `backends`; + `progress` and `report_progress` (all under New); + - adds the column `warren.graphs.reader_vector_quota` (existing graphs + have no quota), the table `warren.vector_quotas` (dumped with the + extension) and an index on `warren.vectors (graph_id, created_by)`; + - replaces the library's PL/pgSQL functions, `aggregate_neighbours`, + `components_vector`, `eigenvector_centrality`, `hits`, + `katz_centrality`, `label_propagation`, `label_propagation_majority`, + `pagerank`, `sssp_vector`, `louvain` and `leiden`, with the same + signatures, so that they report their progress. Their results are + unchanged. +- The update never creates the `DROP OWNED` event trigger: a superuser + opts into it (under New). +- `text` ids change nothing in the catalog. Existing `integer`, `bigint` + and `uuid` graphs, their segments and their shared versions are + unchanged, and read, refresh and compact as before. A build older than + 0.6.0 cannot read a `text` graph, and downgrading is not supported. +- The shared cache's registry is unchanged, so on PostgreSQL 17 and 18 + versions stay shared across the upgrade without a restart. + +### Behaviour changes + +- `refresh` maps each batch of edge rows' endpoints to dense ids on the + compute pool, on up to `warren.threads` threads, for every id type; the + main thread still reads the rows. The edge list, and so the version, is + the one the row-by-row intake made. Each batch is charged to + `warren.max_build_memory`. +- `grant`, `revoke` and `drop_graph` take the graph's row lock, as + `refresh` and the other writers do, so they wait behind a running + `refresh` of the same graph (under Fixes). +- `components`, `minimum_spanning_forest`, `vector_topk` and + `most_similar` make each row's external ids as the row is returned, + rather than all rows before the first. Their rows and order are + unchanged. Such a call keeps the version it reads, private or mapped + from the shared cache, until its last row is read or it ends: a cursor + on one, read across statements, keeps the version after the statement + that opened it, even once it is evicted (under Performance). + +### New + +- **`text` node ids** (DESIGN.md §5.7, §14 items 41 and 42; + docs/semantics.md, "Text ids"). A graph's node query + may return `text` or `varchar` ids, such as IRIs or slugs; its id type + is then `text`. Every function that takes or returns ids does so in + `text`, with no new signature: a string literal needs its cast + (`warren.neighbours('kg', 'https://example.org/a'::text)`), `ARRAY['…']` + is already `text[]`, and the type witness is `NULL::text`. + - Ids compare bytewise in the database encoding (`COLLATE "C"` order), + which is SQL's own equality under every deterministic collation. An + id column whose collation is nondeterministic is refused by + `create_graph` and `refresh` (42804, hinting `COLLATE "C"` or + `lower()`), as are `bpchar`, `name` and `citext` ids, each with the + cast to write. There is no length limit beyond memory. + - A text version stores each id's bytes once, in dense-id order, with + their offsets and their sorted order: at 10M nodes 120 MB plus the + ids' bytes, against 240 MB for `bigint`. It is shared across backends + and read by parallel workers like any other version. One restored + into a database of another encoding is refused (55000). + - Calls that map more than 65,536 ids (`vector_from_query`, `gather`'s + messages, `similarity`'s pairs) build a hash index of the id map once + per call, charged to `warren.max_compute_memory` (80 MB at 10M + nodes), and keep the binary search if it does not fit. - **`warren.set_owner(name, owner regrole)`** gives a graph a new owner, as `ALTER MATERIALIZED VIEW … OWNER TO` does (DESIGN.md §5.7, §14 item 43). The caller must be a member of the current owner and able to `SET ROLE` @@ -36,7 +128,6 @@ owner (or a member with its privileges or the right to `SET ROLE` to it) as well as by a superuser (`drop_graph`, `drop_orphans`), and adopted by a superuser with `warren.set_owner`, which refused it before ("the owner no longer exists"). - - A `drop_vector`, a replacement or `compact`'s move that checked a vector's creator, and then waited behind a `reassign_owned` that changed it, fails with 40001 instead of deleting the new creator's vector. @@ -110,7 +201,7 @@ extension), an index on `warren.vectors (graph_id, created_by)` and the two functions; existing graphs have no quota. -### Fixed +### Fixes - `refresh`, `compact`, `promote`, `drop_version`, `grant`, `revoke`, `drop_graph` and `contract` into an existing quotient check ownership @@ -119,6 +210,51 @@ transfer ran with an ownership its caller no longer had (harness test `set_owner_race`). +### Performance + +- **The 100M-edge soak with `text` ids** (DESIGN.md §12, "Text node ids"; + `bench/results/soak/a-text-m10.json` and `a-m10.json`; PG 16, 4 vCPUs, + `warren.threads = 4`, private cache). Case a with IRIs of 33–36 bytes as + node ids, against the same graph with `bigint` ids, every stage run: + - the version is 1.44 GB against 1.12 GB, and `refresh` takes 110.2 s + against 79.1 s, peaking 0.25 GB higher; + - calls that return a row per reached node build each id's `text` datum: + `bfs` (1.47M rows) 1.42 s against 0.66 s, `components` 10.6 s against + 5.2 s, `reachable_set` 1.42× as long; + - `similarity` of 100,000 pairs builds its per-call index: 2.15 s + against 0.47 s; + - `pagerank` and `eigen` are within 12%; `eigenvector_centrality`, + `katz_centrality` and `hits` take 1.2–1.4× as long, the rest of the + library and `louvain` and `leiden` within 8%, with the same modularity + to 0.001. + + §12's targets missed there, for both id types: `reachable_set` (13 and + 19M edges/s, against 50M) and `bfs` against the recursive CTE (5.2× and + 10.5× faster, against 20×); with `text` ids, `components` also misses + its 10 s. +- **The build maps edge endpoints in parallel** (under Behaviour changes). + In the same soak the `bigint` graph builds in 79.1 s from its table + (0.5.0's soak: 173.1 s), 99.1 s from the generator query (184.5 s) and + 88.3 s again with 0.1% fewer edges (165.3 s), on the same machine. +- **0.5.0's fixes to level graphs and Leiden's refinement at 10M nodes**, + run at that scale for the first time in the same soak: on the + one-relation graph with little community structure, `leiden` completes + in 804 s (16 levels, modularity 0.186, an 8.10 GB peak), where 0.5.0's + soak was refused (53400), and `louvain` takes 528 s (627.5 s). The two + wrote 13.1 and 4.6 GB of level segments and 15.1 and 6.1 GB of WAL, + held until the run's transaction ends (docs/limits.md). +- **Rows made as they are returned** (DESIGN.md §12): `components`, + `minimum_spanning_forest`, `vector_topk` and `most_similar` no longer + hold every row's ids until the call ends. At 5M nodes and 50M edges, + `components` peaks at 0.84 GB with `text` ids (1.28 GB before) and + 0.73 GB with `bigint` (0.80 GB), and `minimum_spanning_forest` at + 1.10 GB with `text` (1.77 GB), its `bigint` peak unchanged; times are + unchanged. `walk` and `neighbours` still build their rows first, bounded + by their arguments or by one node's degree (M11). +- Progress reporting costs a few atomic stores per statement and per + poll: within noise for one `degree` per transaction and for `pagerank`, + and about 1.5 µs a `report_progress` call. + ## 0.5.0 — the shared cache (2026-10-23) Milestone M9 of DESIGN.md §13, the first on the road to 1.0. A loaded diff --git a/Cargo.lock b/Cargo.lock index 4c4df854..fbf6fad2 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1286,7 +1286,7 @@ dependencies = [ [[package]] name = "pg_warren" -version = "0.6.0" +version = "0.7.0" dependencies = [ "libc", "pgrx", @@ -2231,7 +2231,7 @@ dependencies = [ [[package]] name = "warren-core" -version = "0.6.0" +version = "0.7.0" dependencies = [ "criterion", "nalgebra", diff --git a/Cargo.toml b/Cargo.toml index 396042c5..1c815f1e 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -8,7 +8,7 @@ exclude = ["fuzz"] default-members = ["crates/warren-core"] [workspace.package] -version = "0.6.0" +version = "0.7.0" edition = "2021" license = "Apache-2.0" repository = "https://github.com/ExaDev/pg_warren" diff --git a/DESIGN.md b/DESIGN.md index 1294891c..b3e74db7 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -430,7 +430,7 @@ M9 (0.5.0) moves loaded versions into dynamic shared memory so backends share on ### 5.7 Identity, lifecycle and visibility (M10 design) -*Designed 2026-10-23 for M10 (0.6.0), after 0.5.0's release; built so far: steps 1 (the spike), 2 (`IdMap`), 3 (`TextKeys`), 4 (text graphs end to end), 5 (ownership transfer), 6 (dropped roles), 7 (quotas), 8 (the activity table and the cache counters) and 9 (progress). The decisions are §14 items 41–49: items 44, 45 and 46 were decided by the owner on 2026-10-23, as recommended, and the rest are decided here. Claims about Postgres were read in the PG 15–18 sources (`tcop/utility.c`, `tcop/cmdtaglist.h`, `catalog/pg_shdepend.c`, `commands/event_trigger.c`, `utils/backend_progress.h`, `utils/adt/pgstatfuncs.c`, `utils/wait_event.h`, `utils/misc/guc_tables.c`); where a claim needed code to confirm it, step 1 of the work breakdown did so on PG 15, 16 and 18 (Spike results, below), and the text reflects what it found.* M10 has five items, which share little code. `text` node ids change the id map and everything that maps external ids. Ownership transfer, cleanup after a dropped role, and quotas change the catalog's lifecycle. Observability adds a per-backend table in shared memory beside the registry of §5.6. Each part states what Postgres allows an extension on PG 15–18 and where pg_warren stops. Nothing in M10 changes a kernel, the CSR layout, the segment format of existing id types, or the registry's format, so 0.6.0 needs no restart to keep sharing on PG 17 and 18 (§5.6). +*Designed 2026-10-23 for M10, after 0.5.0's release, and released in 0.6.0 (2026-10-24): every step of the work breakdown below is done. The decisions are §14 items 41–49: items 44, 45 and 46 were decided by the owner on 2026-10-23, as recommended, and the rest are decided here. Claims about Postgres were read in the PG 15–18 sources (`tcop/utility.c`, `tcop/cmdtaglist.h`, `catalog/pg_shdepend.c`, `commands/event_trigger.c`, `utils/backend_progress.h`, `utils/adt/pgstatfuncs.c`, `utils/wait_event.h`, `utils/misc/guc_tables.c`); where a claim needed code to confirm it, step 1 of the work breakdown did so on PG 15, 16 and 18 (Spike results, below), and the text reflects what it found.* M10 has five items, which share little code. `text` node ids change the id map and everything that maps external ids. Ownership transfer, cleanup after a dropped role, and quotas change the catalog's lifecycle. Observability adds a per-backend table in shared memory beside the registry of §5.6. Each part states what Postgres allows an extension on PG 15–18 and where pg_warren stops. Nothing in M10 changes a kernel, the CSR layout, the segment format of existing id types, or the registry's format, so 0.6.0 needs no restart to keep sharing on PG 17 and 18 (§5.6). #### Text node ids (§14 items 41, 42) @@ -444,7 +444,7 @@ Knowledge graphs key on strings (IRIs, slugs), and today they must be mapped to Dense → external is O(1): two adjacent offsets and a slice. External → dense is a binary search of `id_order`. Each probe makes about three random reads (the order entry, the offsets, the bytes), where a fixed type's probe makes one, into its pairs. Both directions are stored, so a load builds nothing: the fixed types build their inverse at load (§5.5), but a text version's region holds exactly its segments. *Memory at 10M nodes,* with a mean id length of *L* bytes: 80 MB of offsets, 40 MB of order and 10M·*L* bytes of ids. That is 360 MB at *L* = 24 (slugs), 600 MB at 48 and 760 MB at 64 (typical IRIs). By comparison, `int8` takes 240 MB (16-byte pairs and an 8-byte inverse) and `uuid` 400 MB. The shared cache holds it once per cluster. On disk it is the same size: segments are `STORAGE EXTERNAL`, uncompressed (§5.1). Considered and not taken: - *Pairs of (offset, length, dense id) sorted by id*, the closest shape to the fixed layout. They cost 16–20 bytes a node on top of the bytes, and an inverse is still needed. - *A 128-bit fingerprint per id, stored as a fixed-width id*, which would reuse every `uuid` path. Dense ids would then follow hash order, not text order, so a first build's ids would carry no locality: the random order §14 item 22 measured as the slow case. Also, a collision, which is rare but can be constructed for any unkeyed hash, would merge two nodes unless every lookup also compared bytes, and comparing bytes needs the arena anyway. -- *Front coding*: sorted blocks of 16 ids, each storing only what differs from its predecessor, as RDF dictionaries such as HDT do. IRIs share long prefixes, so it might cut `id_bytes` 2–4×. The cost is a block decode per lookup and per id returned, and a second permutation (dense id → sorted rank). **Deferred** until it can be measured on real IRIs in M11; LiveJournal, Orkut and the road networks have integer ids, so the candidate is a slice of Wikidata or DBpedia (§13). A front-coded `id_bytes` would be a new segment format, read beside this one. +- *Front coding*: sorted blocks of 16 ids, each storing only what differs from its predecessor, as RDF dictionaries such as HDT do. IRIs share long prefixes, so it might cut `id_bytes` 2–4×. The cost is a block decode per lookup and per id returned, and a second permutation (dense id → sorted rank). **Deferred** until it can be measured on real IRIs in M11; the candidates are the IRI-keyed knowledge graphs of M11's suite (§13, `bench/corpus`). A front-coded `id_bytes` would be a new segment format, read beside this one. **Order and equality (item 42): bytewise.** Text ids compare as their bytes in the database encoding (`memcmp`, then length), which is `COLLATE "C"` order, whatever the column's collation. Under any *deterministic* collation, Postgres's own `text` equality is already bytewise: `texteq` compares bytes, and deterministic ICU and libc collations break ties bytewise. So a node and an edge endpoint that are equal in SQL are equal in pg_warren. The collation's order would only have decided which dense ids a first build hands out. A *nondeterministic* collation (for example ICU `und-u-ks-level2`) makes `'Alice' = 'alice'` in SQL. Bytewise equality cannot follow that, so an edge to `'alice'` would be dropped where a SQL join keeps it. **So `create_graph` and every `refresh` refuse an id column (the node query's, and the edge query's source and target) whose collation is nondeterministic** (`get_collation_isdeterministic` of the cursor's `attcollation`), with `datatype_mismatch` and a hint: write `id COLLATE "C"` for exact ids, or normalise in the query (`lower(id)`). Also: - *Types.* `text` and `varchar` are accepted, since they have the same representation. `varchar` ids are read as `text`, and `id_type` records `text`, so node and edge columns may mix the two. Refused, each with the cast to write: `bpchar`, whose equality ignores trailing spaces; `name`; and `citext`, whose case-insensitive operators set the same trap as a nondeterministic collation. @@ -703,13 +703,13 @@ Each step is a pull request that leaves `main` green. - The text id map at 10M nodes: build, sort, merge and hash-index time, lookup latency, and memory, at *L* = 24, 48 and 64. 2. *`warren-core`: `IdMap` with `FixedKeys`,* with no behaviour change: every existing test and proptest passes unchanged, and Miri. *Done:* tests changed only where the types did (`IdMap>`, and `external_id` returning a reference). 3. *`warren-core`: `TextKeys`:* the rebuild with its arena and hash index, the three segment kinds, validation, renumbering, compaction, `diff` and alignment, with the tests above. **Done** (see "Code (item 41)", *Built (step 3)*). -4. *`pg_warren`: text graphs end to end:* `IdType::Text`; the collation, type and encoding checks; the build's intake; the region's arrays and the loader; `external_ids`; every SQL function. Also the `text_ids` tests, and the soak at 10M nodes with synthetic IRIs (`bench/soak.py --ids text`), recorded in §12. +4. *`pg_warren`: text graphs end to end:* `IdType::Text`; the collation, type and encoding checks; the build's intake; the region's arrays and the loader; `external_ids`; every SQL function. Also the `text_ids` tests, and the soak at 10M nodes with synthetic IRIs (`bench/soak.py --ids text`), recorded in §12. **Done** (see "Text node ids", *Built (step 4)*; the soak at half and full scale in §12, and the rows made as they are returned). 5. *Ownership transfer:* `set_owner`, ownership re-checked after the lock, and `set_owner_race`. **Done** (see "Ownership transfer", *Implemented*). 6. *Dropped roles:* `drop_owned`, `reassign_owned`, `orphans`, `drop_orphans` and `on_drop_owned`, as items 44 and 45 decide; `docs/semantics.md`'s procedure; the trusted-install and dump tests. **Done** (see "When an owner role is dropped", *Implemented*). 7. *Quotas:* the schema (`graphs.reader_vector_quota`, `warren.vector_quotas`, the index), `set_vector_quota`, `vector_usage` and enforcement, as item 46 decides; the race tests. **Done** (see "Quotas on readers' vectors", *Implemented*). 8. *The activity table and the cache counters:* `warren.cache()`, `warren.backends()` and the wait event. **Done** (see "Observability", *Implemented (step 8)*). 9. *Progress:* the `warren-core` port, reporting from every call in the table above, `warren.progress()` and `warren.report_progress()`, and the library calling it; the cost measured. **Done** (see "Observability", *Implemented (step 9)*). -10. *Release 0.6.0:* the install script, `sql/pg_warren--0.5.0--0.6.0.sql`, `sql/released.sha256`, and `test/upgrade.py` from every release. +10. *Release 0.6.0:* the install script, `sql/pg_warren--0.5.0--0.6.0.sql`, `sql/released.sha256`, and `test/upgrade.py` from every release. *Done (2026-10-24):* `sql/pg_warren--0.6.0.sql` is frozen, `sql/pg_warren--0.5.0--0.6.0.sql` creates the twelve functions, the quota's column, table and index, and replaces the library's PL/pgSQL so that it reports progress, and `test/upgrade.py` checks every release's path to the version in development. ## 6. Compute layer @@ -1640,7 +1640,7 @@ At full scale every §12 target was met for both id types except `reachable_set` The label changes nothing for a serial plan, and with the label `RESTRICTED` no plan ran the call in a worker (a parallel scan's workers only fed the leader, which made the calls). Marked `SAFE`, a per-row read over 1M rows scales with the workers: against the same read with 0 workers, with the shared cache 2.6–2.8× faster with 2 and 3.2–3.6× with 4 on 4 vCPUs; without it 2.7× and 3.2× at 1M nodes, but 1.6× and 2.0× at 10M, where every worker first loads 1.12 GB. The planner picks a parallel plan by itself only for the 1M-row table (the 100k-row table, 4.4 MB, is under `min_parallel_table_scan_size`), where it is 2.5–2.8× faster with the shared cache and 2.1–2.6× without. What a worker pays first is its own read of the version: attaching and validating a shared one (about 20 ms for 112 MB, so at 10k rows 4 workers gain nothing), or loading a private one (about 0.1 s for 112 MB, 1.5–3 s for 1.12 GB, all workers at once): without the preload, a forced parallel plan over 10k rows is 1.8–3× slower than the leader alone, the regression §14 item 33 foresaw, which the owner accepted there; docs/semantics.md documents it with its remedies (the preload, `ALTER FUNCTION … PARALLEL RESTRICTED`, or `SET max_parallel_workers_per_gather = 0`). Neither the planner's own plans nor any plan with the shared cache was slower. Not measured: the private copies' memory (each worker holds one until the query ends, within `warren.max_cache_memory` and `warren.max_compute_memory`, so a parallel plan that reads a version privately holds up to one per worker plus the leader's, §5.6). -Benchmark harness: generators for random, power-law (preferential attachment), grid, and *evolving* graphs (a base graph plus a parameterised perturbation, so `distance` methods can be evaluated for sensitivity and monotonicity); loads into `nodes`/`edges`; builds; runs the matrix against baselines (recursive CTE, NetworkX, igraph). Commit results per version to `bench/results/`. *Implemented (M2):* `bench/run.sh` runs `bench/matrix.json`; a workload names the functions it `requires` and is recorded as skipped until they exist. SNAP datasets (LiveJournal, Orkut, road networks) as secondary benchmarks. +Benchmark harness: generators for random, power-law (preferential attachment), grid, and *evolving* graphs (a base graph plus a parameterised perturbation, so `distance` methods can be evaluated for sensitivity and monotonicity); loads into `nodes`/`edges`; builds; runs the matrix against baselines (recursive CTE, NetworkX, igraph). Commit results per version to `bench/results/`. *Implemented (M2):* `bench/run.sh` runs `bench/matrix.json`; a workload names the functions it `requires` and is recorded as skipped until they exist. Real graphs are M11's suite (§13, `bench/corpus`), sourced through keepset recipes, which can include SNAP's LiveJournal, Orkut and road networks. ## 13. Repository layout and milestones @@ -1732,11 +1732,27 @@ Test it by installing as a non-superuser in CI and running the suite. **Make the - Per-role quotas on readers' vectors. - Observability: what each backend, or the shared cache, holds and its bytes; cache hits, misses and evictions; and the progress of a long `refresh`, `eigen` or `louvain`, through a view or function a monitoring query can poll. +**Done (0.6.0, 2026-10-24):** +- `text` node ids as §5.7 designs them (§14 items 41 and 42): the id map's three arrays, bytewise comparison with nondeterministic collations refused, the encoding recorded and checked, and every function taking and returning `text` through the existing `anyelement` dispatch; the build maps edge endpoints on the compute pool for every id type, and calls that map more than 65,536 ids index the id map once per call; +- ownership transfer, `warren.set_owner` (item 43), with ownership re-checked once a writer holds the graph's row lock; +- cleanup after a dropped role (items 44 and 45): `reassign_owned`, `drop_owned`, `orphans`, `drop_orphans`, and `on_drop_owned` for a superuser's opt-in `DROP OWNED` trigger, which the extension never creates; +- quotas on readers' persisted vectors (items 46 and 47): `set_vector_quota`, `vector_usage`, enforced under a lock per (graph, role); +- observability (items 48 and 49): the activity table beside the registry, `warren.cache()`, `warren.backends()` and the `WarrenSharedLoad` wait event, `warren.progress()` and `warren.report_progress()`, reported by every long call and the library; +- measured (§12): the 100M-edge soak with IRIs against `bigint` ids, at half and full scale, every stage run: the text version is 28% larger and builds in 1.39× the time, `pagerank`, `eigen` and the community detections are within 12% and the library within 1.0–1.4×, and calls that return a row per node about double; the `bigint` build takes 79 s, from 173 s in 0.5.0's soak; Louvain and Leiden complete at 10M nodes. `components`, `minimum_spanning_forest`, `vector_topk` and `most_similar` make their rows as they are returned; `walk` and `neighbours` are left to M11. Front coding of text ids is deferred to M11's real IRIs (item 41). + **M11 — 0.7.0: proven on real graphs and real deployments.** -- Benchmarks on real data, not only synthetic graphs: - - SNAP's LiveJournal, Orkut and road networks (§12); - - Appendix A's scale targets at 10M, which have not been run; - - a head-to-head with igraph and Neo4j (with and without GDS) on the same machine and graph, published with its caveats. +- **A benchmark and test suite on a range of real graph datasets, sourced through keepset** ([mearman.github.io/keepset](https://mearman.github.io/keepset/), `github.com/Mearman/keepset`). Keepset is a dataset catalogue whose per-dataset recipes fetch each dataset and convert it to a common shape. + - The suite lives in `bench/corpus`. It fetches by recipe and pins what it fetched. + - It covers a spread of domains, sizes and shapes: + - social graphs, road networks and knowledge graphs; + - integer and IRI (`text`) ids; + - directed and undirected, weighted and unweighted; + - from small graphs up to and past 10M nodes. + - Each dataset runs: + - correctness checks against an oracle (NetworkX or igraph) where one is feasible; + - the §12 performance matrix; + - Appendix A's use cases at 10M nodes, which have not been run. + - A head-to-head with igraph and Neo4j (with and without GDS) on the same machine, on graphs from the same suite, published with its caveats. - Robustness: - a soak of several days behind a connection pooler, watching for memory growth; - many backends refreshing and querying one graph at once; diff --git a/README.md b/README.md index 543c44e1..32a35946 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,7 @@ the server headers (`postgresql-server-dev-NN`), `clang`/`libclang-dev`, and cargo-pgrx at exactly the version the extension uses: ```sh -git checkout v0.5.0 # the latest release; main is the version in development +git checkout v0.6.0 # the latest release; main is the version in development cargo install cargo-pgrx --version 0.19.3 --locked cargo pgrx init --pg16 "$(command -v pg_config)" cd crates/pg_warren @@ -282,22 +282,42 @@ Every SQL example in the README and `docs/` runs in CI, in slower; and Louvain and Leiden at 10M nodes still write several GB of level graphs ([docs/limits.md](docs/limits.md)). -- **0.6.0** (in development, M10): identity, lifecycle and visibility: - `text` node ids, ownership transfer, cleanup when an owner role is - dropped, quotas on readers' vectors, and observability (DESIGN.md §13). +- **0.6.0**: identity, lifecycle and visibility, milestone M10. Nodes + can be keyed on strings: `text` is a fourth id type, for IRIs and slugs, + compared bytewise (docs/semantics.md, "Text ids"). A graph's ownership + can be transferred (`warren.set_owner`); a dropped owner role's graphs, + vectors and reader entries are handled by `warren.reassign_owned` and + `warren.drop_owned`, an optional superuser's `DROP OWNED` trigger, and + `warren.orphans()` and `warren.drop_orphans()`; a graph's owner can + bound the vectors other roles persist on it + (`warren.set_vector_quota`). `warren.cache()`, `warren.backends()` and + `warren.progress()` show what each backend's cache holds and counts and + how far each long call has got (DESIGN.md §5.7). Upgrading needs `ALTER + EXTENSION pg_warren UPDATE` (see the changelog). + + In the 100M-edge soak with IRIs as node ids, the version is 28% larger + than with `bigint` ids and the build takes 1.39× as long; `pagerank`, + `eigen` and the community detections are within 12%, the library within + 1.0–1.4×, and calls that return a row per node (`bfs`, `components`) + take about twice as long, since they build each id's `text`. The build maps edge endpoints on the compute pool for every id + type: the `bigint` graph builds in 79 s, from 173 s in 0.5.0's soak. The + caveats: `walk` and `neighbours` still build their rows before + returning the first; and on PostgreSQL 15 and 16, other backends' rows + in `warren.backends()` and `warren.progress()` need + `shared_preload_libraries` ([docs/limits.md](docs/limits.md)). + +- **0.7.0** (in development, M11): proof at scale and in deployment: a + benchmark and test suite on a range of real graph datasets sourced + through [keepset](https://mearman.github.io/keepset/) (`bench/corpus`), + robustness behind connection poolers, on replicas and through crashes, + and packages (DESIGN.md §13). See [CHANGELOG.md](CHANGELOG.md). This is pre-1.0 software: the SQL surface may still change between minor versions, with an upgrade script each time. -Not there yet: the rest of the road to 1.0 (DESIGN.md §13) is text ids, -and observability (M10; ownership transfer, `warren.set_owner`, cleanup -after a dropped role, `warren.reassign_owned`, `warren.drop_owned`, the -optional `DROP OWNED` trigger and `warren.drop_orphans`, quotas on readers' -vectors, `warren.set_vector_quota`, the cache's views, `warren.cache()` and -`warren.backends()`, and progress, `warren.progress()` and -`warren.report_progress()`, are built for 0.6.0); -benchmarks and robustness on real graphs and deployments, and packages -(M11); and a reviewed, frozen SQL surface (M12, 1.0.0). Incremental +Not there yet: the rest of the road to 1.0 (DESIGN.md §13) is a +benchmark and test suite on real graph datasets, robustness in real +deployments, and packages (M11); and a reviewed, frozen SQL surface (M12, 1.0.0). Incremental maintenance, general sparse matrix algebra (contraction's `PᵀAP` is the one sparse product) and a GPU backend are left to 1.x ([docs/limits.md](docs/limits.md)). diff --git a/docs/limits.md b/docs/limits.md index c9bd326c..5b1612f9 100644 --- a/docs/limits.md +++ b/docs/limits.md @@ -11,7 +11,7 @@ is set in the server's configuration and takes effect on reload. | Setting | Default | Range | What it bounds | When exceeded | |---|---|---|---|---| -| `warren.threads` | half the cores, at least 1 | 1–1024 | threads a compute kernel runs on (`spmv`, `spmm`, `vector_map`, reductions, `gather`, `eigen`, `distance`, `triangles`, the sort in `core_number`, the centralities' sources and the similarities) | — | +| `warren.threads` | half the cores, at least 1 | 1–1024 | threads a compute kernel runs on (`spmv`, `spmm`, `vector_map`, reductions, `gather`, `eigen`, `distance`, `triangles`, the sort in `core_number`, the centralities' sources and the similarities), and the mapping of a `refresh`'s edge endpoints | — | | `warren.max_compute_memory` | `4GB` | `64kB` up | memory one call holds: every vector it reads (once, cached or not), every vector and work buffer it allocates, and what its statement holds in the cache past `warren.max_cache_memory` | `configuration_limit_exceeded` (53400), before allocating or loading | | `warren.max_cache_memory` | `4GB` | `64kB` up | what this backend's cache holds *between* statements: cached versions, cached persisted vectors, and every role's transient vectors | versions and persisted vectors no running statement has read are evicted, least recently used first; a transient write that would go over is refused with 53400 | | `warren.shared_cache_memory` | `4GB` | `0` up; `0` turns it off | the cluster's shared versions and persisted vectors, of every database together; set in `postgresql.conf` (or `ALTER SYSTEM`) and reloaded, not per session ([the shared cache](#the-shared-cache)) | unpinned entries no backend maps are evicted, least recently used first; one that still does not fit is loaded into the reading backend's own cache, with a `NOTICE` | diff --git a/docs/semantics.md b/docs/semantics.md index df28de5d..c887589c 100644 --- a/docs/semantics.md +++ b/docs/semantics.md @@ -17,7 +17,8 @@ SELECT warren.create_graph(name, node_query, edge_query, ``` - `node_query` returns one column, the node's **external id**: `integer`, - `bigint` or `uuid`. Its type is the graph's **id type**. + `bigint`, `uuid` or `text` (`varchar` is read as `text`). Its type is + the graph's **id type** ([text ids](#text-ids)). - `edge_query` returns `src, dst` (of the id type), then a `text` **relation** if `has_relations`, then a `double precision` weight if `weighted`. @@ -469,6 +470,10 @@ Functions are polymorphic over the id type and check it at run time. a typed NULL: `warren.components('social', NULL::bigint)`. `gather` takes a second witness for its result type. - `uuid` graphs work the same way (`ARRAY[$1::uuid]`, `NULL::uuid`). +- `text` graphs too (`'https://example.org/a'::text`, `NULL::text`): a + lone string literal needs its `::text`, since Postgres cannot resolve an + untyped literal to `anyelement`, but `ARRAY['…']` is already `text[]`. + A `varchar` argument is widened where the result holds no ids. ```sql SELECT warren.neighbours('social', 3::bigint); @@ -477,6 +482,30 @@ SELECT * FROM warren.components('social', NULL::bigint); SELECT * FROM warren.vector_values('social', 'pr', NULL::bigint); -- after pagerank('social', 'pr') ``` +### Text ids + +- Ids compare **bytewise** in the database encoding, which is `COLLATE "C"` + order, whatever the column's collation. Under a deterministic collation + (every default one) that is SQL's own equality, so an edge joins the + node SQL would join it to. A nondeterministic collation (for example + ICU's case-insensitive ones) makes `'Alice' = 'alice'`, which bytewise + ids cannot follow, so `create_graph` and `refresh` refuse such an id + column (`datatype_mismatch`, 42804): write `id COLLATE "C"` for exact + ids, or normalise in the query (`lower(id)`). `bpchar`, `name` and + `citext` ids are refused too, each with the cast to write. +- The first build numbers dense ids in byte order (`node_order := 'id'`). +- Ids have no length limit beyond memory; the empty string is an id, and a + NULL id is skipped, as for every id type. +- A version records the database encoding, and one restored into a + database of another encoding is refused (55000): create the graph again + there. +- A text graph's id map holds each id's bytes once plus 12 bytes a node + (at 10M nodes, 120 MB plus the ids; `bigint` takes 240 MB). Calls that + map more than 65,536 ids at once (`vector_from_query`, `gather`'s + messages, `similarity`'s pairs) build a hash index of 8 bytes a node for + the call, charged to `warren.max_compute_memory`, and search without it + if it does not fit. + ## Unknown and tombstoned ids An external id the version does not have, or has only as a tombstone, is diff --git a/sql/pg_warren--0.6.0--0.7.0.sql b/sql/pg_warren--0.6.0--0.7.0.sql new file mode 100644 index 00000000..33b2c33e --- /dev/null +++ b/sql/pg_warren--0.6.0--0.7.0.sql @@ -0,0 +1,8 @@ +-- pg_warren 0.6.0 -> 0.7.0 (DESIGN.md §13; CONTRIBUTING.md "Releases and +-- upgrades"). Hand-maintained: every schema change made after 0.6.0 is +-- mirrored here in the same commit, and test/upgrade.py checks that +-- ALTER EXTENSION pg_warren UPDATE from 0.6.0 yields the catalog of a fresh +-- 0.7.0 install. In a trusted install it runs as the bootstrap superuser +-- (DESIGN.md §7): create only in schema warren, schema-qualify, and use only +-- exact-match pg_catalog operators and functions in what runs here. +\echo Use "ALTER EXTENSION pg_warren UPDATE TO '0.7.0'" to load this file. \quit diff --git a/sql/pg_warren--0.7.0.sql b/sql/pg_warren--0.7.0.sql new file mode 100644 index 00000000..d162c0f5 --- /dev/null +++ b/sql/pg_warren--0.7.0.sql @@ -0,0 +1,2388 @@ +/* */ +/* +This file is auto generated by pgrx. + +The ordering of items is not stable, it is driven by a dependency graph. +*/ +/* */ + +/* */ +-- crates/pg_warren/src/extension.rs:18 +CREATE SCHEMA IF NOT EXISTS warren; /* pg_warren::extension::warren */ +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/library/mod.rs:9 +-- requires: +-- extension::warren + +-- warren.leiden (DESIGN.md §6.7): Leiden community detection (Traag, +-- Waltman and van Eck 2019) as a library over the primitives. Each level +-- moves nodes between communities (warren.local_moving, from the partition +-- the level below found) on the level's graph, refines each community +-- into well-connected parts (warren.leiden_refine), lifts both partitions +-- to `graph` (warren.lift), and contracts `graph` by the refined one +-- (warren.contract) into the next level's graph; the next level starts +-- from the unrefined partition, projected onto it (warren.project, min). +-- As warren.louvain's, every level's graph is a quotient of `graph`, and +-- only one exists at a time. Every community found is connected. As +-- Traag et al.'s, it goes on while there is something to aggregate: it +-- stops when a level's local moving leaves every node alone, when the +-- refinement leaves every node alone (the next level's graph would be this +-- one), or after `max_levels`; in the last two cases it returns the +-- refined partition, so every exit returns a connected one. A level's +-- local moving stops when a sweep raises modularity by no more than +-- `tolerance`. That multilevel pass is one iteration; each later one +-- starts its first level's moving from the partition the one before found, +-- not from each node alone, as igraph's `n_iterations` and leidenalg's +-- `optimise_partition` loop do. The run stops after `iterations`, or +-- after an iteration whose partition is the one it started from: with the +-- same start and seed, the next would find it again. A negative +-- `iterations` runs until then, at most 100 iterations (a WARNING if it +-- is still changing). Arguments and output as warren.louvain's; +-- `randomness` is θ of the refinement. `levels_prefix` keeps every +-- iteration's levels in turn (`_1`, … numbered on across +-- iterations), so the last is `out`. The level's graph, a level graph of +-- warren.contract's (`_warren_level_`), is dropped at the end of +-- each iteration. +CREATE FUNCTION warren.leiden(graph text, "out" text, resolution float8 DEFAULT 1.0, + seed bigint DEFAULT 0, max_levels int DEFAULT 100, + tolerance float8 DEFAULT 1e-7, randomness float8 DEFAULT 0.01, + iterations int DEFAULT 2, weighted boolean DEFAULT NULL, levels_prefix text DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS float8 LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['moved', 'refined', 'initial', 'keys', 'down', 'refined_down', 'prev', 'changed']; + g text := graph; + gv bigint := v; + next_g text; + next_v bigint; + level int; + iteration int := 0; + written int := 0; + nodes bigint; + found bigint; + refined bigint; + q float8; +BEGIN + IF "out" IS NULL OR resolution IS NULL OR seed IS NULL OR max_levels IS NULL OR tolerance IS NULL + OR randomness IS NULL OR iterations IS NULL THEN + RAISE EXCEPTION '% must not be NULL', + CASE WHEN "out" IS NULL THEN 'out' WHEN resolution IS NULL THEN 'resolution' + WHEN seed IS NULL THEN 'seed' WHEN max_levels IS NULL THEN 'max_levels' + WHEN tolerance IS NULL THEN 'tolerance' WHEN randomness IS NULL THEN 'randomness' + ELSE 'iterations' END + USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with("out", '_warren_') OR starts_with(levels_prefix, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF max_levels < 1 THEN + RAISE EXCEPTION 'max_levels must be at least 1, not %', max_levels USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF iterations = 0 THEN + RAISE EXCEPTION 'iterations must be at least 1, or negative to run until the partition is stable, not 0' + USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF tolerance = 'NaN'::float8 OR tolerance < 0 THEN + RAISE EXCEPTION 'tolerance must be a number of at least 0, not %', tolerance + USING ERRCODE = 'invalid_parameter_value'; + END IF; + BEGIN + LOOP + iteration := iteration + 1; + -- A later iteration's first level moves from the last partition. + IF iteration > 1 THEN + PERFORM warren.vector_map(graph, '_warren_leiden_prev', '_warren_leiden_down', false, v); + END IF; + level := 0; + refined := NULL; + LOOP + level := level + 1; + PERFORM warren.report_progress('leiden', graph, format('iteration %s, level %s', iteration, level), + level - 1, max_levels); + nodes := (SELECT (e ->> 'node_count')::bigint FROM jsonb_array_elements(warren.info(g) -> 'versions') e + WHERE (e ->> 'version')::bigint = gv); + IF level > 1 THEN + PERFORM warren.vector_from_query(g, '_warren_leiden_keys', + format('SELECT node_id, node_id FROM warren.id_map(%L, NULL::bigint, %s)', g, gv), 'i64', + persist := false, version := gv); + END IF; + found := warren.local_moving(g, '_warren_leiden_moved', + initial := CASE WHEN level > 1 THEN '_warren_leiden_initial' + WHEN iteration > 1 THEN '_warren_leiden_prev' END, + keys := CASE WHEN level > 1 THEN '_warren_leiden_keys' END, + resolution := resolution, seed := seed, tolerance := tolerance, + weighted := CASE WHEN level = 1 THEN weighted ELSE true END, + persist := false, version := gv); + -- This level's partition, on `graph`. + IF level = 1 THEN + PERFORM warren.vector_map(graph, '_warren_leiden_down', '_warren_leiden_moved', false, v); + ELSE + PERFORM warren.lift(g, '_warren_leiden_moved', graph, '_warren_leiden_down', + persist := false, version := gv); + END IF; + -- Every node alone: the partition is this level's nodes, the refined + -- partition of the level below, which is connected. + IF found < nodes THEN + refined := warren.leiden_refine(g, '_warren_leiden_moved', '_warren_leiden_refined', + keys := CASE WHEN level > 1 THEN '_warren_leiden_keys' END, + resolution := resolution, seed := seed, randomness := randomness, + weighted := CASE WHEN level = 1 THEN weighted ELSE true END, + persist := false, version := gv); + IF level = 1 THEN + PERFORM warren.vector_map(graph, '_warren_leiden_refined_down', '_warren_leiden_refined', false, v); + ELSE + PERFORM warren.lift(g, '_warren_leiden_refined', graph, '_warren_leiden_refined_down', + persist := false, version := gv); + END IF; + -- Stopping here, return the refined partition, which is connected, + -- as Traag et al.'s is; the moved one need not be. + IF level >= max_levels OR refined = nodes THEN + PERFORM warren.vector_map(graph, '_warren_leiden_down', '_warren_leiden_refined_down', false, v); + END IF; + END IF; + IF levels_prefix IS NOT NULL THEN + written := written + 1; + PERFORM warren.vector_map(graph, format('%s_%s', levels_prefix, written), '_warren_leiden_down', true, v); + END IF; + EXIT WHEN found = nodes OR level >= max_levels OR refined = nodes; + -- A level graph of its own, named after its version, a name no + -- graph can have had. + next_v := warren.contract(graph, '_warren_leiden_refined_down', NULL, + weighted := coalesce(weighted, true), version := v); + next_g := '_warren_level_' || next_v; + -- The next level starts from the unrefined partition: every member + -- of a refined community is in the same community, so min is it. + PERFORM warren.project(graph, '_warren_leiden_down', next_g, '_warren_leiden_initial', 'min', + persist := false, version := next_v); + IF g <> graph THEN + PERFORM warren.drop_vector(g, '_warren_leiden_' || part, gv, true) FROM unnest(parts) part; + PERFORM warren.drop_graph(g); + END IF; + g := next_g; + gv := next_v; + END LOOP; + IF g <> graph THEN + PERFORM warren.drop_vector(g, '_warren_leiden_' || part, gv, true) FROM unnest(parts) part; + PERFORM warren.drop_graph(g); + g := graph; + gv := v; + END IF; + EXIT WHEN iteration = iterations; + -- Labels are the least dense id of each community's members, so the + -- partitions are the same exactly when the labels are. + IF iteration > 1 THEN + PERFORM warren.vector_map(graph, '_warren_leiden_changed', + 'CASE WHEN _warren_leiden_down = _warren_leiden_prev THEN 0 ELSE 1 END', false, v); + EXIT WHEN warren.vector_reduce(graph, '_warren_leiden_changed', 'sum', v) = 0; + END IF; + IF iterations < 0 AND iteration = 100 THEN + RAISE WARNING 'leiden did not stabilise in 100 iterations'; + EXIT; + END IF; + END LOOP; + PERFORM warren.vector_map(graph, "out", '_warren_leiden_down', true, v); + q := warren.modularity(graph, "out", resolution, weighted, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_leiden_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_leiden_' || part, v, true) FROM unnest(parts) part; + IF g <> graph THEN + PERFORM warren.drop_vector(g, '_warren_leiden_' || part, gv, true) FROM unnest(parts) part; + PERFORM warren.drop_graph(g); + END IF; + RETURN q; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/step.rs:25 +-- requires: +-- extension::warren + + +CREATE TYPE warren.step AS ( + rels text[], + direction text, + filter text[], + min_hops int, + max_hops int +); + +CREATE FUNCTION warren.steps(steps jsonb) RETURNS warren.step[] +IMMUTABLE STRICT PARALLEL SAFE LANGUAGE plpgsql +SET search_path = pg_catalog, pg_temp +AS $$ +DECLARE + step jsonb; + position int := 0; + key text; + result warren.step[] := '{}'; +BEGIN + IF jsonb_typeof(steps) <> 'array' THEN + RAISE EXCEPTION USING ERRCODE = 'invalid_parameter_value', + MESSAGE = format('steps must be a JSON array of step objects, not %s', + jsonb_typeof(steps)); + END IF; + FOR step IN SELECT e FROM jsonb_array_elements(steps) AS x(e) LOOP + position := position + 1; + IF jsonb_typeof(step) <> 'object' THEN + RAISE EXCEPTION USING ERRCODE = 'invalid_parameter_value', + MESSAGE = format('step %s must be an object, not %s', position, + jsonb_typeof(step)); + END IF; + FOR key IN SELECT jsonb_object_keys(step) LOOP + IF key NOT IN ('rels', 'direction', 'filter', 'min_hops', 'max_hops') THEN + RAISE EXCEPTION USING ERRCODE = 'invalid_parameter_value', + MESSAGE = format('step %s has an unknown key "%s"', position, key); + END IF; + END LOOP; + FOREACH key IN ARRAY ARRAY['rels', 'filter'] LOOP + IF jsonb_typeof(step -> key) NOT IN ('array', 'null') THEN + RAISE EXCEPTION USING ERRCODE = 'invalid_parameter_value', + MESSAGE = format('step %s: %s must be an array or null, not %s', position, + key, jsonb_typeof(step -> key)); + END IF; + END LOOP; + result := result || ROW( + CASE WHEN jsonb_typeof(step -> 'rels') = 'array' THEN + ARRAY(SELECT jsonb_array_elements_text(step -> 'rels')) END, + coalesce(step ->> 'direction', 'out'), + CASE WHEN jsonb_typeof(step -> 'filter') = 'array' THEN + ARRAY(SELECT jsonb_array_elements_text(step -> 'filter')) END, + coalesce((step ->> 'min_hops')::int, 1), + (step ->> 'max_hops')::int)::warren.step; + END LOOP; + RETURN result; +END +$$; + +CREATE CAST (jsonb AS warren.step[]) WITH FUNCTION warren.steps(jsonb); +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:7 +-- requires: +-- extension::warren + +-- warren.pagerank (DESIGN.md §6.7): PageRank by power iteration, as +-- NetworkX computes it. Rank flows along edges, split by out-weight (the +-- edge count when unweighted); dangling nodes' rank, and the 1 - damping +-- teleport, go to the personalisation: uniform, or proportional to vector +-- `seed`. Stops when the L1 change falls below (live nodes) × tol, or after +-- max_iter iterations. Writes vector `out` (f64, summing to 1) and returns +-- the iterations run. Constants enter expressions as constant vectors, +-- never as text, so no client setting (extra_float_digits) rounds them. +CREATE FUNCTION warren.pagerank(graph text, "out" text, damping float8 DEFAULT 0.85, + tol float8 DEFAULT 1e-6, max_iter int DEFAULT 100, + seed text DEFAULT NULL, rels text[] DEFAULT NULL, + weighted boolean DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['one', 'out', 'x', 'p', 'total', 'damping', 'teleport', 'msg', 'dangling_x', + 'dangling', 'y', 'next']; + nodes float8; + dangling float8; + delta float8; + i int := 0; +BEGIN + IF starts_with("out", '_warren_') OR starts_with(seed, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + BEGIN + -- Every live node's out-weight: the row sums of A. + PERFORM warren.vector_const(graph, '_warren_pagerank_one', 1, persist := false, version := v); + PERFORM warren.spmv(graph, '_warren_pagerank_one', '_warren_pagerank_out', 'plus_first', + rels := rels, weighted := weighted, persist := false, version := v); + nodes := warren.vector_reduce(graph, '_warren_pagerank_one', 'sum', v); + PERFORM warren.vector_const(graph, '_warren_pagerank_x', 1.0 / nodes, persist := false, version := v); + IF seed IS NULL THEN + PERFORM warren.vector_map(graph, '_warren_pagerank_p', '_warren_pagerank_x', false, v); + ELSE + -- Times the reciprocal: a constant vector is 0 at tombstoned rows, + -- where vector_map evaluates too, so dividing by one would fail. + PERFORM warren.vector_const(graph, '_warren_pagerank_total', 1.0 / warren.vector_reduce(graph, seed, 'sum', v), + persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_pagerank_p', format('%I * _warren_pagerank_total', seed), false, v); + END IF; + PERFORM warren.vector_const(graph, '_warren_pagerank_damping', damping, persist := false, version := v); + PERFORM warren.vector_const(graph, '_warren_pagerank_teleport', 1 - damping, persist := false, version := v); + WHILE i < max_iter LOOP + i := i + 1; + PERFORM warren.report_progress('pagerank', graph, 'iterating', i - 1, max_iter); + PERFORM warren.vector_map(graph, '_warren_pagerank_msg', + 'CASE WHEN _warren_pagerank_out > 0 THEN _warren_pagerank_x / _warren_pagerank_out ELSE 0 END', false, v); + PERFORM warren.vector_map(graph, '_warren_pagerank_dangling_x', + 'CASE WHEN _warren_pagerank_out > 0 THEN 0 ELSE _warren_pagerank_x END', false, v); + dangling := warren.vector_reduce(graph, '_warren_pagerank_dangling_x', 'sum', v); + PERFORM warren.vector_const(graph, '_warren_pagerank_dangling', dangling, persist := false, version := v); + PERFORM warren.spmv(graph, '_warren_pagerank_msg', '_warren_pagerank_y', transpose := true, + rels := rels, weighted := weighted, persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_pagerank_next', + '_warren_pagerank_damping * (_warren_pagerank_y + _warren_pagerank_dangling * _warren_pagerank_p) + + _warren_pagerank_teleport * _warren_pagerank_p', false, v); + delta := warren.vector_norm_diff(graph, '_warren_pagerank_next', '_warren_pagerank_x', 'l1', v, v); + PERFORM warren.vector_map(graph, '_warren_pagerank_x', '_warren_pagerank_next', false, v); + EXIT WHEN delta < nodes * tol; + END LOOP; + PERFORM warren.vector_map(graph, "out", '_warren_pagerank_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_pagerank_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_pagerank_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:24 +-- requires: +-- extension::warren + +-- warren.aggregate_neighbours (DESIGN.md §6.7, A.5): each node's row of +-- matrix `x` replaced by the mean, sum or max of its neighbours' rows +-- (out-neighbours, or in-neighbours with direction := 'in'; distinct +-- neighbours over `rels`; weights are not read), `hops` times over. A node +-- with no neighbours gets NaN for mean (SQL's AVG of nothing), 0 for sum +-- and -Infinity (the least bigint) for max. mean needs an f32 or f64 +-- matrix; the result keeps x's dtype. Writes matrix `out`. +CREATE FUNCTION warren.aggregate_neighbours(graph text, x text, "out" text, agg text DEFAULT 'mean', + hops int DEFAULT 1, rels text[] DEFAULT NULL, + direction text DEFAULT 'out', version bigint DEFAULT NULL) +RETURNS void LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['x', 'degree']; + semiring text := CASE agg WHEN 'mean' THEN 'plus_second' WHEN 'sum' THEN 'plus_second' + WHEN 'max' THEN 'max_second' END; + dtype text; +BEGIN + IF starts_with("out", '_warren_') OR starts_with(x, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF semiring IS NULL THEN + RAISE EXCEPTION 'agg must be mean, sum or max, not %', agg USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF direction NOT IN ('out', 'in') THEN + RAISE EXCEPTION 'direction must be ''out'' or ''in'', not %', direction USING ERRCODE = 'invalid_parameter_value'; + END IF; + dtype := warren.explain_map(graph, format('%I', x), v) ->> 'dtype'; + IF agg = 'mean' AND dtype NOT IN ('f32', 'f64') THEN + RAISE EXCEPTION 'mean needs an f32 or f64 matrix, not %', dtype USING ERRCODE = 'datatype_mismatch'; + END IF; + BEGIN + PERFORM warren.vector_map(graph, '_warren_aggregate_neighbours_x', format('%I', x), false, v); + IF agg = 'mean' THEN + PERFORM warren.vector_degree(graph, '_warren_aggregate_neighbours_degree', direction, rels, dtype, false, v); + END IF; + FOR hop IN 1..hops LOOP + PERFORM warren.report_progress('aggregate_neighbours', graph, 'hop', hop - 1, hops); + PERFORM warren.spmm(graph, '_warren_aggregate_neighbours_x', '_warren_aggregate_neighbours_x', semiring, + transpose := direction = 'in', rels := rels, weighted := false, persist := false, + version := v); + IF agg = 'mean' THEN + PERFORM warren.vector_map(graph, '_warren_aggregate_neighbours_x', + 'CASE WHEN _warren_aggregate_neighbours_degree > 0 + THEN _warren_aggregate_neighbours_x / _warren_aggregate_neighbours_degree END', false, v); + END IF; + END LOOP; + PERFORM warren.vector_map(graph, "out", '_warren_aggregate_neighbours_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_aggregate_neighbours_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_aggregate_neighbours_' || part, v, true) FROM unnest(parts) part; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:23 +-- requires: +-- extension::warren + +-- warren.sssp_vector (DESIGN.md §6.7): single-source shortest-path +-- distances by Bellman-Ford relaxation (min_plus SpMV along edges, +-- accumulating min) until no distance falls, or after max_iter iterations. +-- By default the bound is the live node count, and distances still falling +-- then mean a negative cycle reachable from the source, an error (22023). +-- Weights as spmv reads them (1 if unweighted); unreachable nodes are +-- Infinity. Writes vector `out` (f64) and returns the iterations run. +CREATE FUNCTION warren.sssp_vector(graph text, "out" text, source anyelement, + rels text[] DEFAULT NULL, weighted boolean DEFAULT NULL, + max_iter int DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['x', 'next', 'fell']; + limit_ int; + falling float8; + i int := 0; +BEGIN + IF starts_with("out", '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + BEGIN + PERFORM warren.vector_from_query(graph, '_warren_sssp_vector_x', + format('SELECT %L::%s, 0', source, pg_typeof(source)), + fill := 'Infinity', persist := false, version := v); + IF warren.vector_reduce(graph, '_warren_sssp_vector_x', 'min', v) <> 0 THEN + RAISE EXCEPTION 'node % is not a node of graph "%"', source, graph USING ERRCODE = 'invalid_parameter_value'; + END IF; + limit_ := coalesce(max_iter, (warren.vector_stats(graph, '_warren_sssp_vector_x', v) ->> 'count')::int); + WHILE i < limit_ LOOP + i := i + 1; + PERFORM warren.report_progress('sssp_vector', graph, 'iterating', i - 1, limit_); + PERFORM warren.vector_map(graph, '_warren_sssp_vector_next', '_warren_sssp_vector_x', false, v); + PERFORM warren.spmv(graph, '_warren_sssp_vector_x', '_warren_sssp_vector_next', 'min_plus', + transpose := true, rels := rels, weighted := weighted, accumulate := 'min', + persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_sssp_vector_fell', + 'CASE WHEN _warren_sssp_vector_next < _warren_sssp_vector_x THEN 1 ELSE 0 END', false, v); + falling := warren.vector_reduce(graph, '_warren_sssp_vector_fell', 'sum', v); + PERFORM warren.vector_map(graph, '_warren_sssp_vector_x', '_warren_sssp_vector_next', false, v); + EXIT WHEN falling = 0; + END LOOP; + IF max_iter IS NULL AND falling > 0 THEN + RAISE EXCEPTION 'a negative cycle is reachable from node % of graph "%"', source, graph + USING ERRCODE = 'invalid_parameter_value'; + END IF; + PERFORM warren.vector_map(graph, "out", '_warren_sssp_vector_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_sssp_vector_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_sssp_vector_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:13 +-- requires: +-- extension::warren + +-- warren.components_vector (DESIGN.md §6.7): weakly connected components by +-- min-label propagation (min_second SpMV both ways, accumulating min) until +-- no label changes. Each component is labelled by its smallest dense id, +-- as warren.components labels it. Writes vector `out` (i64) and returns +-- the iterations run. +CREATE FUNCTION warren.components_vector(graph text, "out" text, rels text[] DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['x', 'next']; + delta float8; + i int := 0; +BEGIN + IF starts_with("out", '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + BEGIN + PERFORM warren.vector_dense_id(graph, '_warren_components_vector_x', 'i64', false, v); + LOOP + i := i + 1; + PERFORM warren.report_progress('components_vector', graph, 'iterating', i - 1, NULL); + PERFORM warren.vector_map(graph, '_warren_components_vector_next', '_warren_components_vector_x', false, v); + PERFORM warren.spmv(graph, '_warren_components_vector_x', '_warren_components_vector_next', 'min_second', + rels := rels, accumulate := 'min', persist := false, version := v); + PERFORM warren.spmv(graph, '_warren_components_vector_x', '_warren_components_vector_next', 'min_second', + transpose := true, rels := rels, accumulate := 'min', persist := false, version := v); + delta := warren.vector_norm_diff(graph, '_warren_components_vector_next', '_warren_components_vector_x', + 'l1', v, v); + PERFORM warren.vector_map(graph, '_warren_components_vector_x', '_warren_components_vector_next', false, v); + EXIT WHEN delta = 0; + END LOOP; + PERFORM warren.vector_map(graph, "out", '_warren_components_vector_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_components_vector_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_components_vector_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/library/mod.rs:8 +-- requires: +-- extension::warren + +-- warren.louvain (DESIGN.md §6.7): Louvain community detection (Blondel +-- et al. 2008) as a library over the primitives. Each level moves nodes +-- between communities (warren.local_moving, from each node alone) on the +-- level's graph, lifts the partition found to `graph` (warren.lift), and +-- contracts `graph` by it (warren.contract) into the next level's graph, +-- whose nodes are the communities: contracting `graph` by the composed +-- partition gives the graph that contracting the level's graph would, so +-- every level's graph is a quotient of `graph`, one lift away, and only +-- one exists at a time. It stops when a level merges no node, when a +-- level raises modularity by no more than `tolerance` (NetworkX's +-- `threshold`), or after `max_levels`. The graph is undirected for this +-- (an edge and its reverse add up; edges weighted if `weighted`, NULL: if +-- the graph has weights). Communities are labelled by the smallest dense +-- id of their members: a level's nodes are keyed by those dense ids. Writes +-- vector `out` (i64, on version `version`, NULL: the current one), with +-- each level's partition as `_` if `levels_prefix` +-- is given, and returns the partition's modularity. The level's graph, a +-- level graph of warren.contract's (`_warren_level_`), and the +-- intermediate vectors are dropped at the end. +CREATE FUNCTION warren.louvain(graph text, "out" text, resolution float8 DEFAULT 1.0, + seed bigint DEFAULT 0, max_levels int DEFAULT 100, + tolerance float8 DEFAULT 1e-7, weighted boolean DEFAULT NULL, + levels_prefix text DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS float8 LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['single', 'labels', 'keys', 'down']; + g text := graph; + gv bigint := v; + next_g text; + next_v bigint; + level int := 0; + nodes bigint; + found bigint; + q float8; + q_next float8; +BEGIN + IF "out" IS NULL OR resolution IS NULL OR seed IS NULL OR max_levels IS NULL OR tolerance IS NULL THEN + RAISE EXCEPTION '% must not be NULL', + CASE WHEN "out" IS NULL THEN 'out' WHEN resolution IS NULL THEN 'resolution' + WHEN seed IS NULL THEN 'seed' WHEN max_levels IS NULL THEN 'max_levels' ELSE 'tolerance' END + USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with("out", '_warren_') OR starts_with(levels_prefix, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF max_levels < 1 THEN + RAISE EXCEPTION 'max_levels must be at least 1, not %', max_levels USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF tolerance = 'NaN'::float8 OR tolerance < 0 THEN + RAISE EXCEPTION 'tolerance must be a number of at least 0, not %', tolerance + USING ERRCODE = 'invalid_parameter_value'; + END IF; + BEGIN + PERFORM warren.vector_dense_id(graph, '_warren_louvain_single', persist := false, version := v); + q := warren.modularity(graph, '_warren_louvain_single', resolution, weighted, v); + LOOP + level := level + 1; + PERFORM warren.report_progress('louvain', graph, format('level %s', level), level - 1, max_levels); + nodes := (SELECT (e ->> 'node_count')::bigint FROM jsonb_array_elements(warren.info(g) -> 'versions') e + WHERE (e ->> 'version')::bigint = gv); + IF level > 1 THEN + -- A level's nodes are the communities found below it, keyed by + -- the least dense id of their members in `graph`. + PERFORM warren.vector_from_query(g, '_warren_louvain_keys', + format('SELECT node_id, node_id FROM warren.id_map(%L, NULL::bigint, %s)', g, gv), 'i64', + persist := false, version := gv); + END IF; + found := warren.local_moving(g, '_warren_louvain_labels', + keys := CASE WHEN level > 1 THEN '_warren_louvain_keys' END, + resolution := resolution, seed := seed, tolerance := 0, + weighted := CASE WHEN level = 1 THEN weighted ELSE true END, + persist := false, version := gv); + EXIT WHEN level > 1 AND found = nodes; + q_next := warren.modularity(g, '_warren_louvain_labels', resolution, + CASE WHEN level = 1 THEN weighted ELSE true END, gv); + -- This level's partition, on `graph`. + IF level = 1 THEN + PERFORM warren.vector_map(graph, '_warren_louvain_down', '_warren_louvain_labels', false, v); + ELSE + PERFORM warren.lift(g, '_warren_louvain_labels', graph, '_warren_louvain_down', + persist := false, version := gv); + END IF; + IF levels_prefix IS NOT NULL THEN + PERFORM warren.vector_map(graph, format('%s_%s', levels_prefix, level), '_warren_louvain_down', true, v); + END IF; + EXIT WHEN found = nodes OR level >= max_levels OR q_next - q <= tolerance; + q := q_next; + -- A level graph of its own, named after its version, a name no + -- graph can have had. + next_v := warren.contract(graph, '_warren_louvain_down', NULL, + weighted := coalesce(weighted, true), version := v); + next_g := '_warren_level_' || next_v; + IF g <> graph THEN + PERFORM warren.drop_vector(g, '_warren_louvain_' || part, gv, true) FROM unnest(parts) part; + PERFORM warren.drop_graph(g); + END IF; + g := next_g; + gv := next_v; + END LOOP; + PERFORM warren.vector_map(graph, "out", '_warren_louvain_down', true, v); + q := warren.modularity(graph, "out", resolution, weighted, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_louvain_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_louvain_' || part, v, true) FROM unnest(parts) part; + IF g <> graph THEN + PERFORM warren.drop_vector(g, '_warren_louvain_' || part, gv, true) FROM unnest(parts) part; + PERFORM warren.drop_graph(g); + END IF; + RETURN q; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:18 +-- requires: +-- extension::warren + +-- warren.label_propagation (DESIGN.md §6.7, A.1): min-label propagation. +-- Labels start as vector `seed` (default: each node's dense id); each +-- iteration every node takes the smallest label among itself and its +-- neighbours (both directions, over `rels`), until no label changes or +-- after max_iter iterations. With `mask`, only the nodes it selects (a +-- non-zero value, GraphBLAS semantics) send or take labels; the others +-- keep theirs. It is deterministic min-propagation, not majority-vote +-- label propagation. Writes vector `out` (f64) and returns the iterations +-- run. +CREATE FUNCTION warren.label_propagation(graph text, "out" text, seed text DEFAULT NULL, + max_iter int DEFAULT 100, mask text DEFAULT NULL, + rels text[] DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['x', 'msg', 'next']; + selected text; + delta float8; + i int := 0; +BEGIN + IF starts_with("out", '_warren_') OR starts_with(seed, '_warren_') OR starts_with(mask, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + BEGIN + IF seed IS NULL THEN + PERFORM warren.vector_dense_id(graph, '_warren_label_propagation_x', 'f64', false, v); + ELSE + PERFORM warren.vector_map(graph, '_warren_label_propagation_x', format('%I::float8', seed), false, v); + END IF; + IF mask IS NOT NULL THEN + selected := CASE warren.explain_map(graph, format('%I', mask), v) ->> 'dtype' + WHEN 'bool' THEN format('%I', mask) ELSE format('%I <> 0', mask) END; + END IF; + WHILE i < max_iter LOOP + i := i + 1; + PERFORM warren.report_progress('label_propagation', graph, 'iterating', i - 1, max_iter); + -- Unselected nodes send NULL (NaN), which min never picks over a label. + PERFORM warren.vector_map(graph, '_warren_label_propagation_msg', + CASE WHEN selected IS NULL THEN '_warren_label_propagation_x' + ELSE format('CASE WHEN %s THEN _warren_label_propagation_x END', selected) END, false, v); + PERFORM warren.vector_map(graph, '_warren_label_propagation_next', '_warren_label_propagation_x', false, v); + PERFORM warren.spmv(graph, '_warren_label_propagation_msg', '_warren_label_propagation_next', 'min_second', + rels := rels, mask := mask, accumulate := 'min', persist := false, version := v); + PERFORM warren.spmv(graph, '_warren_label_propagation_msg', '_warren_label_propagation_next', 'min_second', + transpose := true, rels := rels, mask := mask, accumulate := 'min', persist := false, + version := v); + delta := warren.vector_norm_diff(graph, '_warren_label_propagation_next', '_warren_label_propagation_x', + 'l1', v, v); + PERFORM warren.vector_map(graph, '_warren_label_propagation_x', '_warren_label_propagation_next', false, v); + EXIT WHEN delta = 0; + END LOOP; + PERFORM warren.vector_map(graph, "out", '_warren_label_propagation_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_label_propagation_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_label_propagation_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:34 +-- requires: +-- extension::warren + +-- warren.katz_centrality (DESIGN.md §6.7): Katz centrality by power +-- iteration, as NetworkX computes it. Centrality flows along edges, +-- weighted by edge weight (spmv's rule: weights if the graph has them, +-- unless weighted := false), attenuated by alpha, plus each node's beta: +-- the constant `beta`, or beta × vector `seed` (NetworkX's per-node beta). +-- From 0, each iteration takes alpha · Aᵀx + β; it stops when the L1 change +-- falls below (live nodes) × tol, or after max_iter iterations with a +-- WARNING (NetworkX raises). Alpha at or above 1 / the largest eigenvalue +-- diverges: once a value or the L2 norm is not finite the run is refused +-- (22003), as NetworkX raises, and writes nothing. With normalized, the +-- result is scaled to unit L2 norm. Writes vector `out` (f64) and returns +-- the iterations run. Constants enter expressions as constant vectors; a +-- constant vector is 0 at tombstoned rows, where vector_map evaluates too, +-- so a division by one is guarded. +CREATE FUNCTION warren.katz_centrality(graph text, "out" text, alpha float8 DEFAULT 0.1, + beta float8 DEFAULT 1.0, seed text DEFAULT NULL, + normalized boolean DEFAULT true, tol float8 DEFAULT 1e-6, + max_iter int DEFAULT 1000, rels text[] DEFAULT NULL, + weighted boolean DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['one', 'b', 'alpha', 'x', 'y', 'next', 'scale']; + nodes float8; + norm float8; + delta float8; + i int := 0; +BEGIN + IF "out" IS NULL OR alpha IS NULL OR beta IS NULL OR normalized IS NULL OR tol IS NULL OR max_iter IS NULL THEN + RAISE EXCEPTION '% must not be NULL', + CASE WHEN "out" IS NULL THEN 'out' WHEN alpha IS NULL THEN 'alpha' WHEN beta IS NULL THEN 'beta' + WHEN normalized IS NULL THEN 'normalized' WHEN tol IS NULL THEN 'tol' ELSE 'max_iter' END + USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with("out", '_warren_') OR starts_with(seed, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF tol < 0 THEN + RAISE EXCEPTION 'tol must be at least 0, not %', tol USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF max_iter < 1 THEN + RAISE EXCEPTION 'max_iter must be at least 1, not %', max_iter USING ERRCODE = 'invalid_parameter_value'; + END IF; + BEGIN + PERFORM warren.vector_const(graph, '_warren_katz_centrality_one', 1, persist := false, version := v); + nodes := warren.vector_reduce(graph, '_warren_katz_centrality_one', 'sum', v); + IF nodes = 0 THEN + RAISE EXCEPTION 'graph "%" has no live nodes', graph USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF seed IS NULL THEN + PERFORM warren.vector_const(graph, '_warren_katz_centrality_b', beta, persist := false, version := v); + ELSE + -- vector_reduce names a missing seed vector (42704); vector_map would + -- call it a missing column. + PERFORM warren.vector_reduce(graph, seed, 'nnz', v); + PERFORM warren.vector_map(graph, '_warren_katz_centrality_b', format('%I::float8', seed), false, v); + PERFORM warren.vector_const(graph, '_warren_katz_centrality_scale', beta, persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_katz_centrality_b', + '_warren_katz_centrality_scale * _warren_katz_centrality_b', false, v); + END IF; + PERFORM warren.vector_const(graph, '_warren_katz_centrality_alpha', alpha, persist := false, version := v); + PERFORM warren.vector_const(graph, '_warren_katz_centrality_x', 0, persist := false, version := v); + WHILE i < max_iter LOOP + i := i + 1; + PERFORM warren.report_progress('katz_centrality', graph, 'iterating', i - 1, max_iter); + PERFORM warren.spmv(graph, '_warren_katz_centrality_x', '_warren_katz_centrality_y', + transpose := true, rels := rels, weighted := weighted, persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_katz_centrality_next', + '_warren_katz_centrality_alpha * _warren_katz_centrality_y + _warren_katz_centrality_b', false, v); + delta := warren.vector_norm_diff(graph, '_warren_katz_centrality_next', '_warren_katz_centrality_x', + 'l1', v, v); + IF NOT delta < 'Infinity' THEN + RAISE EXCEPTION 'katz_centrality diverged: its values are not finite after % iterations', i + USING ERRCODE = 'numeric_value_out_of_range', + HINT = 'alpha must be below 1 / the largest eigenvalue of the adjacency matrix.'; + END IF; + PERFORM warren.vector_map(graph, '_warren_katz_centrality_x', '_warren_katz_centrality_next', false, v); + EXIT WHEN delta < nodes * tol; + END LOOP; + IF NOT delta < nodes * tol THEN + RAISE WARNING 'katz_centrality did not converge in % iterations', i; + END IF; + IF normalized THEN + norm := warren.vector_reduce(graph, '_warren_katz_centrality_x', 'l2', v); + IF NOT norm < 'Infinity' THEN + RAISE EXCEPTION 'katz_centrality diverged: its L2 norm overflows after % iterations', i + USING ERRCODE = 'numeric_value_out_of_range', + HINT = 'alpha must be below 1 / the largest eigenvalue of the adjacency matrix.'; + END IF; + PERFORM warren.vector_const(graph, '_warren_katz_centrality_scale', CASE WHEN norm = 0 THEN 1 ELSE norm END, + persist := false, version := v); + PERFORM warren.vector_map(graph, "out", + 'CASE WHEN _warren_katz_centrality_scale <> 0 + THEN _warren_katz_centrality_x / _warren_katz_centrality_scale ELSE 0 END', true, v); + ELSE + PERFORM warren.vector_map(graph, "out", '_warren_katz_centrality_x', true, v); + END IF; + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_katz_centrality_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_katz_centrality_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:41 +-- requires: +-- extension::warren + +-- warren.label_propagation_majority (DESIGN.md §6.7): majority-vote label +-- propagation, synchronous and deterministic. Labels start as vector `seed` +-- (default: each node's dense id); each iteration every node takes the most +-- frequent label among its own and its neighbours' (direction ignored, each +-- neighbour once, over `rels`; a self-loop counts the node's own label once +-- more; weights are not read), the smallest on a tie, until no label +-- changes, or after max_iter iterations with a WARNING (a synchronous +-- update can oscillate). NetworkX's label propagation is asynchronous +-- (random order and ties) or semi-synchronous (by colouring), so neither +-- matches step for step. Tier 2: each iteration is one `gather` (mode, with +-- include_self), which counts the labels in-engine, N rows through the +-- executor. Writes vector `out` (f64) and returns the iterations run. +CREATE FUNCTION warren.label_propagation_majority(graph text, "out" text, seed text DEFAULT NULL, + max_iter int DEFAULT 100, rels text[] DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + info jsonb := warren.info(graph); + v bigint := coalesce(version, (info ->> 'current_version')::bigint); + parts text[] := ARRAY['x', 'next']; + -- Each node's own label and its neighbours', counted in-engine; the most + -- votes win, then the smallest label. + votes text := format( + 'SELECT node_id, value + FROM warren.gather(%1$L, %2$L, NULL::%3$s, NULL::float8, ''mode'', ''both'', %4$L::text[], + version := %5$s, include_self := true)', + graph, '_warren_label_propagation_majority_x', info ->> 'id_type', rels, v); + nodes bigint; + delta float8; + i int := 0; +BEGIN + IF "out" IS NULL OR max_iter IS NULL THEN + RAISE EXCEPTION '% must not be NULL', CASE WHEN "out" IS NULL THEN 'out' ELSE 'max_iter' END + USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with("out", '_warren_') OR starts_with(seed, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF max_iter < 1 THEN + RAISE EXCEPTION 'max_iter must be at least 1, not %', max_iter USING ERRCODE = 'invalid_parameter_value'; + END IF; + BEGIN + IF seed IS NULL THEN + PERFORM warren.vector_dense_id(graph, '_warren_label_propagation_majority_x', 'f64', false, v); + ELSE + -- vector_reduce names a missing seed vector (42704); vector_map would + -- call it a missing column. + PERFORM warren.vector_reduce(graph, seed, 'nnz', v); + PERFORM warren.vector_map(graph, '_warren_label_propagation_majority_x', format('%I::float8', seed), false, v); + END IF; + -- No live nodes, no labels: no iterations (NetworkX finds no communities). + nodes := (warren.vector_stats(graph, '_warren_label_propagation_majority_x', v) ->> 'count')::bigint; + WHILE i < max_iter AND nodes > 0 LOOP + i := i + 1; + PERFORM warren.report_progress('label_propagation_majority', graph, 'iterating', i - 1, max_iter); + PERFORM warren.vector_from_query(graph, '_warren_label_propagation_majority_next', votes, 'f64', + persist := false, version := v); + delta := warren.vector_norm_diff(graph, '_warren_label_propagation_majority_next', + '_warren_label_propagation_majority_x', 'l1', v, v); + PERFORM warren.vector_map(graph, '_warren_label_propagation_majority_x', + '_warren_label_propagation_majority_next', false, v); + EXIT WHEN delta = 0; + END LOOP; + IF delta <> 0 THEN + RAISE WARNING 'label_propagation_majority did not converge in % iterations', i; + END IF; + PERFORM warren.vector_map(graph, "out", '_warren_label_propagation_majority_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_label_propagation_majority_' || part, v, true) + FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_label_propagation_majority_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:40 +-- requires: +-- extension::warren + +-- warren.clustering (DESIGN.md §6.7): the local clustering coefficient, +-- as NetworkX's clustering computes it for an undirected, unweighted +-- graph: 2T / (d(d - 1)) from each node's triangles T (warren.triangles: +-- directions and self-loops ignored) and its distinct neighbours d in +-- either direction over `rels`, and 0 where T is 0. Unlike NetworkX, a +-- self-loop counts its node among its own neighbours in d (vector_degree +-- does; no primitive tells which nodes have one). Edge weights are not +-- read. Writes vector `out` (f64). +CREATE FUNCTION warren.clustering(graph text, "out" text, rels text[] DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS void LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['triangles', 'degree']; +BEGIN + IF "out" IS NULL THEN + RAISE EXCEPTION 'out must not be NULL' USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with("out", '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + BEGIN + PERFORM warren.triangles(graph, '_warren_clustering_triangles', rels, false, v); + PERFORM warren.vector_degree(graph, '_warren_clustering_degree', 'both', rels, 'f64', false, v); + PERFORM warren.vector_map(graph, "out", + 'CASE WHEN _warren_clustering_triangles = 0 THEN 0 + ELSE 2 * _warren_clustering_triangles::float8 + / (_warren_clustering_degree * (_warren_clustering_degree - 1)) END', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_clustering_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_clustering_' || part, v, true) FROM unnest(parts) part; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:8 +-- requires: +-- extension::warren + +-- warren.degree_vector (DESIGN.md §6.7): each node's degree in `direction` +-- over `rels`, as vector `out` (f64); returns its vector id. +CREATE FUNCTION warren.degree_vector(graph text, "out" text, direction text DEFAULT 'out', + rels text[] DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS bigint LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +BEGIN + IF starts_with("out", '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + RETURN warren.vector_degree(graph, "out", direction, rels, 'f64', true, version); +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:29 +-- requires: +-- extension::warren + +-- warren.eigenvector_centrality (DESIGN.md §6.7): eigenvector centrality +-- by power iteration, as NetworkX computes it. Centrality flows along edges +-- (the left eigenvector of A: a node is central when central nodes point at +-- it), weighted by edge weight (spmv's rule: weights if the graph has them, +-- unless weighted := false). From the uniform 1/N, each iteration takes +-- x + Aᵀx (the shift by I keeps a periodic graph from oscillating) and +-- scales it to unit L2 norm; it stops when the L1 change falls below +-- (live nodes) × tol, or after max_iter iterations with a WARNING (NetworkX +-- raises). Values too large for double precision (an L2 norm past about +-- 1e154) are refused (22003) and write nothing. Writes vector `out` (f64, +-- unit L2 norm) and returns the iterations run. Constants enter +-- expressions as constant vectors; a constant vector is 0 at tombstoned +-- rows, where vector_map evaluates too, so a division by one is guarded. +CREATE FUNCTION warren.eigenvector_centrality(graph text, "out" text, tol float8 DEFAULT 1e-6, + max_iter int DEFAULT 100, rels text[] DEFAULT NULL, + weighted boolean DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['one', 'x', 'y', 'next', 'scale']; + nodes float8; + norm float8; + delta float8; + i int := 0; +BEGIN + IF "out" IS NULL OR tol IS NULL OR max_iter IS NULL THEN + RAISE EXCEPTION '% must not be NULL', CASE WHEN "out" IS NULL THEN 'out' WHEN tol IS NULL THEN 'tol' ELSE 'max_iter' END + USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with("out", '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF tol < 0 THEN + RAISE EXCEPTION 'tol must be at least 0, not %', tol USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF max_iter < 1 THEN + RAISE EXCEPTION 'max_iter must be at least 1, not %', max_iter USING ERRCODE = 'invalid_parameter_value'; + END IF; + BEGIN + PERFORM warren.vector_const(graph, '_warren_eigenvector_centrality_one', 1, persist := false, version := v); + nodes := warren.vector_reduce(graph, '_warren_eigenvector_centrality_one', 'sum', v); + IF nodes = 0 THEN + RAISE EXCEPTION 'graph "%" has no live nodes', graph USING ERRCODE = 'invalid_parameter_value'; + END IF; + PERFORM warren.vector_const(graph, '_warren_eigenvector_centrality_x', 1.0 / nodes, persist := false, + version := v); + WHILE i < max_iter LOOP + i := i + 1; + PERFORM warren.report_progress('eigenvector_centrality', graph, 'iterating', i - 1, max_iter); + PERFORM warren.spmv(graph, '_warren_eigenvector_centrality_x', '_warren_eigenvector_centrality_y', + transpose := true, rels := rels, weighted := weighted, persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_eigenvector_centrality_next', + '_warren_eigenvector_centrality_x + _warren_eigenvector_centrality_y', false, v); + norm := warren.vector_reduce(graph, '_warren_eigenvector_centrality_next', 'l2', v); + IF NOT norm < 'Infinity' THEN + RAISE EXCEPTION 'eigenvector_centrality diverged: its values are not finite after % iterations', i + USING ERRCODE = 'numeric_value_out_of_range', + HINT = 'The edge weights are too large: their squares overflow double precision.'; + END IF; + PERFORM warren.vector_const(graph, '_warren_eigenvector_centrality_scale', + CASE WHEN norm = 0 THEN 1 ELSE norm END, persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_eigenvector_centrality_next', + 'CASE WHEN _warren_eigenvector_centrality_scale <> 0 + THEN _warren_eigenvector_centrality_next / _warren_eigenvector_centrality_scale ELSE 0 END', false, v); + delta := warren.vector_norm_diff(graph, '_warren_eigenvector_centrality_next', + '_warren_eigenvector_centrality_x', 'l1', v, v); + PERFORM warren.vector_map(graph, '_warren_eigenvector_centrality_x', '_warren_eigenvector_centrality_next', + false, v); + EXIT WHEN delta < nodes * tol; + END LOOP; + IF NOT delta < nodes * tol THEN + RAISE WARNING 'eigenvector_centrality did not converge in % iterations', i; + END IF; + PERFORM warren.vector_map(graph, "out", '_warren_eigenvector_centrality_x', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_eigenvector_centrality_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_eigenvector_centrality_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/library/mod.rs:39 +-- requires: +-- extension::warren + +-- warren.hits (DESIGN.md §6.7): HITS hubs and authorities by power +-- iteration, as NetworkX computes them (its default, method := +-- 'power_iteration'). From hubs uniform at 1/N, each iteration takes the +-- authorities a = Aᵀh (a node pointed at by good hubs) and the hubs h = A·a +-- (a node pointing at good authorities), weighted by edge weight (spmv's +-- rule: weights if the graph has them, unless weighted := false), and +-- scales h to a largest value of 1; it stops when the L1 change of h falls +-- below tol (not scaled by N, as NetworkX), or after max_iter iterations +-- with a WARNING (NetworkX raises). With normalized, each result sums to +-- 1; otherwise each has a largest value of 1. A zero sum or maximum (no +-- edges) leaves the zeros as they are. Values too large for double +-- precision are refused (22003) and write nothing. Writes vectors `hubs` +-- and `authorities` (f64) and returns the iterations run. A constant +-- vector is 0 at tombstoned rows, where vector_map evaluates too, so a +-- division by one is guarded. +CREATE FUNCTION warren.hits(graph text, hubs text, authorities text, tol float8 DEFAULT 1e-8, + max_iter int DEFAULT 100, normalized boolean DEFAULT true, + rels text[] DEFAULT NULL, weighted boolean DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS int LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + v bigint := coalesce(version, (warren.info(graph) ->> 'current_version')::bigint); + parts text[] := ARRAY['one', 'x', 'a', 'next', 'scale']; + nodes float8; + scale float8; + hub_scale float8; + delta float8; + i int := 0; +BEGIN + IF hubs IS NULL OR authorities IS NULL OR normalized IS NULL OR tol IS NULL OR max_iter IS NULL THEN + RAISE EXCEPTION '% must not be NULL', + CASE WHEN hubs IS NULL THEN 'hubs' WHEN authorities IS NULL THEN 'authorities' + WHEN normalized IS NULL THEN 'normalized' WHEN tol IS NULL THEN 'tol' ELSE 'max_iter' END + USING ERRCODE = 'null_value_not_allowed'; + END IF; + IF starts_with(hubs, '_warren_') OR starts_with(authorities, '_warren_') THEN + RAISE EXCEPTION 'vector names beginning with _warren_ are reserved for the library''s intermediates' + USING ERRCODE = 'reserved_name'; + END IF; + IF hubs = authorities THEN + RAISE EXCEPTION 'hubs and authorities must be different vectors' USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF tol < 0 THEN + RAISE EXCEPTION 'tol must be at least 0, not %', tol USING ERRCODE = 'invalid_parameter_value'; + END IF; + IF max_iter < 1 THEN + RAISE EXCEPTION 'max_iter must be at least 1, not %', max_iter USING ERRCODE = 'invalid_parameter_value'; + END IF; + BEGIN + PERFORM warren.vector_const(graph, '_warren_hits_one', 1, persist := false, version := v); + nodes := warren.vector_reduce(graph, '_warren_hits_one', 'sum', v); + IF nodes = 0 THEN + RAISE EXCEPTION 'graph "%" has no live nodes', graph USING ERRCODE = 'invalid_parameter_value'; + END IF; + PERFORM warren.vector_const(graph, '_warren_hits_x', 1.0 / nodes, persist := false, version := v); + WHILE i < max_iter LOOP + i := i + 1; + PERFORM warren.report_progress('hits', graph, 'iterating', i - 1, max_iter); + PERFORM warren.spmv(graph, '_warren_hits_x', '_warren_hits_a', transpose := true, rels := rels, + weighted := weighted, persist := false, version := v); + PERFORM warren.spmv(graph, '_warren_hits_a', '_warren_hits_next', rels := rels, weighted := weighted, + persist := false, version := v); + scale := warren.vector_reduce(graph, '_warren_hits_next', 'max', v); + IF NOT abs(scale) < 'Infinity' THEN + RAISE EXCEPTION 'hits diverged: its values are not finite after % iterations', i + USING ERRCODE = 'numeric_value_out_of_range', + HINT = 'The edge weights are too large: their products overflow double precision.'; + END IF; + PERFORM warren.vector_const(graph, '_warren_hits_scale', CASE WHEN scale = 0 THEN 1 ELSE scale END, + persist := false, version := v); + PERFORM warren.vector_map(graph, '_warren_hits_next', + 'CASE WHEN _warren_hits_scale <> 0 THEN _warren_hits_next / _warren_hits_scale ELSE 0 END', false, v); + delta := warren.vector_norm_diff(graph, '_warren_hits_next', '_warren_hits_x', 'l1', v, v); + PERFORM warren.vector_map(graph, '_warren_hits_x', '_warren_hits_next', false, v); + EXIT WHEN delta < tol; + END LOOP; + IF NOT delta < tol THEN + RAISE WARNING 'hits did not converge in % iterations', i; + END IF; + IF normalized THEN + hub_scale := warren.vector_reduce(graph, '_warren_hits_x', 'sum', v); + scale := warren.vector_reduce(graph, '_warren_hits_a', 'sum', v); + ELSE + hub_scale := 1; + scale := warren.vector_reduce(graph, '_warren_hits_a', 'max', v); + END IF; + IF NOT abs(hub_scale) < 'Infinity' OR NOT abs(scale) < 'Infinity' THEN + RAISE EXCEPTION 'hits diverged: its values are not finite after % iterations', i + USING ERRCODE = 'numeric_value_out_of_range', + HINT = 'The edge weights are too large: their products overflow double precision.'; + END IF; + PERFORM warren.vector_const(graph, '_warren_hits_scale', CASE WHEN hub_scale = 0 THEN 1 ELSE hub_scale END, + persist := false, version := v); + PERFORM warren.vector_map(graph, hubs, + 'CASE WHEN _warren_hits_scale <> 0 THEN _warren_hits_x / _warren_hits_scale ELSE 0 END', true, v); + PERFORM warren.vector_const(graph, '_warren_hits_scale', CASE WHEN scale = 0 THEN 1 ELSE scale END, + persist := false, version := v); + PERFORM warren.vector_map(graph, authorities, + 'CASE WHEN _warren_hits_scale <> 0 THEN _warren_hits_a / _warren_hits_scale ELSE 0 END', true, v); + EXCEPTION WHEN OTHERS OR query_canceled THEN + PERFORM warren.drop_vector(graph, '_warren_hits_' || part, v, true) FROM unnest(parts) part; + RAISE; + END; + PERFORM warren.drop_vector(graph, '_warren_hits_' || part, v, true) FROM unnest(parts) part; + RETURN i; +END $$; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/readers.rs:12 +-- pg_warren::catalog::readers::grant + +CREATE FUNCTION warren."grant"(name text, role regrole) RETURNS void +STRICT VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'grant_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/cache/warm.rs:37 +-- pg_warren::cache::warm::evict +CREATE FUNCTION warren."evict"( + "name" TEXT /* & str */ +) RETURNS bigint /* i64 */ +STRICT VOLATILE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'evict_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/components.rs:23 +-- pg_warren::traversal_sql::components::components + +CREATE FUNCTION warren.components(graph text, id_type anyelement, version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, component_id bigint) +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'components_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/bfs.rs:87 +-- pg_warren::traversal_sql::bfs::reachable_set + +CREATE FUNCTION warren.reachable_set(graph text, start anyarray, max_depth int DEFAULT NULL, + rels text[] DEFAULT NULL, direction text DEFAULT 'out', + version bigint DEFAULT NULL) +RETURNS SETOF anyelement +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'reachable_set_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/product.rs:172 +-- pg_warren::compute_sql::product::spmv +CREATE FUNCTION warren."spmv"( + "graph" TEXT, /* Option < & str > */ + "x" TEXT, /* Option < & str > */ + "y" TEXT, /* Option < & str > */ + "semiring" TEXT DEFAULT 'plus_times', /* Option < & str > */ + "transpose" bool DEFAULT false, /* Option < bool > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "mask" TEXT DEFAULT NULL, /* Option < & str > */ + "mask_complement" bool DEFAULT false, /* Option < bool > */ + "replace" bool DEFAULT false, /* Option < bool > */ + "accumulate" TEXT DEFAULT NULL, /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS void +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'spmv_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/drop.rs:16 +-- pg_warren::compute_sql::drop::drop_vector +CREATE FUNCTION warren."drop_vector"( + "graph" TEXT, /* Option < & str > */ + "name" TEXT, /* Option < & str > */ + "version" bigint DEFAULT NULL, /* Option < i64 > */ + "if_exists" bool DEFAULT false /* Option < bool > */ +) RETURNS void +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'drop_vector_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/constructors.rs:93 +-- pg_warren::compute_sql::constructors::vector_degree +CREATE FUNCTION warren."vector_degree"( + "graph" TEXT, /* Option < & str > */ + "name" TEXT, /* Option < & str > */ + "direction" TEXT DEFAULT 'out', /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "dtype" TEXT DEFAULT 'f64', /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_degree_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/structure.rs:137 +-- pg_warren::compute_sql::structure::dag_longest_path +CREATE FUNCTION warren."dag_longest_path"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS double precision /* Option < f64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'dag_longest_path_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/structure.rs:61 +-- pg_warren::compute_sql::structure::strong_components_vector +CREATE FUNCTION warren."strong_components_vector"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'strong_components_vector_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/dropped_roles/orphans.rs:34 +-- pg_warren::catalog::dropped_roles::orphans::orphans +CREATE FUNCTION warren."orphans"() RETURNS TABLE ( + "kind" TEXT, /* String */ + "graph" TEXT, /* String */ + "role" oid /* pg_sys :: Oid */ +) +STRICT VOLATILE PARALLEL SAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'orphans_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/projection.rs:26 +-- pg_warren::community_sql::projection::project +CREATE FUNCTION warren."project"( + "graph" TEXT, /* Option < & str > */ + "x" TEXT, /* Option < & str > */ + "out_graph" TEXT, /* Option < & str > */ + "y" TEXT, /* Option < & str > */ + "agg" TEXT DEFAULT 'sum', /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS void +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'project_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/paths.rs:116 +-- pg_warren::traversal_sql::paths::dijkstra + +CREATE FUNCTION warren.dijkstra(graph text, start anyelement, target anyelement, + rels text[] DEFAULT NULL, direction text DEFAULT 'out', + version bigint DEFAULT NULL) +RETURNS TABLE (path anyarray, cost float8) +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'dijkstra_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/similarity.rs:245 +-- pg_warren::compute_sql::similarity::similar_pairs + +CREATE FUNCTION warren.similar_pairs(graph text, id_type anyelement, k bigint DEFAULT 10, + measure text DEFAULT 'jaccard', nodes text DEFAULT NULL, + rels text[] DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS TABLE (source anyelement, target anyelement, score float8) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'similar_pairs_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/centrality.rs:89 +-- pg_warren::compute_sql::centrality::betweenness_centrality +CREATE FUNCTION warren."betweenness_centrality"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "k" bigint DEFAULT NULL, /* Option < i64 > */ + "seed" bigint DEFAULT 0, /* Option < i64 > */ + "normalized" bool DEFAULT true, /* Option < bool > */ + "endpoints" bool DEFAULT false, /* Option < bool > */ + "directed" bool DEFAULT true, /* Option < bool > */ + "sources" TEXT DEFAULT NULL, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'betweenness_centrality_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/summary.rs:22 +-- pg_warren::compute_sql::summary::vector_topk + +CREATE FUNCTION warren.vector_topk(graph text, name text, k int, id_type anyelement, + descending bool DEFAULT true, version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, value float8) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'vector_topk_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/constructors.rs:132 +-- pg_warren::compute_sql::constructors::vector_dense_id +CREATE FUNCTION warren."vector_dense_id"( + "graph" TEXT, /* Option < & str > */ + "name" TEXT, /* Option < & str > */ + "dtype" TEXT DEFAULT 'i64', /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_dense_id_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/activity/progress.rs:145 +-- pg_warren::activity::progress::progress + +CREATE FUNCTION warren."progress"() RETURNS TABLE ( + "pid" int, + "database" oid, + "role" regrole, + "command" text, + "graph" text, + "version" bigint, + "phase" text, + "done" bigint, + "total" bigint, + "operation" text, + "operation_phase" text, + "operation_done" bigint, + "operation_total" bigint, + "detail" jsonb, + "command_start" timestamptz, + "operation_start" timestamptz +) +VOLATILE PARALLEL SAFE +LANGUAGE c AS 'MODULE_PATHNAME', 'progress_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/structure.rs:26 +-- pg_warren::compute_sql::structure::core_number +CREATE FUNCTION warren."core_number"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'core_number_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/structure.rs:176 +-- pg_warren::compute_sql::structure::minimum_spanning_forest + +CREATE FUNCTION warren.minimum_spanning_forest(graph text, id_type anyelement, rels text[] DEFAULT NULL, + weighted bool DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS TABLE (source anyelement, target anyelement, weight float8) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'minimum_spanning_forest_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/constructors.rs:38 +-- pg_warren::compute_sql::constructors::vector_const +CREATE FUNCTION warren."vector_const"( + "graph" TEXT, /* Option < & str > */ + "name" TEXT, /* Option < & str > */ + "value" double precision, /* Option < f64 > */ + "dtype" TEXT DEFAULT 'f64', /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_const_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/vector/quota/sql.rs:22 +-- pg_warren::vector::quota::sql::set_vector_quota + +CREATE FUNCTION warren."set_vector_quota"(name text, bytes bigint, role regrole DEFAULT NULL) +RETURNS void VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'set_vector_quota_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/promote.rs:14 +-- pg_warren::catalog::promote::promote +CREATE FUNCTION warren."promote"( + "name" TEXT, /* & str */ + "version" bigint /* i64 */ +) RETURNS void +STRICT +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'promote_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/paths.rs:71 +-- pg_warren::traversal_sql::paths::shortest_path + +CREATE FUNCTION warren.shortest_path(graph text, start anyelement, target anyelement, + max_depth int DEFAULT NULL, rels text[] DEFAULT NULL, + direction text DEFAULT 'out', version bigint DEFAULT NULL) +RETURNS anyarray +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'shortest_path_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/build/refresh.rs:22 +-- pg_warren::build::refresh::refresh +CREATE FUNCTION warren."refresh"( + "name" TEXT, /* & str */ + "promote" bool DEFAULT true, /* bool */ + "label" TEXT DEFAULT NULL /* Option < & str > */ +) RETURNS bigint /* i64 */ + +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'refresh_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/structure.rs:101 +-- pg_warren::compute_sql::structure::topological_order +CREATE FUNCTION warren."topological_order"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'topological_order_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/neighbourhood.rs:54 +-- pg_warren::traversal_sql::neighbourhood::degree + +CREATE FUNCTION warren.degree(graph text, node anyelement, direction text DEFAULT 'out', + rels text[] DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS bigint +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'degree_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/spectral_sql/distance/mod.rs:97 +-- pg_warren::spectral_sql::distance::distance +CREATE FUNCTION warren."distance"( + "graph" TEXT, /* Option < & str > */ + "v1" bigint, /* Option < i64 > */ + "v2" bigint, /* Option < i64 > */ + "method" TEXT, /* Option < & str > */ + "vectors_prefix" TEXT DEFAULT NULL, /* Option < & str > */ + "k" INT DEFAULT NULL, /* Option < i32 > */ + "x" TEXT DEFAULT NULL, /* Option < & str > */ + "y" TEXT DEFAULT NULL, /* Option < & str > */ + "compared_rows" TEXT DEFAULT 'shared', /* Option < & str > */ + "matrix" TEXT DEFAULT NULL, /* Option < & str > */ + "which" TEXT DEFAULT 'largest', /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "symmetrize" bool DEFAULT true, /* Option < bool > */ + "tol" double precision DEFAULT 1e-6, /* Option < f64 > */ + "max_iter" INT DEFAULT NULL, /* Option < i32 > */ + "dtype" TEXT DEFAULT 'f32', /* Option < & str > */ + "graph2" TEXT DEFAULT NULL /* Option < & str > */ +) RETURNS jsonb /* JsonB */ +STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'distance_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/owner/reassign.rs:22 +-- pg_warren::catalog::owner::reassign::reassign_owned + +CREATE FUNCTION warren."reassign_owned"("old" regrole, "new" regrole) RETURNS bigint +STRICT VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'reassign_owned_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/drop.rs:29 +-- pg_warren::catalog::drop::drop_graph +CREATE FUNCTION warren."drop_graph"( + "name" TEXT /* & str */ +) RETURNS void +STRICT +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'drop_graph_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/dropped_roles/event_trigger.rs:21 +-- pg_warren::catalog::dropped_roles::event_trigger::on_drop_owned + +CREATE FUNCTION warren."on_drop_owned"() RETURNS event_trigger +VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'on_drop_owned_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/map/mod.rs:91 +-- pg_warren::compute_sql::map::vector_map +CREATE FUNCTION warren."vector_map"( + "graph" TEXT, /* Option < & str > */ + "y" TEXT, /* Option < & str > */ + "expr" TEXT, /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS void +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_map_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/drop.rs:13 +-- pg_warren::catalog::drop::drop_version +CREATE FUNCTION warren."drop_version"( + "name" TEXT, /* & str */ + "version" bigint /* i64 */ +) RETURNS void +STRICT +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'drop_version_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/cache/shared/introspection.rs:136 +-- pg_warren::cache::shared::introspection::shared_cache +CREATE FUNCTION warren."shared_cache"() RETURNS TABLE ( + "kind" TEXT, /* String */ + "graph" TEXT, /* Option < String > */ + "version" bigint, /* Option < i64 > */ + "vector" TEXT, /* Option < String > */ + "vector_id" bigint, /* Option < i64 > */ + "state" TEXT, /* Option < String > */ + "bytes" bigint, /* i64 */ + "pins" INT, /* i32 */ + "mappers" INT, /* i32 */ + "last_use" bigint, /* i64 */ + "cap" bigint /* Option < i64 > */ +) +STRICT VOLATILE PARALLEL SAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'shared_cache_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/activity/backends.rs:120 +-- pg_warren::activity::backends::backends + +CREATE FUNCTION warren."backends"() RETURNS TABLE ( + "kind" text, + "pid" int, + "database" oid, + "role" regrole, + "private_bytes" bigint, + "transient_bytes" bigint, + "mapped_bytes" bigint, + "lookups" bigint, + "private_hits" bigint, + "shared_hits" bigint, + "waits" bigint, + "private_loads" bigint, + "shared_loads" bigint, + "loaded_bytes" bigint, + "load_seconds" float8, + "private_evictions" bigint, + "shared_evictions" bigint, + "refusals" jsonb, + "since" timestamptz +) +VOLATILE PARALLEL SAFE +LANGUAGE c AS 'MODULE_PATHNAME', 'backends_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/reduce.rs:47 +-- pg_warren::compute_sql::reduce::vector_norm_diff +CREATE FUNCTION warren."vector_norm_diff"( + "graph" TEXT, /* Option < & str > */ + "a" TEXT, /* Option < & str > */ + "b" TEXT, /* Option < & str > */ + "norm" TEXT DEFAULT 'l2', /* Option < & str > */ + "a_version" bigint DEFAULT NULL, /* Option < i64 > */ + "b_version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS double precision /* f64 */ +STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_norm_diff_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/paths.rs:27 +-- pg_warren::traversal_sql::paths::reachable + +CREATE FUNCTION warren.reachable(graph text, start anyelement, target anyelement, + max_depth int DEFAULT NULL, rels text[] DEFAULT NULL, + direction text DEFAULT 'out', version bigint DEFAULT NULL) +RETURNS boolean +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'reachable_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/similarity.rs:48 +-- pg_warren::compute_sql::similarity::similarity + +CREATE FUNCTION warren.similarity(graph text, pairs text, id_type anyelement, + measure text DEFAULT 'jaccard', rels text[] DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS TABLE (source anyelement, target anyelement, score float8) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'similarity_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/read.rs:134 +-- pg_warren::compute_sql::read::id_map + +CREATE FUNCTION warren.id_map(graph text, id_type anyelement, version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, dense_id bigint, live boolean) +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'id_map_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/gather/mod.rs:354 +-- pg_warren::compute_sql::gather::gather + +CREATE FUNCTION warren.gather(graph text, messages text, id_type anyelement, + value_type anycompatible, combine text DEFAULT 'collect', + direction text DEFAULT 'in', rels text[] DEFAULT NULL, + mask text DEFAULT NULL, mask_complement boolean DEFAULT false, + version bigint DEFAULT NULL, include_self boolean DEFAULT false) +RETURNS TABLE (node_id anyelement, value anycompatible) +VOLATILE PARALLEL UNSAFE LANGUAGE c AS 'MODULE_PATHNAME', 'gather_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/stats.rs:30 +-- pg_warren::traversal_sql::stats::last_stats +CREATE FUNCTION warren."last_stats"() RETURNS jsonb /* Option < JsonB > */ +STRICT VOLATILE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'last_stats_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/map/mod.rs:156 +-- pg_warren::compute_sql::map::explain_map +CREATE FUNCTION warren."explain_map"( + "graph" TEXT, /* Option < & str > */ + "expr" TEXT, /* Option < & str > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS jsonb /* JsonB */ +STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'explain_map_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/constructors.rs:163 +-- pg_warren::compute_sql::constructors::triangles +CREATE FUNCTION warren."triangles"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'triangles_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/contract.rs:41 +-- pg_warren::community_sql::contract::contract +CREATE FUNCTION warren."contract"( + "graph" TEXT, /* Option < & str > */ + "label_vector" TEXT, /* Option < & str > */ + "out_graph" TEXT, /* Option < & str > */ + "weighted" bool DEFAULT true, /* Option < bool > */ + "drop_self_loops" bool DEFAULT false, /* Option < bool > */ + "promote" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'contract_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/similarity.rs:182 +-- pg_warren::compute_sql::similarity::most_similar + +CREATE FUNCTION warren.most_similar(graph text, node anyelement, k bigint DEFAULT 10, + measure text DEFAULT 'jaccard', rels text[] DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, score float8) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'most_similar_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/bfs.rs:37 +-- pg_warren::traversal_sql::bfs::bfs + +CREATE FUNCTION warren.bfs(graph text, start anyarray, max_depth int, + direction text DEFAULT 'out', rels text[] DEFAULT NULL, + node_filter "any" DEFAULT NULL, max_results bigint DEFAULT NULL, + version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, depth int, via anyelement) +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'bfs_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/summary.rs:89 +-- pg_warren::compute_sql::summary::vector_stats +CREATE FUNCTION warren."vector_stats"( + "graph" TEXT, /* Option < & str > */ + "name" TEXT, /* Option < & str > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS jsonb /* JsonB */ +STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_stats_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/local_moving.rs:31 +-- pg_warren::community_sql::local_moving::local_moving +CREATE FUNCTION warren."local_moving"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "initial" TEXT DEFAULT NULL, /* Option < & str > */ + "keys" TEXT DEFAULT NULL, /* Option < & str > */ + "resolution" double precision DEFAULT 1.0, /* Option < f64 > */ + "seed" bigint DEFAULT 0, /* Option < i64 > */ + "max_sweeps" INT DEFAULT 100, /* Option < i32 > */ + "tolerance" double precision DEFAULT 1e-7, /* Option < f64 > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'local_moving_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/vector/quota/sql.rs:97 +-- pg_warren::vector::quota::sql::vector_usage + +CREATE FUNCTION warren."vector_usage"(name text) +RETURNS TABLE (role regrole, vectors bigint, bytes bigint, quota bigint) +STRICT STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'vector_usage_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/read.rs:100 +-- pg_warren::compute_sql::read::matrix_rows + +CREATE FUNCTION warren.matrix_rows(graph text, name text, id_type anyelement, + version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, "row" float8[]) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'matrix_rows_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/neighbourhood.rs:83 +-- pg_warren::traversal_sql::neighbourhood::has_edge + +CREATE FUNCTION warren.has_edge(graph text, src anyelement, dst anyelement, + rel text DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS boolean +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'has_edge_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/extension.rs:91 +-- pg_warren::extension::version +CREATE FUNCTION warren."version"() RETURNS TEXT /* & '_ str */ +STRICT STABLE PARALLEL SAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'version_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/local_moving.rs:100 +-- pg_warren::community_sql::local_moving::leiden_refine +CREATE FUNCTION warren."leiden_refine"( + "graph" TEXT, /* Option < & str > */ + "communities" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "keys" TEXT DEFAULT NULL, /* Option < & str > */ + "resolution" double precision DEFAULT 1.0, /* Option < f64 > */ + "seed" bigint DEFAULT 0, /* Option < i64 > */ + "randomness" double precision DEFAULT 0.01, /* Option < f64 > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'leiden_refine_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/modularity.rs:26 +-- pg_warren::community_sql::modularity::modularity +CREATE FUNCTION warren."modularity"( + "graph" TEXT, /* Option < & str > */ + "labels" TEXT, /* Option < & str > */ + "resolution" double precision DEFAULT 1.0, /* Option < f64 > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS double precision /* Option < f64 > */ +STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'modularity_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/cache/warm.rs:18 +-- pg_warren::cache::warm::prewarm +CREATE FUNCTION warren."prewarm"( + "name" TEXT, /* & str */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* i64 */ +VOLATILE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'prewarm_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/activity/command.rs:164 +-- pg_warren::activity::command::report_progress +CREATE FUNCTION warren."report_progress"( + "command" TEXT, /* Option < & str > */ + "graph" TEXT, /* Option < & str > */ + "phase" TEXT, /* Option < & str > */ + "done" bigint DEFAULT NULL, /* Option < i64 > */ + "total" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS void +VOLATILE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'report_progress_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/cache/introspection.rs:282 +-- pg_warren::cache::introspection::cache +CREATE FUNCTION warren."cache"() RETURNS TABLE ( + "kind" TEXT, /* String */ + "graph" TEXT, /* Option < String > */ + "version" bigint, /* Option < i64 > */ + "vector" TEXT, /* Option < String > */ + "vector_id" bigint, /* Option < i64 > */ + "bytes" bigint, /* i64 */ + "state" TEXT, /* Option < String > */ + "last_use" bigint, /* Option < i64 > */ + "cap" bigint /* Option < i64 > */ +) +STRICT VOLATILE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'cache_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/spectral_sql/eigen.rs:93 +-- pg_warren::spectral_sql::eigen::eigen +CREATE FUNCTION warren."eigen"( + "graph" TEXT, /* Option < & str > */ + "k" INT, /* Option < i32 > */ + "matrix" TEXT DEFAULT 'adjacency', /* Option < & str > */ + "which" TEXT DEFAULT 'largest', /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "symmetrize" bool DEFAULT true, /* Option < bool > */ + "tol" double precision DEFAULT 1e-6, /* Option < f64 > */ + "max_iter" INT DEFAULT NULL, /* Option < i32 > */ + "vectors_prefix" TEXT DEFAULT NULL, /* Option < & str > */ + "version" bigint DEFAULT NULL, /* Option < i64 > */ + "dtype" TEXT DEFAULT 'f32', /* Option < & str > */ + "persist" bool DEFAULT true /* Option < bool > */ +) RETURNS TABLE ( + "i" INT, /* i32 */ + "eigenvalue" double precision, /* f64 */ + "residual" double precision /* f64 */ +) +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'eigen_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/create_graph.rs:42 +-- pg_warren::catalog::create_graph::create_graph +CREATE FUNCTION warren."create_graph"( + "name" TEXT, /* & str */ + "node_query" TEXT, /* & str */ + "edge_query" TEXT, /* & str */ + "directed" bool DEFAULT true, /* bool */ + "weighted" bool DEFAULT false, /* bool */ + "has_relations" bool DEFAULT false, /* bool */ + "build_reverse" bool DEFAULT true, /* bool */ + "retain_versions" INT DEFAULT 2, /* i32 */ + "store_merged" bool DEFAULT true, /* bool */ + "node_order" TEXT DEFAULT 'id' /* & str */ +) RETURNS void +STRICT +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'create_graph_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/step.rs:94 +-- pg_warren::traversal_sql::step::s + +CREATE FUNCTION warren.s(rels text[], direction text DEFAULT 'out', filter "any" DEFAULT NULL, + min_hops int DEFAULT 1, max_hops int DEFAULT NULL) +RETURNS warren.step +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 's_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/version_sql/compact.rs:26 +-- pg_warren::version_sql::compact::compact +CREATE FUNCTION warren."compact"( + "name" TEXT, /* Option < & str > */ + "node_order" TEXT DEFAULT NULL /* Option < & str > */ +) RETURNS bigint /* i64 */ + +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'compact_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/product.rs:210 +-- pg_warren::compute_sql::product::spmm +CREATE FUNCTION warren."spmm"( + "graph" TEXT, /* Option < & str > */ + "x" TEXT, /* Option < & str > */ + "y" TEXT, /* Option < & str > */ + "semiring" TEXT DEFAULT 'plus_times', /* Option < & str > */ + "transpose" bool DEFAULT false, /* Option < bool > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "mask" TEXT DEFAULT NULL, /* Option < & str > */ + "mask_complement" bool DEFAULT false, /* Option < bool > */ + "replace" bool DEFAULT false, /* Option < bool > */ + "accumulate" TEXT DEFAULT NULL, /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS void +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'spmm_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/centrality.rs:157 +-- pg_warren::compute_sql::centrality::harmonic_centrality +CREATE FUNCTION warren."harmonic_centrality"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "k" bigint DEFAULT NULL, /* Option < i64 > */ + "seed" bigint DEFAULT 0, /* Option < i64 > */ + "directed" bool DEFAULT true, /* Option < bool > */ + "sources" TEXT DEFAULT NULL, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'harmonic_centrality_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/info.rs:16 +-- pg_warren::catalog::info::info +CREATE FUNCTION warren."info"( + "name" TEXT /* & str */ +) RETURNS jsonb /* JsonB */ +STRICT STABLE PARALLEL SAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'info_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/dropped_roles/orphans.rs:70 +-- pg_warren::catalog::dropped_roles::orphans::drop_orphans +CREATE FUNCTION warren."drop_orphans"() RETURNS bigint /* i64 */ +STRICT +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'drop_orphans_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/reduce.rs:19 +-- pg_warren::compute_sql::reduce::vector_reduce +CREATE FUNCTION warren."vector_reduce"( + "graph" TEXT, /* Option < & str > */ + "x" TEXT, /* Option < & str > */ + "op" TEXT, /* Option < & str > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS double precision /* Option < f64 > */ +STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_reduce_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/walk.rs:38 +-- pg_warren::traversal_sql::walk::walk + +CREATE FUNCTION warren.walk(graph text, start anyarray, steps warren.step[], + mode text DEFAULT 'ends', distinct_ends boolean DEFAULT true, + max_results bigint DEFAULT NULL, reverse boolean DEFAULT false, + version bigint DEFAULT NULL) +RETURNS TABLE (end_id anyelement, path anyarray, rels text[], count bigint) +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'walk_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/traversal_sql/neighbourhood.rs:25 +-- pg_warren::traversal_sql::neighbourhood::neighbours + +CREATE FUNCTION warren.neighbours(graph text, node anyelement, direction text DEFAULT 'out', + rels text[] DEFAULT NULL, version bigint DEFAULT NULL) +RETURNS SETOF anyelement +STABLE PARALLEL SAFE LANGUAGE c AS 'MODULE_PATHNAME', 'neighbours_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/version_sql/diff.rs:23 +-- pg_warren::version_sql::diff::diff +CREATE FUNCTION warren."diff"( + "name" TEXT, /* & str */ + "v1" bigint, /* i64 */ + "v2" bigint /* i64 */ +) RETURNS jsonb /* JsonB */ +STRICT STABLE PARALLEL RESTRICTED +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'diff_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/from_query.rs:56 +-- pg_warren::compute_sql::from_query::vector_from_query +CREATE FUNCTION warren."vector_from_query"( + "graph" TEXT, /* Option < & str > */ + "name" TEXT, /* Option < & str > */ + "query" TEXT, /* Option < & str > */ + "dtype" TEXT DEFAULT 'f64', /* Option < & str > */ + "fill" double precision DEFAULT 0, /* Option < f64 > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'vector_from_query_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/dropped_roles/drop_owned.rs:16 +-- pg_warren::catalog::dropped_roles::drop_owned::drop_owned + +CREATE FUNCTION warren."drop_owned"(role regrole) RETURNS bigint +STRICT VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'drop_owned_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/owner/mod.rs:27 +-- pg_warren::catalog::owner::set_owner + +CREATE FUNCTION warren."set_owner"(name text, owner regrole) RETURNS void +STRICT VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'set_owner_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/centrality.rs:125 +-- pg_warren::compute_sql::centrality::closeness_centrality +CREATE FUNCTION warren."closeness_centrality"( + "graph" TEXT, /* Option < & str > */ + "out" TEXT, /* Option < & str > */ + "k" bigint DEFAULT NULL, /* Option < i64 > */ + "seed" bigint DEFAULT 0, /* Option < i64 > */ + "wf_improved" bool DEFAULT true, /* Option < bool > */ + "directed" bool DEFAULT true, /* Option < bool > */ + "sources" TEXT DEFAULT NULL, /* Option < & str > */ + "rels" TEXT[] DEFAULT NULL, /* :: std :: option :: Option < Vec < Option < String > > > */ + "weighted" bool DEFAULT NULL, /* Option < bool > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS bigint /* Option < i64 > */ +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'closeness_centrality_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/readers.rs:25 +-- pg_warren::catalog::readers::revoke + +CREATE FUNCTION warren."revoke"(name text, role regrole) RETURNS void +STRICT VOLATILE LANGUAGE c AS 'MODULE_PATHNAME', 'revoke_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/compute_sql/read.rs:64 +-- pg_warren::compute_sql::read::vector_values + +CREATE FUNCTION warren.vector_values(graph text, name text, id_type anyelement, + version bigint DEFAULT NULL) +RETURNS TABLE (node_id anyelement, value float8) +STABLE PARALLEL RESTRICTED LANGUAGE c AS 'MODULE_PATHNAME', 'vector_values_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/community_sql/projection.rs:60 +-- pg_warren::community_sql::projection::lift +CREATE FUNCTION warren."lift"( + "out_graph" TEXT, /* Option < & str > */ + "y" TEXT, /* Option < & str > */ + "graph" TEXT, /* Option < & str > */ + "x" TEXT, /* Option < & str > */ + "persist" bool DEFAULT true, /* Option < bool > */ + "version" bigint DEFAULT NULL /* Option < i64 > */ +) RETURNS void +VOLATILE PARALLEL UNSAFE +LANGUAGE c /* Rust */ +AS 'MODULE_PATHNAME', 'lift_wrapper'; +/* */ + +/* */ +-- crates/pg_warren/src/catalog/bootstrap.rs:8 +-- finalize + +CREATE TABLE warren.graphs ( + graph_id serial PRIMARY KEY, + name text UNIQUE NOT NULL, + owner regrole NOT NULL, + readers regrole[] NOT NULL DEFAULT '{}', + source text NOT NULL DEFAULT 'query' + CHECK (source IN ('query'::pg_catalog.text, 'contraction'::pg_catalog.text)), + node_query text, + edge_query text, + search_path text NOT NULL, + id_type regtype NOT NULL, + directed boolean NOT NULL DEFAULT true, + build_reverse boolean NOT NULL DEFAULT true, + weighted boolean NOT NULL DEFAULT false, + has_relations boolean NOT NULL DEFAULT false, + retain_versions int NOT NULL DEFAULT 2 CHECK (retain_versions >= 1), + current_version bigint, + store_merged boolean NOT NULL DEFAULT true, + node_order text NOT NULL DEFAULT 'id' + CHECK (node_order IN ('id'::pg_catalog.text, 'locality'::pg_catalog.text)), + reader_vector_quota bigint CHECK (reader_vector_quota >= 0) +); + +CREATE TABLE warren.versions ( + version_id bigserial PRIMARY KEY, + graph_id int NOT NULL REFERENCES warren.graphs ON DELETE CASCADE, + seq bigint NOT NULL, + staged boolean NOT NULL DEFAULT true, + built_at timestamptz NOT NULL, + built_lsn pg_lsn NOT NULL, + node_count bigint NOT NULL, + id_space bigint NOT NULL, + edge_count bigint NOT NULL, + dropped_edges bigint NOT NULL, + deduplicated_edges bigint NOT NULL, + target_width smallint NOT NULL CHECK (target_width IN (4, 8)), + compacted_from bigint REFERENCES warren.versions ON DELETE SET NULL, + bytes bigint NOT NULL, + label text, + parent_version bigint REFERENCES warren.versions ON DELETE SET NULL, + label_vector bigint, + UNIQUE (graph_id, seq) +); + +CREATE TABLE warren.segments ( + version_id bigint NOT NULL REFERENCES warren.versions ON DELETE CASCADE, + segment text NOT NULL, + chunk_no int NOT NULL, + data bytea NOT NULL, + PRIMARY KEY (version_id, segment, chunk_no) +); +ALTER TABLE warren.segments ALTER COLUMN data SET STORAGE EXTERNAL; + +CREATE TABLE warren.vectors ( + vector_id bigserial PRIMARY KEY, + graph_id int NOT NULL REFERENCES warren.graphs ON DELETE CASCADE, + version_id bigint NOT NULL REFERENCES warren.versions ON DELETE CASCADE, + name text NOT NULL, + dtype text NOT NULL, + width int NOT NULL DEFAULT 1, + length bigint NOT NULL, + created_at timestamptz NOT NULL DEFAULT now(), + created_by regrole NOT NULL, + UNIQUE (graph_id, version_id, name) +); + +CREATE TABLE warren.vector_chunks ( + vector_id bigint REFERENCES warren.vectors ON DELETE CASCADE, + chunk_no int, + data bytea NOT NULL, + PRIMARY KEY (vector_id, chunk_no) +); +-- IEEE floats rarely compress, so skip pglz's attempt on every write. +ALTER TABLE warren.vector_chunks ALTER COLUMN data SET STORAGE EXTERNAL; +-- A quota sums one role's vectors on one graph (DESIGN.md §5.7). +CREATE INDEX vectors_created_by ON warren.vectors (graph_id, created_by); + +-- Per-role quotas on readers' persisted vectors, overriding +-- graphs.reader_vector_quota (DESIGN.md §5.7, §14 item 46). +CREATE TABLE warren.vector_quotas ( + graph_id int NOT NULL REFERENCES warren.graphs ON DELETE CASCADE, + role regrole NOT NULL, + bytes bigint NOT NULL CHECK (bytes >= 0), + PRIMARY KEY (graph_id, role) +); + +-- graphs.current_version is not a foreign key: with versions.graph_id it +-- would make a cycle, and pg_dump cannot order the data of tables in a +-- foreign-key cycle, so a dump would not restore (DESIGN.md §5.1). These +-- triggers check what that key checked on updates and deletes, and more: +-- the current version is one of the graph's own versions, and cannot be +-- deleted while it is current. They are not equivalent to a key: inserts +-- are not checked (a restore only inserts; cache::open checks what it +-- reads), and they lock only the rows their queries lock. (versions. +-- label_vector has no foreign key for the same reason; a trigger below +-- clears it instead.) +CREATE FUNCTION warren.current_version_is_the_graphs() RETURNS trigger +LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +BEGIN + -- FOR KEY SHARE, as a foreign key's check locks: a concurrent delete of + -- the version waits for this transaction, then fails in its trigger. + IF NEW.current_version IS NOT NULL AND NOT EXISTS ( + SELECT FROM warren.versions v + WHERE v.version_id = NEW.current_version AND v.graph_id = NEW.graph_id + FOR KEY SHARE) THEN + RAISE EXCEPTION USING ERRCODE = 'foreign_key_violation', + MESSAGE = format('version %s is not a version of graph "%s"', NEW.current_version, NEW.name); + END IF; + RETURN NEW; +END $$; +CREATE TRIGGER current_version_is_the_graphs BEFORE UPDATE OF current_version ON warren.graphs + FOR EACH ROW EXECUTE FUNCTION warren.current_version_is_the_graphs(); + +CREATE FUNCTION warren.current_version_is_kept() RETURNS trigger +LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + current_id bigint; +BEGIN + -- FOR SHARE reads the graph's latest row, not the snapshot's: a + -- promotion committed after this transaction's snapshot fails the + -- delete (serialization_failure under REPEATABLE READ, 23503 under READ + -- COMMITTED), and one in progress makes it wait. Deleted with its + -- graph, the version finds no row. + SELECT g.current_version INTO current_id FROM warren.graphs g + WHERE g.graph_id = OLD.graph_id FOR SHARE; + IF current_id = OLD.version_id THEN + RAISE EXCEPTION USING ERRCODE = 'foreign_key_violation', + MESSAGE = format('version %s is the current version of its graph', OLD.version_id); + END IF; + RETURN OLD; +END $$; +CREATE TRIGGER current_version_is_kept BEFORE DELETE ON warren.versions + FOR EACH ROW EXECUTE FUNCTION warren.current_version_is_kept(); +-- versions.label_vector names the vector a contraction was contracted on +-- (DESIGN.md §6.6). It cannot be a foreign key: with vectors.version_id it +-- would make a cycle. When that vector is deleted, the versions contracted +-- on it forget it, so the link never dangles. +CREATE INDEX versions_label_vector ON warren.versions (label_vector) + WHERE label_vector IS NOT NULL; +-- Retention counts the versions contracted from each version, and an +-- ON DELETE SET NULL finds them: both look versions up by parent. +CREATE INDEX versions_parent_version ON warren.versions (parent_version) + WHERE parent_version IS NOT NULL; +CREATE FUNCTION warren.label_vector_is_forgotten() RETURNS trigger +LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$ +BEGIN + UPDATE warren.versions SET label_vector = NULL + WHERE label_vector OPERATOR(pg_catalog.=) OLD.vector_id; + RETURN OLD; +END $$; +CREATE TRIGGER label_vector_is_forgotten AFTER DELETE ON warren.vectors + FOR EACH ROW EXECUTE FUNCTION warren.label_vector_is_forgotten(); +-- Trigger functions cannot be called directly; nobody needs EXECUTE on them. +REVOKE EXECUTE ON FUNCTION warren.current_version_is_the_graphs(), warren.current_version_is_kept(), + warren.label_vector_is_forgotten() + FROM PUBLIC; + +-- Only warren.* functions touch these, as the catalog owner (§7). +REVOKE ALL ON ALL TABLES IN SCHEMA warren FROM PUBLIC; +REVOKE ALL ON ALL SEQUENCES IN SCHEMA warren FROM PUBLIC; +GRANT USAGE ON SCHEMA warren TO PUBLIC; + +SELECT pg_catalog.pg_extension_config_dump('warren.graphs', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.graphs_graph_id_seq', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.versions', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.versions_version_id_seq', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.segments', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.vectors', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.vectors_vector_id_seq', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.vector_chunks', ''); +SELECT pg_catalog.pg_extension_config_dump('warren.vector_quotas', ''); +/* */ + diff --git a/sql/released.commits b/sql/released.commits index 78eac1f8..6d0aec79 100644 --- a/sql/released.commits +++ b/sql/released.commits @@ -3,3 +3,4 @@ b711d515aeb3d201c11bf867c88825bc39e97d2d 0.2.0 0f1c1de03f4bc357db0117ef1e8d5e4b7d4eff4f 0.3.0 ee856402bde0f9602e05a791db2a1c23439fde4b 0.4.0 5f4176f4e4c8980acf751a342d8622fdb93838c0 0.5.0 +7d129002bbb621056cf72796bc63a2cad8e83241 0.6.0 diff --git a/sql/released.sha256 b/sql/released.sha256 index c27fd88c..c0e1e720 100644 --- a/sql/released.sha256 +++ b/sql/released.sha256 @@ -3,3 +3,4 @@ 5606cfc0b8059fbb1b1046bdb3d671a2fe7c7ad6451993dbbf91115ee28340a3 pg_warren--0.3.0.sql bd2be6fee5169c5566e250c3db4724380ad5f02c40d7c48be68f4610edce03e3 pg_warren--0.4.0.sql 3b4a6793ae6ae2126c5a1031f0f958083dbb3f01c3d6b5def9cc81eb20ca6b9c pg_warren--0.5.0.sql +3cf77ecc93e3e6f973eb3dc66c89125ec906329ec4b2f7d247cb48dc8094604f pg_warren--0.6.0.sql