Skip to content

Quotas on readers' vectors (M10 step 7) - #59

Merged
wfaithfull merged 8 commits into
mainfrom
m10-quotas
Oct 23, 2026
Merged

wfaithfull merged 8 commits into
mainfrom
m10-quotas

Conversation

@wfaithfull

@wfaithfull wfaithfull commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

M10 step 7, implementing DESIGN.md §5.7 "Quotas on readers' vectors" (§14 items 46, 47).

Schema

  • New column graphs.reader_vector_quota, with a CHECK (>= 0) constraint.
  • New table warren.vector_quotas:
    • registered with pg_extension_config_dump;
    • revoked from PUBLIC;
    • rows are deleted with their graph.
  • New index on warren.vectors (graph_id, created_by).
  • Both the 0.5.0→0.6.0 upgrade script and the 0.6.0 install script carry these changes.

Functions

  • warren.set_vector_quota(name, bytes, role DEFAULT NULL)
    • The graph's owner only. It takes the graph's row lock, then re-checks ownership.
    • A NULL role sets the graph's default quota. NULL bytes removes the quota.
    • Role '-' is refused with 22023. Setting a quota for an oid that names no role is refused with 42704; removing a dropped role's quota is allowed.
  • warren.vector_usage(name): the owner sees every role's usage; a reader sees only its own row.
  • info now includes reader_vector_quota.

Enforcement

Every persisted vector write goes through Vectors::save, which calls quota::admit first. That covers every vector-writing function and the library's results. The lock, the usage read and the decision live in catalog::quota.

  • Members of the owner role are exempt and take no lock.
  • Every other write proceeds in this order:
    1. Take a transaction-scoped, advisory-type lock on (graph, role), on the leader, with pg_warren's own objsubid.
    2. Read the quota.
    3. Sum the role's usage under the latest snapshot, including what this transaction has already written.
    4. Refuse with 53400 only if usage after the write would be over the quota and higher than before it.
  • So replacing a vector with one no larger always succeeds, and a quota lowered below current usage only blocks growth.
  • reassign_owned charges the vectors it moves to the new role's quota, on every graph where neither the new role nor the caller is exempt, and refuses with 53400 past it.

Scope

A quota bounds a role's vectors on another owner's graph. It does not bound total catalog storage: create_graph, and the quotient contract creates for its caller, both give the caller storage it is exempt on. Per-role storage overall is option 46c, which is deferred. This is documented in DESIGN §5.7, §14 item 46 and docs/semantics.md.

Dropped roles

  • drop_owned and the DROP OWNED event trigger delete a role's quota rows only:
    • on graphs whose owner the caller is a member of;
    • or everywhere, for the extension owner or a superuser.
  • So a reader cannot delete its own quota. Any rows left behind are listed by orphans() and removed by drop_orphans, or by the graph's owner.

Tests

  • Unit tests for the pure quota decision.
  • pg_tests:
    • a write over the quota is refused;
    • a replacement no larger than the old vector succeeds;
    • the owner is exempt;
    • a quota lowered below usage blocks only growth;
    • the latest snapshot sees the transaction's own writes;
    • reassign_owned past the new role's quota is refused, and a move to the owner is not counted;
    • drop_owned and DROP OWNED of oneself keep the role's quota row.
  • Harness vector_quota_race (dblink) covers:
    • a second writer that waits for the lock, then is refused;
    • a REPEATABLE READ writer;
    • a quota committed while another write of the same role is still uncommitted;
    • a reassign_owned queued behind a writer.
  • Coverage of quotas added to dump.py and upgrade.py.
  • A security review's exploit probes for the reassign_owned and drop_owned bypasses are now refused.

Run locally

  • pgrx tests: PG 15, 16 and 18 pass. PG 17 isn't set up on this machine; CI covers it.
  • run.py: 52 of 52 pass, plain and --trusted.
  • dump.py and upgrade.py pass.
  • fmt, clippy -D warnings and rustdoc are clean.

CLAUDE.md: the module list now includes vector::quota and catalog::quota, and the parallel-labels rule follows DESIGN §9.3.

🤖 Generated with Claude Code

https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke

graphs.reader_vector_quota, warren.vector_quotas (dumped with the
extension, revoked from PUBLIC) and an index on vectors (graph_id,
created_by); warren.set_vector_quota and warren.vector_usage; info
reports reader_vector_quota. Every persisted write (Vectors::save) is
admitted by vector::quota::admit: members of the owner role are exempt;
where a quota applies, it takes a transaction-scoped advisory-type lock
on (graph, role) with pg_warren's own objsubid, sums the role's usage
under the latest snapshot and refuses with 53400 only a write that takes
usage past the quota and above what it was (DESIGN.md §5.7, §14 items 46
and 47). The upgrade script from 0.5.0 adds the schema and functions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke
clean_up_after (drop_owned and the DROP OWNED trigger) deletes the
role's rows of warren.vector_quotas, orphans() lists rows naming dropped
roles as kind 'quota', and drop_orphans removes them in a fourth locked
pass, at the seams step 6 left (DESIGN.md §5.7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke
vector_quota_race: a second session of one role waits on the (graph,
role) lock and is refused, or admitted when both fit; a REPEATABLE READ
writer counts a write committed after its snapshot. dump.py dumps and
restores a default and a per-role quota; upgrade.py checks the update
leaves no quota and that one set afterwards refuses a reader's write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke
DESIGN.md §5.7 records what step 7 built and the decisions it made
(removing a role's quota, the order of a counted write, the lock,
vector_usage's rows), the catalog in §5.1, and the function reference and
parallel labels in §9.3; docs/semantics.md, CHANGELOG.md and README.md
describe the new functions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke
PR #59 security review:
- reassign_owned counted no vectors it moved, so a reader could reset its
  usage by moving its vectors to a group it is a member of. The move now
  takes the (graph, new role) lock and is refused (53400) past the new
  role's quota, unless it or the caller is exempt.
- drop_owned and the DROP OWNED trigger deleted all of a role's quota
  rows, so a role could shed its own quota. They now remove them only on
  graphs whose owner the caller is a member of (every one for a caller
  that may drop orphans); orphans() lists the rest once the role is gone.
- Every non-exempt persisted write takes the (graph, role) lock before it
  reads the quota, so a quota set while a write under none is uncommitted
  counts that write.
- set_vector_quota refuses '-' (22023) and a role oid that names no role
  (42704) instead of an internal error.
- The ledger (lock, quota, usage, decision) moves to catalog::quota, which
  vector::quota and reassign_owned share.
- DESIGN.md §5.7 and §14 items 44, 46, 47, docs/semantics.md and the
  CHANGELOG: the move counted, drop_owned's quotas, the 53400 a racing
  replacement may get, and the quota's scope (not a bound on total
  storage; a quotient inherits no quota).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke
@wfaithfull
wfaithfull merged commit 52c272c into main Oct 23, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants