[dr0uleau]

Summary

Buzz clients need to recover task changes after a disconnect. This adds incremental reads of changed artifacts with database modification timestamps and a 120-second overlap. Each result contains the current revision or an explicit removal. A separate read of known IDs confirms accepted edits and restores access state.

Pages use a timestamp and a stable ID to handle equal timestamps. Access grants include older records. Access policy changes require discovery and revalidation of saved IDs. The reader does not load a list of all readable channels. Ordinary artifact writes update their own rows.

Incremental reads and WebSocket Sync

This HTTP protocol maintains an eventually consistent local store. It uses artifacts.incremental and "artifact":"incremental". It does not prove a complete set at one database snapshot. Absence never proves deletion or loss of access.

WebSocket Sync serves a separate use case: a complete set at EOSE, followed by live updates. It owns artifacts.sync. Both protocols can coexist. The second PR to merge must reuse the shared access helper in artifact_query.rs when it resolves the overlapping edits.

The gate is BUZZ_ARTIFACT_INCREMENTAL_ENABLED. Errors use the incremental_ prefix. Unlisted open channels require membership or an explicit #h scope. Listing changes reset incremental progress. Incremental reads exclude TTL artifacts. Known-ID reads return remove for them. The #h scope limits candidates before pagination. Initial reads omit deleted heads. Later reads retain deletion removals.

Why the new timestamps are needed

created_at comes from the event signer. received_at records event arrival. Neither advances when access changes or an event is redacted. Sync needs database timestamps for those changes:

Column	Why sync needs it
artifact_heads.modified_at	Finds current records that change through an edit, deletion, or redaction.
artifact_revisions.modified_at	Dates retained move records. This lets sync remove a task from its former channel when the reader cannot access its new channel.
channel_members.sync_modified_at	Finds grants, rejoins, and removals. A grant must include older tasks whose own timestamps do not change.
communities.artifact_policy_modified_at	Requests recovery after a relevant channel visibility, TTL, or deletion change. Clients revalidate saved IDs. Ordinary task writes do not update it.

The database sets these timestamps with clock_timestamp(). The 120-second overlap covers the accepted delay between timestamp assignment and commit visibility.

BUZZ_ARTIFACT_INCREMENTAL_ENABLED defaults to off. Apply migrations 0076–0078 and upgrade all writers before enablement. The 120-second delay bound is an operating assumption. Deployment steps cover index prebuilds and rollback.

Related issue

No duplicate found. Related work: read permissions, statement limits, and atomic channel settings.

Testing

Signed HTTP E2E passes on f3d6ac852e7ce4e2cc53bd891cef87832a868d72 after CI passes. It includes main through ef4130da4. A standalone local relay uses a fresh PostgreSQL 17 database, Redis, and signed NIP-98 HTTP requests. All 78 migrations apply through relay startup. All behavioral writes and reads use HTTP.

The tests cover create, edit, delta reads, known-ID recovery, private moves, access grants, removals, rejoins, visibility resets, redaction, and deletion. Unlisted tests cover hidden broad discovery, named scope, known-ID reads, member access, and policy resets. They also cover bootstrap across two pages, overlap replay, and 101 move removals across pages. Decoded cursors contain no private revision IDs. With the incremental gate off, artifacts.incremental is absent and a signed incremental request returns 503 incremental_unavailable. Unsigned requests are rejected. Signed tests also cover TTL exclusion, current-ID removals for TTL artifacts, moves into TTL channels, deleted initial heads, and scoped pages with exactly 100 and 3 upserts.

A separate local scale test on babcdb766 reads 10,050 tasks across 60,002 memberships. It uses seeded fixtures and a second relay state with the same signing key. The initial read takes 8.31 seconds over 101 pages with 4,923-byte events. A read with no changes takes 5.70 milliseconds. The candidate keeps the same database query. These are local measurements, not production limits. Cold pages repeat candidate scans and access checks. Writer load needs validation before fleet enablement.

cargo test -p buzz-relay sync_downloads_ten_thousand_tasks_with_sixty_thousand_memberships -- --ignored --nocapture

The behavioral run uses cargo build -p buzz-relay --bin buzz-relay, then cargo run -p buzz-relay --example artifact_incremental_final_oct30_f3. The temporary harness drives the built relay and is removed after the run. The worktree is clean at the tested SHA. It does not exercise a native client. Git and S3 behavior are outside this validation.

The protocol defines progress, removals, limits, and recovery. Relay verification covers the signed HTTP path. Native client verification belongs to the client PR.

Generated with Codex

[github-actions]

🔐 Codex Security Review

Status: review required for the current range.

The current range is d263e2c7f7126b9a5caf1507140bb32e2c4e0ea3...f3d6ac852e7ce4e2cc53bd891cef87832a868d72.
A new review must complete for this exact range. When manual authorization
is required, a user with write access must comment exactly
@buzz-security-review f3d6ac852e7ce4e2cc53bd891cef87832a868d72 to authorize a new review.
Any previous review applies only to its recorded range.

[wesbillman]

🤖 Star Lord’s automated source review via Wes’s account.

One P2 correctness finding is inline: a concurrent first artifact can escape the conditional policy reset when its channel becomes private or is deleted. No code changes were applied.

Reviewed head 6a52195bd9a387e4871e3689b5c942ae42a5d597 against base/merge-base 2f2f6e2774fb0899d2a927f5e5b1987603765f31, with independent Rocket (systems/database), Mantis (correctness/security/concurrency), and Groot (protocol/user flows) lanes plus my integration.

Source-only review; no tests, builds, SQL or app execution. A single CI snapshot showed 50 successful, 28 skipped and 1 in-progress checks. Production query costs, migrations/rollback, client recovery and the documented 120-second stamp-to-visibility assumption remain unverified. Matching TTL artifacts intentionally return 422; clients must account for that permanent-channel-only scope.

Optional documentation clarification: spell out applying 0068 after its index prebuild, then 0069, before building the 0070 indexes; the current runbook jumps from the 0068 prebuild to 0069. Public PR description inspected; no attached images or actionable disclosure found.

This is a non-blocking COMMENT review, not approval or merge authorization.

[wesbillman]

🤖 Star Lord’s automated source follow-up via Wes’s account.

The prior P2 is fixed: actual visibility/TTL changes and successful channel deletion now advance the policy stamp even when the channel has no artifacts. The first-artifact interleaving test covers both transitions, and the deployment instructions now include 0068 application and migration-ledger recording.

One P2 remains inline, independently confirmed from the new security-review evidence: a continuation can expose an inaccessible destination revision ID. This is separate from the repaired policy-stamp race.

Reviewed head 357da40b3941efd083ea053fab56dbef3cce619c against base/merge-base 347ce88d5dd8bc631c073f1947fea6b75d5a43c0, with independent Rocket (systems/database), Mantis (correctness/security), and Groot (protocol/user flows) lanes plus my integration. Range-diff preserves all six previously reviewed feature patches; this pass covers the follow-up changes and the newly evidenced cursor issue.

Source only: no tests, builds, SQL or apps executed. One CI snapshot showed 29 successful, 21 skipped and 5 in-progress checks. Runtime interleavings, rollout/rollback, reset costs, client recovery and the 120-second visibility assumption remain unverified. Public description inspected; no attached images or actionable disclosure there. Non-blocking COMMENT review; no approval or merge authorization.

[wesbillman]

🤖 Star Lord’s automated source follow-up via Wes’s account.

The prior P2 cursor disclosure is fixed. Move-source continuation positions now contain artifact IDs instead of inaccessible destination revision IDs; DISTINCT preserves unique equal-timestamp positions. The current-ID read retains the caller’s query predicates and intersects them with the requested artifact IDs.

No new actionable findings in the follow-up from 357da40b3941efd083ea053fab56dbef3cce619c to reviewed head 4019c1df8d266701de685f9c714f747c1781967a, against base/merge-base 347ce88d5dd8bc631c073f1947fea6b75d5a43c0. Rocket, Mantis, and Groot independently reviewed systems/pagination, correctness/security, and product/protocol behavior; Star Lord integrated the results. The public description and changed commit/source material had no actionable disclosure finding; the description contains no attached images.

Source only: no tests, SQL, builds, or app workflows were executed. The new tests exercise the production HTTP router with fixture identity headers and direct database ingestion; this review does not establish signed-HTTP authentication coverage. CI snapshot: 43 successful checks, 28 skipped, 3 still running. Query-plan cost of DISTINCT, production latency/concurrency, client recovery and native accessibility/focus behavior remain unverified. The documented disabled-by-default rollout gate and 120-second overlap assumption remain applicable. This COMMENT is not approval or merge authorization.

[ngoodger-block]

🤖 Reviewed the sync core (timestamps, paging, move/grant/removal paths, lock ordering) and it looks sound; two issues inline. One operational risk: every channel delete (unconditionally) or visibility/unlisted/TTL change resets every client in the community to a full re-bootstrap, and with the per-process Semaphore(4) + try_acquire the resulting burst mostly gets a 503 with no Retry-After. Worth adding client backoff or a Retry-After before fleet enablement.

[ngoodger-block]

🤖 Both inline comments are fixed in f3d6ac8 (TTL channels are excluded from candidates; #h now limits candidates before pagination, and the initial read skips deleted heads). Two non-blocking notes: on populated databases, 0076/0078 will fail unless the indexes are built concurrently before deploy (relevant if BUZZ_AUTO_MIGRATE is on), and before turning on BUZZ_ARTIFACT_INCREMENTAL_ENABLED across the fleet, please add jitter to the client's fixed 60s capacity backoff (or have the relay send Retry-After) so community-wide resets don't retry in lockstep against Semaphore(4).