Skip to content

fix(relay): deny media reads outside the channels a file is shared in - #8444

Open
ngoodger-block wants to merge 68 commits into
block:mainfrom
ngoodger-block:fix/channel-scoped-media
Open

ngoodger-block wants to merge 68 commits into
block:mainfrom
ngoodger-block:fix/channel-scoped-media

Conversation

@ngoodger-block

@ngoodger-block ngoodger-block commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #8451 (records where media is shared and backfills it). Until it merges, this diff includes it — review the commits after the merge of feat/media-shares. The default read mode is log-only; decide whether to accept persistent backfill grants before the first backfill, then repair gaps before enforce.

Why?

GET/HEAD /media only checks community membership, so anyone in a community who learns an attachment's hash — which is in every media URL — can download a file posted in a private channel or DM they cannot read.

What?

BUZZ_MEDIA_READ_MODE controls file access independently of authentication and community checks: off skips the new check, log-only (default) counts outcomes without denying reads, and enforce applies it. The setting loads at startup, so mode changes need a rolling restart but no code rebuild. Unknown values fail startup.

In enforce mode, GET and HEAD require a live share or uploader preview permission. A private file is readable only through an accessible channel; deleted or purged source messages grant nothing. Edit grants also require the original message to remain live. Author deletion and moderator removal revoke that message’s grants, including retained and edit-added attachments. Other undeleted shares keep working. Unauthorized reads return 404; lookup failures return 503 and are logged.

Community owners and admins who can view the moderation queue may also review a file attached to a live message with an open report. Each exceptional GET or HEAD requires a durable audit entry before storage access. Closing the report, deleting the logical message, or removing the queue role ends that exception; edit attachments follow the same rule. Report notes, unrelated hashes, blob-only reports, and other communities grant no access; reports create no public shares. Disabled or failed auditing blocks the exception.

The elected usage-poller leader reclaims inactive grants in bounded pages. It removes obsolete public profile/emoji grants and compacts inactive channel history to at most one row per community and file. That last row preserves the uploader-preview restriction after message deletion; it grants no read access. Cleanup uses community admission and skips quiescing/deleting communities.

Rollout

Before the first backfill, explicitly accept grants from reconsidered references or defer until refusal-preserving behavior exists. Then deploy recording on every pod, run buzz-admin backfill-media-shares, and inspect buzz_media_read_access_total{mode="log-only",outcome="deny"} and outcome="error" before enforcing. The backfill now restores historical uploader proof from valid retained upload audits; missing evidence and unparsed references still need repair before enforcement. See operator instructions for mode changes, counters, and rollback.

Any backfill run, including the first, can grant previously refused references after their authors gain access. Those grants persist across read-mode changes and later prevention of reevaluation. Every invocation restarts the scan; unwanted existing grants need a provenance-based repair before enforcement.

The conformance table and NIP-FI captured-header note now describe this policy. Share creation already requires upload proof or a readable share in #8451.

Related issue

Closes the known limitation noted in conformance_multitenant.rs media_blossom. Read-enforcement half of the file-access stack after #8451.

Testing

On an isolated relay backed by PostgreSQL 17, Redis, and a disposable S3 HTTP fixture, ran RELAY_HTTP_URL=http://localhost:13071 cargo test -p buzz-test-client --test e2e_media_extended --test e2e_media_review -- --ignored --nocapture --test-threads=1 with BUZZ_MEDIA_READ_MODE=enforce: all 29 tests passed.

Repeated test_media_read_rollout_policy with the mode unset and with off: both passed, and Prometheus recorded two would-deny reads in the default log-only mode. The report journey submits a real NIP-56 report, verifies moderator GET/HEAD and their audit entries, rejects an unrelated attachment and another member, then confirms report closure revokes access. It also covers reported edit attachments and deletion of the original message while the report remains open. Removing the open-status SQL condition made the regression fail; restoring it passed.

The actual Desktop edit/upload/profile workflow smoke test remains pending: an isolated native demo build started, but UI automation returned cgWindowNotFound before the edit flow could be exercised.

All 29 live media journeys passed again with a five-second usage-poller interval. Deletion and profile journeys waited for periodic cleanup: old public grants disappeared, and deleted channel attachments remained 404 after compaction. The private-channel journey added then removed a member using real signed membership events, verifying GET and HEAD changed from 200 to 404 while the owner retained access. Removing the membership tombstone check made that regression fail. Removing the live-source check or deleting the final channel boundary also failed authorization regressions.

The fresh security workflow failed before analysis because checkout refused fork code in a privileged pull_request_target job; the protection remains enabled, and this run provides no security review result.

Generated with Codex

Linear: BUZZ-358

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

🔐 Codex Security Review

Status: review required for the current range.

The current range is dc25198d530acc6237a0320b697365f29daf0ea6...e25d56f182a6f87df5334999c2fcb107275a202e.
A new review must complete for this exact range. When manual authorization
is required, a user with write access must comment exactly
@buzz-security-review e25d56f182a6f87df5334999c2fcb107275a202e to authorize a new review.
Any previous review applies only to its recorded range.

Add `media_shares` and `media_uploaders` (migration 0067) and start filling
them, without changing who can read media yet. Each local `/media/<sha256>`
path in a stored event shares that blob into the event's channel (or with the
whole community when channel-less), in the same transaction as the event,
when its author can read the blob or uploaded those bytes; each upload records
its uploader under the serving-write lease. `buzz-admin backfill-media-shares`
records shares for events stored before this, so channel-scoped reads can be
enforced next without hiding existing media.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Clients upload a file before attaching it, so record the uploader first and
keep the test valid once reads deny files not yet shared into any channel.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
@ngoodger-block ngoodger-block changed the title fix(relay): scope media reads to the channels a blob is attached in fix(relay): deny media reads outside the channels a file is shared in Oct 19, 2026
@ngoodger-block
ngoodger-block marked this pull request as ready for review October 19, 2026 11:32
@ngoodger-block
ngoodger-block requested a review from a team as a code owner October 19, 2026 11:32

@wpfleger96 wpfleger96 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 thanks for this. the GET/HEAD check itself looks right: both run it before touching storage, thumbnails are checked against the original hash, and a refused read gets the same 404 as a missing file. one blocking issue inline (agent avatars uploaded by the owner end up readable only by the owner), plus a few smaller items.

two things about the stack as a whole:

  1. tightening the no-shares arm of readable_sql! is what closes the "anyone with the hash can share an unshared file" hole from the #8451 review, since record_media_shares uses the same macro. landing it here means it reaches prod after #8451 and the backfill, so the hole stays open that whole time. could the predicate change move into #8451, leaving this PR as just the read enforcement? it also can't revoke public grants created while only #8451 is deployed, and the backfill replay issue from that review is the same code here.
  2. the PostgreSQL lane is red at this head, 1051 of 1052: migration_schema_fence_objects_are_optional_for_serving_and_deletion fails with community deletion catalog drift (missing=media_shares,media_uploaders). the fixture migrates through 66 (deletion.rs:4402) but EXPECTED_SCOPED_TABLES now includes the 0067 tables, so bumping it to 67 should fix it. the cause is in #8451, so it fails there too.

a few questions on rollout:

  • there's no off switch or log-only mode, so at deploy any file the backfill missed (unparsed references, files uploaded before #8451 with no uploader row, the avatar case inline) 404s for everyone but its uploader, and the only way back is a redeploy. NIP-FI already has off / log-only / enforce. is the same worth doing here, so the "would deny" counts can be checked before enforcing?
  • NIP-56 reports only land in moderation_reports, never events, so a report never creates a share. a mod who isn't in the private channel now gets a 404 on the reported file. should mods reviewing an open report be able to read it?
  • the media row in docs/multi-tenant-conformance.md still lists the channel-ACL read gate as a remaining gap, and the NIP-FI read-proof note still says a captured header reads any media blob. is updating those meant to be part of this?

Comment thread crates/buzz-relay/src/api/media.rs Resolved
@wpfleger96

Copy link
Copy Markdown
Member

@buzz-security-review ffaa6fb

ngoodger-block and others added 5 commits October 20, 2026 08:32
Trust references only from events received before migration 0067 was
installed; later events get the live, checked recording, so a backfill run
after rollout never shares a blob that live recording refused. Insert shares
in (hash, channel) order from both the backfill and live recording so the two
cannot deadlock, and read media paths in JSON content that escapes `/`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Require upload proof or a recorded share when recording new references. Let an agent publish only its profile picture using its registered owner's upload proof. Keep the historical backfill cutoff on communities for both schema setup paths, and advance the deletion fixture through migration 67.

Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
@ngoodger-block

ngoodger-block commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 Added off/log-only/enforce with a log-only default, audited owner/admin access limited to open reports’ live attachments, and both documentation corrections in dd02e04, with passing runtime regressions and Desktop smoke testing still pending.

@wpfleger96 wpfleger96 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 thanks for working through all of this. the earlier feedback is addressed: owner-uploaded agent avatars now get a share and have a cross-signer E2E test, the strict share rule lives in #8451, the migration-67 fixture is fixed, the off / log-only / enforce switch has counters and rollback docs, moderators can review attachments on open reports with audited reads, lookup failures are logged, and the conformance table and NIP-FI note are updated. the two deferrals (recovering past uploaders from audit_log, the removed-member regression) are fine as follow-ups.

two things before this can be approved:

  1. this branch contains #8451 unchanged, so it also carries the backfill blocker from that review: post-cutoff events are re-checked with current permissions, so a refused reference can become a community-wide share once its author gains access. once that fix lands in #8451 and is merged here, this clears.
  2. one wording fix in the NIP-FI note (inline).

Comment thread docs/nips/NIP-FI.md Resolved
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source review via Wes’s account, integrating independent Rocket (systems/lifecycle), Mantis (correctness/security), and Groot (product/user-flow) reviews.

Reviewed head dd02e04fb7bff7dc5ec533cd03bd623005d48415 against base 6e6173acd58029bac800583c4cc7482dafed693a, including the stacked #8451 changes. Two P2 correctness findings are inline. The backfill finding also confirms the existing review discussion; the edited-message deletion case is additional.

[P3] Public PR material: remove the internal Linear tracker link from “Related issue” and replace it with a public issue or a self-contained description. This public repository currently exposes an internal workspace/issue identifier there. No credential exposure is claimed, and the internal issue contents were not accessed.

Optional documentation precision: in docs/nips/NIP-FI.md:722–726, describe the file policy in protocol terms and keep deployment-mode names in the operator guide. Qualify “upload proof” as preview permission only while no channel share exists.

Validation: source only; no tests, builds, SQL, or apps executed. One CI snapshot showed 65 successful checks and 31 skipped; that does not validate the reported journeys. Desktop enforce-mode edit/upload/profile behavior, recovery/focus after media 404/503 responses, rolling-deploy provenance recovery, and writer-pool performance remain unverified. The PR description has no attached images. Minimalism/elegance: 9/10; correctness: 7/10 pending the two inline repairs. This is a non-blocking COMMENT review, not approval or merge authorization.

@ngoodger-block

ngoodger-block commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 @wpfleger96 Please re-review the behavior-only NIP-FI wording and inherited backfill clarification at a8b3dd55e7e4082587f078b8e4abd5f90af74166; enforcement still requires an explicit decision on replaying earlier refusals. The stack is refreshed from main, and focused media, database, membership-write, and event-insert checks passed. The fresh security workflow stopped before analysis because actions/checkout refuses fork code in a pull_request_target run; a maintainer needs to address the workflow before the security review can complete.

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source review via Wes’s account, integrating independent Rocket (systems/lifecycle), Mantis (correctness/security/concurrency), and Groot (product/operator flows) reviews.

Reviewed head a8b3dd55e7e4082587f078b8e4abd5f90af74166, base 7e6461242aa7d98f132919dd672f7cc5f4abbba6. This follow-up covers the three new documentation/test commits and membership-writer integration since dd02e04fb7bff7dc5ec533cd03bd623005d48415; the preceding 24 feature patches are unchanged.

One new P2 rollout-documentation finding is inline. No new defect found in the adapted fixture calls: membership changes commit before the database-backed authorization check, so discarding the returned snapshot is harmless in this test. The NIP-FI wording now describes protocol policy separately from rollout configuration.

The existing lifecycle/backfill and public-description observations remain in the previous review; the documentation changes do not repair those code paths, and they are not duplicated here.

Source-only review; no code, SQL, tests, builds, apps, or deployments executed. Public description inspected; no attached images. Linked internal issue content was not inspected. One CI snapshot showed 20 successful, 20 skipped, 18 in-progress and 1 failed check; failure cause and runtime correctness are unverified. The description’s test results were not independently reproduced or established for this exact head. Desktop edit/upload/profile workflow, database concurrency, rollout behavior, and error/retry focus remain unverified. This COMMENT is not approval or merge authorization.

Comment thread docs/operator-media-access.md Resolved
ngoodger-block and others added 6 commits October 22, 2026 20:11
Add `media_shares` and `media_uploaders` (migration 0067) and start filling
them, without changing who can read media yet. Each local `/media/<sha256>`
path in a stored event shares that blob into the event's channel (or with the
whole community when channel-less), in the same transaction as the event,
when its author can read the blob or uploaded those bytes; each upload records
its uploader under the serving-write lease. `buzz-admin backfill-media-shares`
records shares for events stored before this, so channel-scoped reads can be
enforced next without hiding existing media.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Clients upload a file before attaching it, so record the uploader first and
keep the test valid once reads deny files not yet shared into any channel.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Trust references only from events received before migration 0067 was
installed; later events get the live, checked recording, so a backfill run
after rollout never shares a blob that live recording refused. Insert shares
in (hash, channel) order from both the backfill and live recording so the two
cannot deadlock, and read media paths in JSON content that escapes `/`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Require upload proof or a recorded share when recording new references. Let an agent publish only its profile picture using its registered owner's upload proof. Keep the historical backfill cutoff on communities for both schema setup paths, and advance the deletion fixture through migration 67.

Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
@ngoodger-block

ngoodger-block commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 @wpfleger96 please re-review this updated stack at 2cf0a83: edit grants follow the original message lifetime, legacy profile/emoji replacements recover eligible references, the rollout decision precedes the first backfill, and the media migration now has a tested five-second lock budget; #8535 is the schema prerequisite, all 29 real relay media journeys passed, and the actual Desktop UI smoke test remains pending.

The security workflow has failed before analysis because checkout blocks fork PR code under pull_request_target; this is not a security pass.

Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
* feat/media-shares:
  fix(db): bound the media migration lock wait

Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source review via Wes’s account, integrating independent Rocket (systems/lifecycle), Mantis (correctness/security/concurrency), and Groot (product/user-flow) reviews.

Reviewed head 7b8bee1e07f43ff4b7653a480865766197ce1007 against base 347ce88d5dd8bc631c073f1947fea6b75d5a43c0, focusing on the follow-up since b267129ff7d64e87b23cda3a89977e8eeadb4a3e and its supported callers.

The prior edit-lifetime defect is repaired in source: ordinary and report-based reads require the edit’s original message to remain live, while independent live shares survive. Legacy profile/emoji replacement recovers eligible pre-cutoff references before retiring the old head within the admitted transaction. The schema prerequisite removes blanket bootstrap fencing and the new media tables’ FKs/fences consistently, retaining application admission and purge inventory. The rollout now requires the accept/defer decision before the first backfill, and the internal tracker link has been removed from the public description. No new defect found in these follow-up repairs.

Retained P2: bound migration 0068’s lock wait. The existing finding on stacked #8451 still applies here: migrations/0068_media_access.sql:2–3 requests ACCESS EXCLUSIVE on communities with no local timeout, while runtime/migration.rs disables session lock/statement timeouts. A conflicting transaction can leave the migration and subsequent community operations waiting indefinitely. Add a short SET LOCAL lock_timeout before the ALTER. This is carried code, independently confirmed in this review; the existing inline is linked rather than duplicated.

Backfill still reevaluates earlier refusals using current permissions. This remains an explicitly documented retained limitation, not a repaired code path; the revised instructions correctly require accepting the persistent resulting grants before any run or deferring backfill. Changing read modes does not revoke those grants.

Validation: source only; no tests, SQL, builds, apps, backfill, or deployments executed. Public description/discussion inspected; no attached images. The described 29-test run was not independently reproduced or bound to this exact head. One CI snapshot showed 46 successful, 29 skipped, 8 running, and 1 failed security-review check; failure cause was not independently verified. Runtime revocation/concurrency, query cost, rollout behavior, and native edit/upload/profile recovery and focus after 404/503 remain unverified; the Desktop smoke test is still pending. Minimalism/elegance 9/10; correctness 8/10 pending the retained lock bound. Non-blocking COMMENT, not approval or merge authorization.

Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
* feat/media-shares:
  fix(media): reclaim inactive grants and cover upload attribution

Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
* feat/media-shares:
  fix(media): run cleanup before telemetry collection

Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
* feat/media-shares:
  fix(media): finish deferred provenance and access feedback

Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
@ngoodger-block

ngoodger-block commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 @wpfleger96 please re-review 6483588: the deferred feedback is now implemented, including uploader-handler coverage, inactive-grant cleanup, historical uploader recovery, private-kind exclusions, accurate error logging, positive batch sizes, and removed-member access coverage; all 29 live media journeys and the targeted mutation checks passed.

The author’s accepted backfill replay limitation remains documented; its conversations were closed as an accepted limitation, not a code fix. The actual Desktop UI smoke test remains pending. The fresh security workflow failed at the fork-checkout guard before analysis; the protection remains enabled, so there is no new security review result.

ngoodger-block and others added 2 commits October 23, 2026 13:41
The optional-fence test assumed every scoped table is fenced, so the first
tenant tables added under docs/database.md (scoped but admitted only in the
app) broke it. Assert that fences exist exactly when the fence objects do and
cover only scoped tables.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
* feat/media-shares:
  test(db): expect only legacy tables to carry community write fences

Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
ngoodger-block added a commit that referenced this pull request Oct 27, 2026
…trap (#8620)

### Why?

This is a cleanup that brings the schema bootstrap in line with how the
codebase works today, as asked for by
[`docs/database.md`](https://github.com/block/buzz/blob/dc25198d530acc6237a0320b697365f29daf0ea6/docs/database.md#L103-L124)
(#8273). Writes are now blocked during community deletion by application
admission (`begin_community_write_transaction`), and that doc's Rule 1
says new tables must not add triggers. But
`schema/community_write_fence.sql` still looped over every table with a
`community_id` column and attached the old fence trigger to it. That
meant a new table got a trigger on a fresh database and none on a
migrated one, so the two schema paths disagreed and new tenant tables
were blocked. The doc says this bootstrap change must land "before the
feature, or as the feature's first PR."

### What?

Deletes the blanket fence loop and its `\i` from `schema/schema.sql`.
Existing legacy tables keep their fences through their explicit
`schema/triggers/` files, so no current table loses its fence and
runtime behavior is unchanged. Adds a test that a new `community_id`
table stays unfenced while `events` and `users` stay fenced, and updates
`docs/database.md` and `ARCHITECTURE.md` to match. It stands on its own;
the media-sharing PRs (#8451, #8444) are the first to rely on it.

buzz-review-completed

---------

Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
wesbillman pushed a commit that referenced this pull request Nov 2, 2026
…ests (#8740)

## Summary

Two buzz-relay test fixes from the 23–30 Oct flake audit. Both are
test-only. This is one of six PRs split from the reviewed `flakey-tests`
branch (siblings below).

### `thread_window_bridge_restarts_both_aux_hops_after_replica_failure`
(3 runs)

- **Runs:** 40799617319, 40924179886 and 40930100994 (#8697, docs only),
all on 10-29 in PostgreSQL Tests, attempt 1 (jobs 123208043033,
123632125866, 123651641392). None of the branches touches thread_window,
and each run passed on a re-run. Signature:
`failure_postgres_tests.rs:46`, "barrier must precede closure
completion", left: 2, right: 1.
- **Cause:** `pause_after_aux_page` polls the request until a
`thread_window_aux` span closes, then expects exactly one. The fixture's
1,001 reactions make two replica pages at `AUX_LIMIT` 1000, and each
page is one round trip on the transaction already held. A loaded runner
can deschedule the test thread after page 2's query is written. Postgres
then answers first, and the same poll reads page 2 too.
- **Change:** `test_support` gains a `PausingProxy`, a trimmed copy of
buzz-db's private one with only the hold-everything switch. The replica
reader pool connects through it. `AuxPages` pauses it when the first aux
span closes. That runs inside the request's poll, after page 1's
response is consumed and before page 2's query is written, so page 2
stays pending however fast Postgres answers. The test resumes the proxy
after it terminates the reader's backend. The held page then sees the
termination, and the closure restarts on the writer, as before. The
other three `pause_after_aux_page` callers read from the writer and pass
`None`.

### `api::media` tests that need Postgres (fail every time Flake Check
selects them)

- **Run:** 40954304982 (#8658), 10-29, Flake check (Rust), job
123730434146. All six `test_state()` tests panic at the community-seed
`expect` after the 30 s sqlx acquire timeout.
- **Cause:** when a hunk lands outside a test body, Flake Check selects
every test in the file. It skips only `postgres_tests::`, `postgres_*`
binaries and `external_infra`, and its job has no database. These six
tests seed `relay.example` through Postgres but sat in plain `mod tests`
without `#[ignore]`. So any PR that changes non-test code in `media.rs`
fails that lane, and at least 8 open PRs touch the file. No other lane
ran these tests either.
- **Change:** move `test_state`, the `media_get_*` helpers,
`media_request` and the six tests into `tests::postgres_tests`. Mark
each test `#[ignore = "requires Postgres"]`, and point `test_state` at
`test_support::database_url()`, as the module's other states do. The
PostgreSQL lane now runs them, and Flake Check skips them. The change
also drops imports `mod tests` no longer uses, and updates the Justfile
comment that gave `api::media` as the reason the infra-free lane is
scoped to `api::admin`.
- When `media.rs` is staged, the pre-commit secret scan reads the whole
file and flags the module's existing public ES256 test key
(`TEST_EC_PKCS8_PEM`). The key is now written on one line with a sadscan
marker, as `invites.rs` already does. Its bytes are unchanged.

### Related issue

None found. **Conflicts with #8451 and #8444**, which both already
conflict with `main`. The one-line `TEST_EC_PKCS8_PEM` rewrite is the
only new conflict hunk with each. The test move merges cleanly with
both.

### Testing

Verified locally:
- Each of the four `pause_after_aux_page` tests passes when run alone
against local Postgres through `scripts/postgres-test-run.sh`.
- A forced check, not committed: after the barrier, the test gave page 2
another 500 ms over ten re-polls and asserted the page count stayed at
1. It passed with the pause and failed with `None`, with the CI
signature (left: 2, right: 1). A true same-poll burst needs the OS to
preempt between sqlx's write and read, so it was not forced directly.
- `check-postgres-test-discovery.py` accepts the touched files and all
621 files under `crates/`, and `scripts/test-postgres-test-discovery.sh`
passes. Each of the six media tests passes when run alone against local
Postgres.
- `cargo fmt --check` and `cargo clippy -p buzz-relay --all-targets --
-D warnings` are clean under 1.98.1.

Only hosted CI can show the full lanes, which were not run locally. The
six media tests have never run in CI, so this PR's PostgreSQL lane is
their first hosted run.

Pushed with `git push --no-verify`, because the pre-push suites fail
when the laptop is under load.

### Readiness

An agent review of the combined `flakey-tests` branch (b432453af) found
no blockers. `buzz-review-completed` is not added, because the human
test step (AGENTS.md "Before opening a PR", step 3) hasn't happened yet.

### Siblings

- #8739 ci: Hermit fetch deadline and sccache fallback
- #8741 test(mobile): RunnerTests title fade and alert wait
- #8742 fix(desktop): deep-linked message hold (product change)
- #8743 test(desktop): four specs racing scroll settles and focus
- #8744 test(desktop): five settle classes and remote-owned-mentions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Matt Toohey <contact@matttoohey.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
bradseiler added a commit that referenced this pull request Nov 5, 2026
## Summary

Apply relay mode consistently across protected HTTP, root/audio
WebSocket, Git, media and V1 admission. Off requires no NIP-FI,
including for stored required communities. Shadow enforces only stored
community requirements and observes other communities. Enforce requires
NIP-FI for every community. Durable requirements survive all mode
changes.

Off retains ordinary tenant routing and authentication. It skips FI
policy/dependency reads for admission, uses cached WebSocket tenant
routing, and loads no FI binding snapshot at startup. Discovery reports
`requires_nip_fi=false` and no FI capability in Off. Evaluating modes
retain same-row WebSocket binding and immutable HTTP policy tenants,
including host remaps, pending deletion and unarchive races. Global
Enforce discovery retains its existing compatibility behavior.

Host-specific root NIP-11 and `/info` expose the additive
`requires_nip_fi` field with no-store. In evaluating modes, an
unavailable policy returns 503 and an unknown or non-serving host
returns 404. Off uses ordinary routing; a discovery lookup failure
returns 404. Stored requirements in Shadow require usable FI
dependencies.

Admission does not establish activation. BUZZ-512 owns existing-session
application, drain and mode-aware readback; its separate revision must
report stored requirements as suspended in Off. Production baseline
verification and human acceptance remain open.

Community status now shares the bounded host cache: 10,000 entries,
one-second leases measured before checkout, coalesced fills, and no
cached misses or errors. Git reuses one request policy. Root/audio
retain the final primary binding check: two reads on a cold successful
upgrade, one on a cache hit. Off creates no verifier or JWKS source and
reads no FI policy. Proof metrics count stored-required Shadow
enforcement.

Acceptance locks and validates the community row, takes the matching
host advisory lock, and drains optional leases for 1.1 seconds before
commit. Reads begun after commit see required across participating relay
instances, even without Redis invalidation. The row lock closes single
and double remap races. Admissions already in flight remain subject to
BUZZ-512 activation fencing.

### Related issue


[BUZZ-511](https://linear.app/squareup/issue/BUZZ-511/relay-enforce-per-community-nip-fi-policy-and-publish-compatible).
[BUZZ-510 / #8985](#8985) is merged;
this PR now targets main.

The restack retains main’s #8974 denial telemetry and artifact
capabilities. Related work includes #8526 tenant lookup errors and #8444
media authorization.

### Rollout and containment

Keep `BUZZ_NIP_FI_UPGRADES_ENABLED=false`. [The rollout
plan](docs/community-auth-upgrades.md#staging-and-containment) starts in
staging with zero required rows and a 30-minute before/after traffic
comparison. Initial abort limits include any policy timeout/503,
policy-read p99 above 50 ms, writer acquisition p99 above 20 ms or twice
baseline, and a five-minute 503 increase above 0.1 percentage point.
These are initial limits to validate against measured staging results,
not production capacity claims.

Verify signed/unsigned free controls, real required root/audio upgrades,
dependency failures, restart behavior and discovery before expansion.
Also watch binding timeouts/errors and their current generic 404
responses on known serving hosts alongside writer-pool pressure;
evaluating root/audio binding has a 500 ms budget; Off uses ordinary
cached routing. In Shadow, a pre-511 rollback loses enforcement of
accepted community requirements. Stop new upgrade calls, retain accepted
policy and receipts, and repair with an enforcing binary or
operator-approved containment. Never clear durable policy to restore
access. Off deliberately suspends FI enforcement and must be an explicit
operator mode choice. No production activation is claimed.

All acceptance paths must implement the lease protocol before upgrades
are enabled. Keep upgrades disabled through a mixed-version rollout. For
rollback, disable new acceptance before removing the protocol while
caching pods remain. A cold policy fill uses a six-statement transaction
and a 400 ms server timeout within the 500 ms lookup budget. A contended
policy fill returns 503. During the drain, archive, deletion, ownership,
issuer-binding and competing acceptance writers for that community may
wait; ordinary serving reads and writes retain their existing behavior.
Host-binding changes retain the existing one-second routing bound. Local
and Redis invalidation accelerate refresh; safety does not depend on
delivery. Deploy the verified Shadow revocation behavior from #8991
before accepting required communities in Shadow.

### Testing

Published review-fix head: `39dfe79b1fc04f10ee3eb94d6ea42cf686bafe1e`,
tree `1e2e8d3abd39ee1d6821defe8c8d34de73726e08`. McKeon cleared this
cache delta. Full affected auth (291), DB (192), pubsub (31) and relay
(1,682) package tests passed, including integration binaries. Two
previously documented hardcoded-Redis audit tests remained excluded.
All-target clippy, all 1,297 native PostgreSQL tests and the full pubsub
Redis suite passed. The full ACP package and `just ci` passed after
rebuilding cached test executables that embedded a removed checkout
path. No ACP source changed. Tauri's generated lockfile delta was
archived and restored; it is excluded from this candidate.

Both positive lease regressions passed. Four deliberately broken
variants failed at the intended assertions: removing the drain left an
optional lease alive after 202.935667 ms; skipping the fill lock failed
contention refusal; moving the timestamp after the read failed the
anchor check; taking the host lock before the row failed the
remap-blocking check. Exact source was restored after each mutation.
These are local validation results, not a deployed-fleet acceptance
claim. All applicable normal push hooks passed. GitHub reports the exact
tested head and tree; the published code has no delta from the reviewed
candidate.

Earlier mode-correction evidence follows for context.

Current mode-correction head:
`aa9abf07284b56a88eb78a2014e590f39d705288`, on main
`af54258f19ac32667c027f889652bdeb479c8e13`. Its code equals reviewed
`f1e7da5fc5`, plus the independently reviewed operator documentation
follow-up; all 19 earlier remote patches are preserved. McKeon verified
the correction and its mapping to the tested candidate.

The mode-correction candidate passed full auth (291) and relay (1,679)
package tests, clippy, and four explicit native PostgreSQL policy
workflows. There were 623 ignored relay tests and two excluded Redis
tests. Real PostgreSQL checks cover an ordinary unsigned caller on a
stored-required host in Off with no FI token or dependencies, discovery
false, durable policy retained, root/audio upgrades, host remaps and the
unarchive race in evaluating modes. The `STORED_OFF` mutation fails the
enum truth table only; `OFF_PRIMARY` fails cached upgrade routing.
Restored source hashes match. Earlier failed compile/fixture attempts
remain recorded.

The first normal push hook attempt passed Tauri checks but failed the
gateway startup verifier test (82/83 gateway tests passed). The remote
remained unchanged. The retry rebuilt gateway artifacts with separate
root and Tauri caches, then passed every normal push hook and published
this head. No gateway source changes or hook bypasses were needed.
Current-head hosted CI is tracked separately. Human acceptance and the
staging binding-timeout/404 workflow remain open. The following retained
validation and controls are historical and describe behavior before the
mode correction; their Off-enforcement statements are superseded by the
Summary above.


Published head: `9a717fa89725f9430ff4a8bb78cd600affb8836d`, on main
`1055f79d048ef22c878d02a7afb04f49c753564f`. CI, three schema checks,
2,136 ordinary tests, all 291 auth tests, 12 Redis tests, all 1,300
native PostgreSQL tests and discovery passed. The required wrapper
passed units, then exited 1 at unavailable Docker bootstrap. BUZZ-510 is
complete at `95c8d31c835d6f7ff6e432fb1ed248771d4f06dd`: CI, three schema
checks, 2,127 ordinary tests, 12 Redis tests, all 1,295 native Postgres
tests and discovery passed; its wrapper passed units and failed at
unavailable Docker bootstrap.

The first native run at historical BUZZ-510 `5380cb72` passed 1,291
tests and failed two event-follow-up queue assertions and the scale-sync
case. All three passed serially, then the full 1,294-test run passed
with normal concurrency. The original cause remains unresolved; failed
runs and resource samples are retained.

Standalone BUZZ-511 checks exposed access-slot callers, a Clippy warning
and early HTTP 503 attribution that had remained in the child. All are
now corrected in the parent. The unchanged real-router refusal
regression passed before the latest commit. BUZZ-512's final tree is
unchanged by moving these parent prerequisites. The remaining 46-file
child delta was audited; no additional parent prerequisite was found.
Earlier compile and native failures remain archived.

The following earlier-head results and controls are historical; the
current-head validation is reported above.

At `48388834f99b8d045e243b619904cb3e27b47146`, based on BUZZ-510
`130f14a675edd7cbe6a69d1c9d7818bd1a51f46c` and reconciled main
`152441ae0a56b204b42b6faa56a36ad2b98218b1`:

- `just ci`: passed.
- Four-package ordinary suite: 2,069 passed; full `buzz-auth` package:
291 passed.
- Native Redis: 12 passed. Native PostgreSQL: 1,261 passed. PostgreSQL
discovery checks passed.
- Required `just test`: units passed, then exited 1 because Docker
service bootstrap was unavailable. Native service results above are
separate.
- Real-router remap checks reject an A-only member under B policy, avoid
A data after B admission, and advertise B in discovery despite a stale A
cache. Removing each tenant pin separately fails its intended witness;
restored source passes. An unmapped host cannot reuse a cached tenant.
- Deadline witnesses advance their clock during real HTTP signature
verification and retain socket primary-lookup barriers. Removing either
final enforced or shadow HTTP deadline check separately fails its
intended witness; restored source passes both deadlines and the
single-resolution control.
- Telemetry regression: unsigned dev proofs now expect the earlier
NIP-98 `missing` denial; signed mismatched-key proofs retain pairing
denial and signer attribution. Removing unsigned-proof rejection fails
the exact two-counter assertion. Restored full relay package: 1,654
passed.
- Preserved failed runs: two prior bottom-layer desktop gates failed an
immediate inventory assertion, repaired with a bounded wait for the same
observable arrays and verified with delayed-notification and
frozen-provider controls. An earlier BUZZ-511 native run passed 1,242
tests and failed two deadline fixtures that relied on the tenant lookup
removed by the policy pin; the test-only adaptation above replaces that
seam. These failures remain archived. The first native run on
bottom-layer `130f14a6` passed 1,255 tests and failed partition
contention; its focused original test and fresh full 1,256-test run
passed without source changes. The cause remains unresolved. The first
CI run at `d8d88433` failed the old telemetry expectation; the test-only
correction above preserves exact denial counts and attribution.

Earlier evidence at `dddf05d8be8195d567db0f989f70825b1246a6c1`
(historical head):

- `cargo nextest run -p buzz-auth -p buzz-db -p buzz-relay --lib
--tests`: 2,139/2,139 passed.
- `scripts/postgres-test-run.sh -p buzz-db -p buzz-relay --lib --tests
--success-output final`: 1,236/1,236 passed with native PostgreSQL 16
and Redis. Docker's legacy `just test` wrapper was unavailable.
- `just ci`: passed, including repository checks, tests and client
builds.
- **Pre-correction behavior (superseded):** Real ES256 assertions
through TCP root/audio upgrades: valid evidence upgrades; missing
evidence denies in Off/Shadow, including submitted/inventoried/approved
deletion stages. Restoring the old query makes the regression fail on an
assertion-less Off/submitted/root HTTP 101.
- **Pre-correction behavior (superseded):** Deterministic unarchive
race: Notify barriers pause an actual archived-host policy miss; the
same required row becomes serving before binding. Both root/audio deny
missing assertions and admit valid ES256 assertions in Off/Shadow.
Ignoring bound-row policy makes this test fail on an assertion-less HTTP
101.
- The full rerun exposed an existing cancellation fixture whose
synthetic five-second replica fence expired between probes. Refreshing
that test fence per operation preserves its timeout assertions; the full
suite now passes.
- Local probe: 100 serial uncached primary policy reads took 12.33 ms.
Production latency is unmeasured.

No packaged-app, human or deployed-fleet acceptance run. Live activation
awaits BUZZ-512 and the staging verification tickets.

Published with the three-layer stack in one atomic push through normal
hooks. The published stack includes main
`1055f79d048ef22c878d02a7afb04f49c753564f`, including access logging,
storage reporting and mobile read-state changes. Access-slot integration
and early HTTP refusal attribution belong to BUZZ-511; activation
callback refusal close owners belong to BUZZ-512. Every changed layer
requires review at its published head and new-head CI. Fresh trunk,
remote membership, queue and immutable-lease guards passed before
publication. All nine normal pre-push hook lanes passed; no hooks were
bypassed. Remote heads, chained bases, membership, ready/draft states
and native tracking were verified after publication.

Trunk’s merged 0081 is unchanged. The parent upgrade expansion is 0082
and activation is 0083, with contiguous migrator versions, updated
documentation and a populated-upgrade fixture starting at 81. Trunk’s
storage-test repair supersedes our earlier local repair.

Before publication, main advanced three commits from
`1055f79d048ef22c878d02a7afb04f49c753564f` to `ba7d7825ea` (desktop
deep-link settling and mobile pairing/sign-in). The normal branch-skew
guard passed. Those later trunk commits are outside this validated
published head; current-trunk CI and delta review remain required before
merge.

Generated with Codex

---------

Signed-off-by: Brad Seiler <seiler@squareup.com>
Co-authored-by: Honor <e0bf479eeebe47dde1e35705212a4df70616a7bc834d907ddfb18cf0405f8b1b@seiler-test.communities.public.builderlabstaging.xyz>
Beige-Coffee added a commit to Beige-Coffee/buzz-selfhost that referenced this pull request Nov 6, 2026
The read answer says plainly that any member with a file's link can open it, even from a private
channel or a DM, that removing someone from a channel doesn't take a file back, and that Buzz's
setting to limit files to their channels starts off when it ships (block/buzz#8444). The push
sentence names what Block's gateway sees, and a third note says the relay's log records who read
what (block/buzz#8376, in the pin).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Beige-Coffee added a commit to Beige-Coffee/buzz-selfhost that referenced this pull request Nov 6, 2026
File reads check community membership only (block/buzz media.rs get_blob and head_blob, at the
pin). The fix, block/buzz#8444, is still open and starts in log-only mode, so the paragraph tells
people to keep restricted files out of Buzz until a release has the setting and it's on. It also
says the relay's log names each member with what they read (#8376, in the pin).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bradseiler added a commit that referenced this pull request Nov 6, 2026
## Summary

Removed private-channel members can currently download attachments if
they know the URL. With `BUZZ_MEDIA_READ_MODE=enforce`, fresh GET, HEAD
and Range requests check current membership and readable references on
the primary database. Removed members receive 404; remaining members
retain access, and rejoining restores access. Thumbnail aliases use the
parent attachment’s permissions.

Publication checks also prevent copied private URLs from granting new
access. They cover attachments, link previews, profile pictures,
banners, group pictures and emoji. Persona creation and changed-avatar
edits, plus team and individual snapshot imports, transfer animated
posters and animations under the agent’s signer before profile
publication. Failed persona transfers preserve existing local records;
import failures appear in the results.

Authorization uses bounded queries, a two-second statement deadline and
transaction-local JIT suppression. An index on edit targets avoids broad
scans. Publications have a 512-reference limit, and capacity waits up to
500 ms before refusal.

Protection defaults off. Existing downloaded bytes cannot be recalled.
All modes stop creating immutable HTTP cache entries.

### Related issue

Related approaches: #7565, #5185, #8451 and #8444. This change evaluates
live stored references directly.

### Testing

Published head: `3d1aa7431cc495a001a0f13e95e2e422c5a77ab1`, rebased on
main `2cd40badb85de059a6916206e133da7a0a88fb2e`. Push hooks were
**SKIPPED** with explicit author approval. Current-head HTTP, full
package, PostgreSQL migration/media and profile-filter mutation checks
are pending; CI is pending. The earlier complete HTTP journey passed at
`f20b449262` (identical source tree), with its evidence attributed
separately. Earlier failing preview/report-fixture runs are preserved;
they do not count as passes.

The HTTP journey covers upload/publication, fresh GET/HEAD/Range
revocation and rejoin, emoji replacement, edited previews, persona
visibility and non-reference decoys. The migration regression forces an
index timeout, verifies no 0086 history or media tables, then proves a
migrator through the base's 0085 can restart.

Human attachment removal/rejoin testing passed on the pinned
`1f08a8e0fd2edf19dc94321f7258518149fe1bf6` trial. Native animated
persona edits and team/individual import acceptance remain incomplete.
That old-snapshot database stays intact and cannot run the reconciled
binary; a new trial needs a separate fresh database.

### Rollout prerequisites

- Build and verify the reference index before shadow mode. Small
databases can auto-migrate through one atomic 0086 migration within the
configured timeouts. It builds the index before creating the media
tables, so failure records no new migration version. Large deployments
must disable auto-migration and prebuild indexes concurrently.
- **Enforce remains blocked unless the operator can rule out SDK/CLI
attachment edits and the unverified agent-harness edit producer.**
SDK/CLI edits still omit attachment metadata. That behavior is deferred,
not fixed. Removing this blocker requires latest-effective attachment
metadata across successive edits, no restoration of removed files, and
an upload → successive edits → GET/HEAD regression.
- Validate client signer compatibility and split emoji publications
above 512 references before enforce mode.
- Complete the human workflows above in an isolated community with
enforce enabled.
- Follow the [index deployment
guide](https://github.com/block/buzz/blob/honor/private-channel-media/docs/private-media-index-deployment.md)
and [operator
guidance](https://github.com/block/buzz/blob/honor/private-channel-media/docs/operator-private-media.md)
for rollout and rollback.

The [prebuild timeout
finding](#9077 (comment))
and [avatar-transfer
finding](#9077 (comment))
are implemented and independently reviewed.

### Schema change checklist

- [x] Additive expand and code deployment; no backfill or contract
phase. Prebuild/verify before shadow under either schema deployment
path.
- [x] No SQL function, trigger, procedure, rule or foreign key added.
Signed events are untouched.
- [x] No migration mutates existing rows.
- [x] Migration and desired-state schema match. Catalog parity, target
parsing, future partition inheritance and upgrade refusal are covered.
- [x] Old/new binaries coexist for serving. Suspend community deletion
during expansion and while rolled back to a binary with the old deletion
catalog.
- [x] Keep auto-migration disabled on old binaries after schema
expansion; sqlx rejects unknown versions.
- [x] Migration lock timeout 1 s; statement timeout 5 s. Small populated
databases build inline within those budgets; large deployments use
concurrent child-index prebuild/attachment with auto-migration off.
Missing, invalid or wrong indexes fail verification.

Generated with Codex

---------

Signed-off-by: Brad Seiler <seiler@squareup.com>
Co-authored-by: Honor <e0bf479eeebe47dde1e35705212a4df70616a7bc834d907ddfb18cf0405f8b1b@seiler-test.communities.public.builderlabstaging.xyz>

This branch had an error being deployed

1 failed deployment
codex-review — e25d56f1 Deployed Oct 23, 2026 by ngoodger-block via Run Codex Security Review #9595
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants