Repository navigation
fix(relay): deny media reads outside the channels a file is shared in - #8444
ngoodger-block wants to merge 68 commits into
Conversation
🔐 Codex Security Review
|
Add `media_shares` and `media_uploaders` (migration 0067) and start filling them, without changing who can read media yet. Each local `/media/<sha256>` path in a stored event shares that blob into the event's channel (or with the whole community when channel-less), in the same transaction as the event, when its author can read the blob or uploaded those bytes; each upload records its uploader under the serving-write lease. `buzz-admin backfill-media-shares` records shares for events stored before this, so channel-scoped reads can be enforced next without hiding existing media. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Clients upload a file before attaching it, so record the uploader first and keep the test valid once reads deny files not yet shared into any channel. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
wpfleger96
left a comment
There was a problem hiding this comment.
🤖 thanks for this. the GET/HEAD check itself looks right: both run it before touching storage, thumbnails are checked against the original hash, and a refused read gets the same 404 as a missing file. one blocking issue inline (agent avatars uploaded by the owner end up readable only by the owner), plus a few smaller items.
two things about the stack as a whole:
- tightening the no-shares arm of
readable_sql!is what closes the "anyone with the hash can share an unshared file" hole from the #8451 review, sincerecord_media_sharesuses the same macro. landing it here means it reaches prod after #8451 and the backfill, so the hole stays open that whole time. could the predicate change move into #8451, leaving this PR as just the read enforcement? it also can't revoke public grants created while only #8451 is deployed, and the backfill replay issue from that review is the same code here. - the PostgreSQL lane is red at this head, 1051 of 1052:
migration_schema_fence_objects_are_optional_for_serving_and_deletionfails withcommunity deletion catalog drift (missing=media_shares,media_uploaders). the fixture migrates through 66 (deletion.rs:4402) butEXPECTED_SCOPED_TABLESnow includes the 0067 tables, so bumping it to 67 should fix it. the cause is in #8451, so it fails there too.
a few questions on rollout:
- there's no off switch or log-only mode, so at deploy any file the backfill missed (unparsed references, files uploaded before #8451 with no uploader row, the avatar case inline) 404s for everyone but its uploader, and the only way back is a redeploy. NIP-FI already has off / log-only / enforce. is the same worth doing here, so the "would deny" counts can be checked before enforcing?
- NIP-56 reports only land in
moderation_reports, neverevents, so a report never creates a share. a mod who isn't in the private channel now gets a 404 on the reported file. should mods reviewing an open report be able to read it? - the media row in
docs/multi-tenant-conformance.mdstill lists the channel-ACL read gate as a remaining gap, and the NIP-FI read-proof note still says a captured header reads any media blob. is updating those meant to be part of this?
|
@buzz-security-review ffaa6fb |
Trust references only from events received before migration 0067 was installed; later events get the live, checked recording, so a backfill run after rollout never shares a blob that live recording refused. Insert shares in (hash, channel) order from both the backfill and live recording so the two cannot deadlock, and read media paths in JSON content that escapes `/`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Require upload proof or a recorded share when recording new references. Let an agent publish only its profile picture using its registered owner's upload proof. Keep the historical backfill cutoff on communities for both schema setup paths, and advance the deletion fixture through migration 67. Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
|
🤖 Added off/log-only/enforce with a log-only default, audited owner/admin access limited to open reports’ live attachments, and both documentation corrections in dd02e04, with passing runtime regressions and Desktop smoke testing still pending. |
wpfleger96
left a comment
There was a problem hiding this comment.
🤖 thanks for working through all of this. the earlier feedback is addressed: owner-uploaded agent avatars now get a share and have a cross-signer E2E test, the strict share rule lives in #8451, the migration-67 fixture is fixed, the off / log-only / enforce switch has counters and rollback docs, moderators can review attachments on open reports with audited reads, lookup failures are logged, and the conformance table and NIP-FI note are updated. the two deferrals (recovering past uploaders from audit_log, the removed-member regression) are fine as follow-ups.
two things before this can be approved:
- this branch contains #8451 unchanged, so it also carries the backfill blocker from that review: post-cutoff events are re-checked with current permissions, so a refused reference can become a community-wide share once its author gains access. once that fix lands in #8451 and is merged here, this clears.
- one wording fix in the NIP-FI note (inline).
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
wesbillman
left a comment
There was a problem hiding this comment.
🤖 Star Lord’s automated source review via Wes’s account, integrating independent Rocket (systems/lifecycle), Mantis (correctness/security), and Groot (product/user-flow) reviews.
Reviewed head dd02e04fb7bff7dc5ec533cd03bd623005d48415 against base 6e6173acd58029bac800583c4cc7482dafed693a, including the stacked #8451 changes. Two P2 correctness findings are inline. The backfill finding also confirms the existing review discussion; the edited-message deletion case is additional.
[P3] Public PR material: remove the internal Linear tracker link from “Related issue” and replace it with a public issue or a self-contained description. This public repository currently exposes an internal workspace/issue identifier there. No credential exposure is claimed, and the internal issue contents were not accessed.
Optional documentation precision: in docs/nips/NIP-FI.md:722–726, describe the file policy in protocol terms and keep deployment-mode names in the operator guide. Qualify “upload proof” as preview permission only while no channel share exists.
Validation: source only; no tests, builds, SQL, or apps executed. One CI snapshot showed 65 successful checks and 31 skipped; that does not validate the reported journeys. Desktop enforce-mode edit/upload/profile behavior, recovery/focus after media 404/503 responses, rolling-deploy provenance recovery, and writer-pool performance remain unverified. The PR description has no attached images. Minimalism/elegance: 9/10; correctness: 7/10 pending the two inline repairs. This is a non-blocking COMMENT review, not approval or merge authorization.
|
🤖 @wpfleger96 Please re-review the behavior-only NIP-FI wording and inherited backfill clarification at |
wesbillman
left a comment
There was a problem hiding this comment.
🤖 Star Lord’s automated source review via Wes’s account, integrating independent Rocket (systems/lifecycle), Mantis (correctness/security/concurrency), and Groot (product/operator flows) reviews.
Reviewed head a8b3dd55e7e4082587f078b8e4abd5f90af74166, base 7e6461242aa7d98f132919dd672f7cc5f4abbba6. This follow-up covers the three new documentation/test commits and membership-writer integration since dd02e04fb7bff7dc5ec533cd03bd623005d48415; the preceding 24 feature patches are unchanged.
One new P2 rollout-documentation finding is inline. No new defect found in the adapted fixture calls: membership changes commit before the database-backed authorization check, so discarding the returned snapshot is harmless in this test. The NIP-FI wording now describes protocol policy separately from rollout configuration.
The existing lifecycle/backfill and public-description observations remain in the previous review; the documentation changes do not repair those code paths, and they are not duplicated here.
Source-only review; no code, SQL, tests, builds, apps, or deployments executed. Public description inspected; no attached images. Linked internal issue content was not inspected. One CI snapshot showed 20 successful, 20 skipped, 18 in-progress and 1 failed check; failure cause and runtime correctness are unverified. The description’s test results were not independently reproduced or established for this exact head. Desktop edit/upload/profile workflow, database concurrency, rollout behavior, and error/retry focus remain unverified. This COMMENT is not approval or merge authorization.
Add `media_shares` and `media_uploaders` (migration 0067) and start filling them, without changing who can read media yet. Each local `/media/<sha256>` path in a stored event shares that blob into the event's channel (or with the whole community when channel-less), in the same transaction as the event, when its author can read the blob or uploaded those bytes; each upload records its uploader under the serving-write lease. `buzz-admin backfill-media-shares` records shares for events stored before this, so channel-scoped reads can be enforced next without hiding existing media. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Clients upload a file before attaching it, so record the uploader first and keep the test valid once reads deny files not yet shared into any channel. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Trust references only from events received before migration 0067 was installed; later events get the live, checked recording, so a backfill run after rollout never shares a blob that live recording refused. Insert shares in (hash, channel) order from both the backfill and live recording so the two cannot deadlock, and read media paths in JSON content that escapes `/`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Require upload proof or a recorded share when recording new references. Let an agent publish only its profile picture using its registered owner's upload proof. Keep the historical backfill cutoff on communities for both schema setup paths, and advance the deletion fixture through migration 67. Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
|
🤖 @wpfleger96 please re-review this updated stack at 2cf0a83: edit grants follow the original message lifetime, legacy profile/emoji replacements recover eligible references, the rollout decision precedes the first backfill, and the media migration now has a tested five-second lock budget; #8535 is the schema prerequisite, all 29 real relay media journeys passed, and the actual Desktop UI smoke test remains pending. The security workflow has failed before analysis because checkout blocks fork PR code under |
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
* feat/media-shares: fix(db): bound the media migration lock wait Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
wesbillman
left a comment
There was a problem hiding this comment.
🤖 Star Lord’s automated source review via Wes’s account, integrating independent Rocket (systems/lifecycle), Mantis (correctness/security/concurrency), and Groot (product/user-flow) reviews.
Reviewed head 7b8bee1e07f43ff4b7653a480865766197ce1007 against base 347ce88d5dd8bc631c073f1947fea6b75d5a43c0, focusing on the follow-up since b267129ff7d64e87b23cda3a89977e8eeadb4a3e and its supported callers.
The prior edit-lifetime defect is repaired in source: ordinary and report-based reads require the edit’s original message to remain live, while independent live shares survive. Legacy profile/emoji replacement recovers eligible pre-cutoff references before retiring the old head within the admitted transaction. The schema prerequisite removes blanket bootstrap fencing and the new media tables’ FKs/fences consistently, retaining application admission and purge inventory. The rollout now requires the accept/defer decision before the first backfill, and the internal tracker link has been removed from the public description. No new defect found in these follow-up repairs.
Retained P2: bound migration 0068’s lock wait. The existing finding on stacked #8451 still applies here: migrations/0068_media_access.sql:2–3 requests ACCESS EXCLUSIVE on communities with no local timeout, while runtime/migration.rs disables session lock/statement timeouts. A conflicting transaction can leave the migration and subsequent community operations waiting indefinitely. Add a short SET LOCAL lock_timeout before the ALTER. This is carried code, independently confirmed in this review; the existing inline is linked rather than duplicated.
Backfill still reevaluates earlier refusals using current permissions. This remains an explicitly documented retained limitation, not a repaired code path; the revised instructions correctly require accepting the persistent resulting grants before any run or deferring backfill. Changing read modes does not revoke those grants.
Validation: source only; no tests, SQL, builds, apps, backfill, or deployments executed. Public description/discussion inspected; no attached images. The described 29-test run was not independently reproduced or bound to this exact head. One CI snapshot showed 46 successful, 29 skipped, 8 running, and 1 failed security-review check; failure cause was not independently verified. Runtime revocation/concurrency, query cost, rollout behavior, and native edit/upload/profile recovery and focus after 404/503 remain unverified; the Desktop smoke test is still pending. Minimalism/elegance 9/10; correctness 8/10 pending the retained lock bound. Non-blocking COMMENT, not approval or merge authorization.
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
* feat/media-shares: fix(media): reclaim inactive grants and cover upload attribution Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
* feat/media-shares: fix(media): run cleanup before telemetry collection Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
* feat/media-shares: fix(media): finish deferred provenance and access feedback Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
|
🤖 @wpfleger96 please re-review 6483588: the deferred feedback is now implemented, including uploader-handler coverage, inactive-grant cleanup, historical uploader recovery, private-kind exclusions, accurate error logging, positive batch sizes, and removed-member access coverage; all 29 live media journeys and the targeted mutation checks passed. The author’s accepted backfill replay limitation remains documented; its conversations were closed as an accepted limitation, not a code fix. The actual Desktop UI smoke test remains pending. The fresh security workflow failed at the fork-checkout guard before analysis; the protection remains enabled, so there is no new security review result. |
The optional-fence test assumed every scoped table is fenced, so the first tenant tables added under docs/database.md (scoped but admitted only in the app) broke it. Assert that fences exist exactly when the fence objects do and cover only scoped tables. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
* feat/media-shares: test(db): expect only legacy tables to carry community write fences Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
…trap (#8620) ### Why? This is a cleanup that brings the schema bootstrap in line with how the codebase works today, as asked for by [`docs/database.md`](https://github.com/block/buzz/blob/dc25198d530acc6237a0320b697365f29daf0ea6/docs/database.md#L103-L124) (#8273). Writes are now blocked during community deletion by application admission (`begin_community_write_transaction`), and that doc's Rule 1 says new tables must not add triggers. But `schema/community_write_fence.sql` still looped over every table with a `community_id` column and attached the old fence trigger to it. That meant a new table got a trigger on a fresh database and none on a migrated one, so the two schema paths disagreed and new tenant tables were blocked. The doc says this bootstrap change must land "before the feature, or as the feature's first PR." ### What? Deletes the blanket fence loop and its `\i` from `schema/schema.sql`. Existing legacy tables keep their fences through their explicit `schema/triggers/` files, so no current table loses its fence and runtime behavior is unchanged. Adds a test that a new `community_id` table stays unfenced while `events` and `users` stay fenced, and updates `docs/database.md` and `ARCHITECTURE.md` to match. It stands on its own; the media-sharing PRs (#8451, #8444) are the first to rely on it. buzz-review-completed --------- Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ests (#8740) ## Summary Two buzz-relay test fixes from the 23–30 Oct flake audit. Both are test-only. This is one of six PRs split from the reviewed `flakey-tests` branch (siblings below). ### `thread_window_bridge_restarts_both_aux_hops_after_replica_failure` (3 runs) - **Runs:** 40799617319, 40924179886 and 40930100994 (#8697, docs only), all on 10-29 in PostgreSQL Tests, attempt 1 (jobs 123208043033, 123632125866, 123651641392). None of the branches touches thread_window, and each run passed on a re-run. Signature: `failure_postgres_tests.rs:46`, "barrier must precede closure completion", left: 2, right: 1. - **Cause:** `pause_after_aux_page` polls the request until a `thread_window_aux` span closes, then expects exactly one. The fixture's 1,001 reactions make two replica pages at `AUX_LIMIT` 1000, and each page is one round trip on the transaction already held. A loaded runner can deschedule the test thread after page 2's query is written. Postgres then answers first, and the same poll reads page 2 too. - **Change:** `test_support` gains a `PausingProxy`, a trimmed copy of buzz-db's private one with only the hold-everything switch. The replica reader pool connects through it. `AuxPages` pauses it when the first aux span closes. That runs inside the request's poll, after page 1's response is consumed and before page 2's query is written, so page 2 stays pending however fast Postgres answers. The test resumes the proxy after it terminates the reader's backend. The held page then sees the termination, and the closure restarts on the writer, as before. The other three `pause_after_aux_page` callers read from the writer and pass `None`. ### `api::media` tests that need Postgres (fail every time Flake Check selects them) - **Run:** 40954304982 (#8658), 10-29, Flake check (Rust), job 123730434146. All six `test_state()` tests panic at the community-seed `expect` after the 30 s sqlx acquire timeout. - **Cause:** when a hunk lands outside a test body, Flake Check selects every test in the file. It skips only `postgres_tests::`, `postgres_*` binaries and `external_infra`, and its job has no database. These six tests seed `relay.example` through Postgres but sat in plain `mod tests` without `#[ignore]`. So any PR that changes non-test code in `media.rs` fails that lane, and at least 8 open PRs touch the file. No other lane ran these tests either. - **Change:** move `test_state`, the `media_get_*` helpers, `media_request` and the six tests into `tests::postgres_tests`. Mark each test `#[ignore = "requires Postgres"]`, and point `test_state` at `test_support::database_url()`, as the module's other states do. The PostgreSQL lane now runs them, and Flake Check skips them. The change also drops imports `mod tests` no longer uses, and updates the Justfile comment that gave `api::media` as the reason the infra-free lane is scoped to `api::admin`. - When `media.rs` is staged, the pre-commit secret scan reads the whole file and flags the module's existing public ES256 test key (`TEST_EC_PKCS8_PEM`). The key is now written on one line with a sadscan marker, as `invites.rs` already does. Its bytes are unchanged. ### Related issue None found. **Conflicts with #8451 and #8444**, which both already conflict with `main`. The one-line `TEST_EC_PKCS8_PEM` rewrite is the only new conflict hunk with each. The test move merges cleanly with both. ### Testing Verified locally: - Each of the four `pause_after_aux_page` tests passes when run alone against local Postgres through `scripts/postgres-test-run.sh`. - A forced check, not committed: after the barrier, the test gave page 2 another 500 ms over ten re-polls and asserted the page count stayed at 1. It passed with the pause and failed with `None`, with the CI signature (left: 2, right: 1). A true same-poll burst needs the OS to preempt between sqlx's write and read, so it was not forced directly. - `check-postgres-test-discovery.py` accepts the touched files and all 621 files under `crates/`, and `scripts/test-postgres-test-discovery.sh` passes. Each of the six media tests passes when run alone against local Postgres. - `cargo fmt --check` and `cargo clippy -p buzz-relay --all-targets -- -D warnings` are clean under 1.98.1. Only hosted CI can show the full lanes, which were not run locally. The six media tests have never run in CI, so this PR's PostgreSQL lane is their first hosted run. Pushed with `git push --no-verify`, because the pre-push suites fail when the laptop is under load. ### Readiness An agent review of the combined `flakey-tests` branch (b432453af) found no blockers. `buzz-review-completed` is not added, because the human test step (AGENTS.md "Before opening a PR", step 3) hasn't happened yet. ### Siblings - #8739 ci: Hermit fetch deadline and sccache fallback - #8741 test(mobile): RunnerTests title fade and alert wait - #8742 fix(desktop): deep-linked message hold (product change) - #8743 test(desktop): four specs racing scroll settles and focus - #8744 test(desktop): five settle classes and remote-owned-mentions 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Matt Toohey <contact@matttoohey.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
## Summary Apply relay mode consistently across protected HTTP, root/audio WebSocket, Git, media and V1 admission. Off requires no NIP-FI, including for stored required communities. Shadow enforces only stored community requirements and observes other communities. Enforce requires NIP-FI for every community. Durable requirements survive all mode changes. Off retains ordinary tenant routing and authentication. It skips FI policy/dependency reads for admission, uses cached WebSocket tenant routing, and loads no FI binding snapshot at startup. Discovery reports `requires_nip_fi=false` and no FI capability in Off. Evaluating modes retain same-row WebSocket binding and immutable HTTP policy tenants, including host remaps, pending deletion and unarchive races. Global Enforce discovery retains its existing compatibility behavior. Host-specific root NIP-11 and `/info` expose the additive `requires_nip_fi` field with no-store. In evaluating modes, an unavailable policy returns 503 and an unknown or non-serving host returns 404. Off uses ordinary routing; a discovery lookup failure returns 404. Stored requirements in Shadow require usable FI dependencies. Admission does not establish activation. BUZZ-512 owns existing-session application, drain and mode-aware readback; its separate revision must report stored requirements as suspended in Off. Production baseline verification and human acceptance remain open. Community status now shares the bounded host cache: 10,000 entries, one-second leases measured before checkout, coalesced fills, and no cached misses or errors. Git reuses one request policy. Root/audio retain the final primary binding check: two reads on a cold successful upgrade, one on a cache hit. Off creates no verifier or JWKS source and reads no FI policy. Proof metrics count stored-required Shadow enforcement. Acceptance locks and validates the community row, takes the matching host advisory lock, and drains optional leases for 1.1 seconds before commit. Reads begun after commit see required across participating relay instances, even without Redis invalidation. The row lock closes single and double remap races. Admissions already in flight remain subject to BUZZ-512 activation fencing. ### Related issue [BUZZ-511](https://linear.app/squareup/issue/BUZZ-511/relay-enforce-per-community-nip-fi-policy-and-publish-compatible). [BUZZ-510 / #8985](#8985) is merged; this PR now targets main. The restack retains main’s #8974 denial telemetry and artifact capabilities. Related work includes #8526 tenant lookup errors and #8444 media authorization. ### Rollout and containment Keep `BUZZ_NIP_FI_UPGRADES_ENABLED=false`. [The rollout plan](docs/community-auth-upgrades.md#staging-and-containment) starts in staging with zero required rows and a 30-minute before/after traffic comparison. Initial abort limits include any policy timeout/503, policy-read p99 above 50 ms, writer acquisition p99 above 20 ms or twice baseline, and a five-minute 503 increase above 0.1 percentage point. These are initial limits to validate against measured staging results, not production capacity claims. Verify signed/unsigned free controls, real required root/audio upgrades, dependency failures, restart behavior and discovery before expansion. Also watch binding timeouts/errors and their current generic 404 responses on known serving hosts alongside writer-pool pressure; evaluating root/audio binding has a 500 ms budget; Off uses ordinary cached routing. In Shadow, a pre-511 rollback loses enforcement of accepted community requirements. Stop new upgrade calls, retain accepted policy and receipts, and repair with an enforcing binary or operator-approved containment. Never clear durable policy to restore access. Off deliberately suspends FI enforcement and must be an explicit operator mode choice. No production activation is claimed. All acceptance paths must implement the lease protocol before upgrades are enabled. Keep upgrades disabled through a mixed-version rollout. For rollback, disable new acceptance before removing the protocol while caching pods remain. A cold policy fill uses a six-statement transaction and a 400 ms server timeout within the 500 ms lookup budget. A contended policy fill returns 503. During the drain, archive, deletion, ownership, issuer-binding and competing acceptance writers for that community may wait; ordinary serving reads and writes retain their existing behavior. Host-binding changes retain the existing one-second routing bound. Local and Redis invalidation accelerate refresh; safety does not depend on delivery. Deploy the verified Shadow revocation behavior from #8991 before accepting required communities in Shadow. ### Testing Published review-fix head: `39dfe79b1fc04f10ee3eb94d6ea42cf686bafe1e`, tree `1e2e8d3abd39ee1d6821defe8c8d34de73726e08`. McKeon cleared this cache delta. Full affected auth (291), DB (192), pubsub (31) and relay (1,682) package tests passed, including integration binaries. Two previously documented hardcoded-Redis audit tests remained excluded. All-target clippy, all 1,297 native PostgreSQL tests and the full pubsub Redis suite passed. The full ACP package and `just ci` passed after rebuilding cached test executables that embedded a removed checkout path. No ACP source changed. Tauri's generated lockfile delta was archived and restored; it is excluded from this candidate. Both positive lease regressions passed. Four deliberately broken variants failed at the intended assertions: removing the drain left an optional lease alive after 202.935667 ms; skipping the fill lock failed contention refusal; moving the timestamp after the read failed the anchor check; taking the host lock before the row failed the remap-blocking check. Exact source was restored after each mutation. These are local validation results, not a deployed-fleet acceptance claim. All applicable normal push hooks passed. GitHub reports the exact tested head and tree; the published code has no delta from the reviewed candidate. Earlier mode-correction evidence follows for context. Current mode-correction head: `aa9abf07284b56a88eb78a2014e590f39d705288`, on main `af54258f19ac32667c027f889652bdeb479c8e13`. Its code equals reviewed `f1e7da5fc5`, plus the independently reviewed operator documentation follow-up; all 19 earlier remote patches are preserved. McKeon verified the correction and its mapping to the tested candidate. The mode-correction candidate passed full auth (291) and relay (1,679) package tests, clippy, and four explicit native PostgreSQL policy workflows. There were 623 ignored relay tests and two excluded Redis tests. Real PostgreSQL checks cover an ordinary unsigned caller on a stored-required host in Off with no FI token or dependencies, discovery false, durable policy retained, root/audio upgrades, host remaps and the unarchive race in evaluating modes. The `STORED_OFF` mutation fails the enum truth table only; `OFF_PRIMARY` fails cached upgrade routing. Restored source hashes match. Earlier failed compile/fixture attempts remain recorded. The first normal push hook attempt passed Tauri checks but failed the gateway startup verifier test (82/83 gateway tests passed). The remote remained unchanged. The retry rebuilt gateway artifacts with separate root and Tauri caches, then passed every normal push hook and published this head. No gateway source changes or hook bypasses were needed. Current-head hosted CI is tracked separately. Human acceptance and the staging binding-timeout/404 workflow remain open. The following retained validation and controls are historical and describe behavior before the mode correction; their Off-enforcement statements are superseded by the Summary above. Published head: `9a717fa89725f9430ff4a8bb78cd600affb8836d`, on main `1055f79d048ef22c878d02a7afb04f49c753564f`. CI, three schema checks, 2,136 ordinary tests, all 291 auth tests, 12 Redis tests, all 1,300 native PostgreSQL tests and discovery passed. The required wrapper passed units, then exited 1 at unavailable Docker bootstrap. BUZZ-510 is complete at `95c8d31c835d6f7ff6e432fb1ed248771d4f06dd`: CI, three schema checks, 2,127 ordinary tests, 12 Redis tests, all 1,295 native Postgres tests and discovery passed; its wrapper passed units and failed at unavailable Docker bootstrap. The first native run at historical BUZZ-510 `5380cb72` passed 1,291 tests and failed two event-follow-up queue assertions and the scale-sync case. All three passed serially, then the full 1,294-test run passed with normal concurrency. The original cause remains unresolved; failed runs and resource samples are retained. Standalone BUZZ-511 checks exposed access-slot callers, a Clippy warning and early HTTP 503 attribution that had remained in the child. All are now corrected in the parent. The unchanged real-router refusal regression passed before the latest commit. BUZZ-512's final tree is unchanged by moving these parent prerequisites. The remaining 46-file child delta was audited; no additional parent prerequisite was found. Earlier compile and native failures remain archived. The following earlier-head results and controls are historical; the current-head validation is reported above. At `48388834f99b8d045e243b619904cb3e27b47146`, based on BUZZ-510 `130f14a675edd7cbe6a69d1c9d7818bd1a51f46c` and reconciled main `152441ae0a56b204b42b6faa56a36ad2b98218b1`: - `just ci`: passed. - Four-package ordinary suite: 2,069 passed; full `buzz-auth` package: 291 passed. - Native Redis: 12 passed. Native PostgreSQL: 1,261 passed. PostgreSQL discovery checks passed. - Required `just test`: units passed, then exited 1 because Docker service bootstrap was unavailable. Native service results above are separate. - Real-router remap checks reject an A-only member under B policy, avoid A data after B admission, and advertise B in discovery despite a stale A cache. Removing each tenant pin separately fails its intended witness; restored source passes. An unmapped host cannot reuse a cached tenant. - Deadline witnesses advance their clock during real HTTP signature verification and retain socket primary-lookup barriers. Removing either final enforced or shadow HTTP deadline check separately fails its intended witness; restored source passes both deadlines and the single-resolution control. - Telemetry regression: unsigned dev proofs now expect the earlier NIP-98 `missing` denial; signed mismatched-key proofs retain pairing denial and signer attribution. Removing unsigned-proof rejection fails the exact two-counter assertion. Restored full relay package: 1,654 passed. - Preserved failed runs: two prior bottom-layer desktop gates failed an immediate inventory assertion, repaired with a bounded wait for the same observable arrays and verified with delayed-notification and frozen-provider controls. An earlier BUZZ-511 native run passed 1,242 tests and failed two deadline fixtures that relied on the tenant lookup removed by the policy pin; the test-only adaptation above replaces that seam. These failures remain archived. The first native run on bottom-layer `130f14a6` passed 1,255 tests and failed partition contention; its focused original test and fresh full 1,256-test run passed without source changes. The cause remains unresolved. The first CI run at `d8d88433` failed the old telemetry expectation; the test-only correction above preserves exact denial counts and attribution. Earlier evidence at `dddf05d8be8195d567db0f989f70825b1246a6c1` (historical head): - `cargo nextest run -p buzz-auth -p buzz-db -p buzz-relay --lib --tests`: 2,139/2,139 passed. - `scripts/postgres-test-run.sh -p buzz-db -p buzz-relay --lib --tests --success-output final`: 1,236/1,236 passed with native PostgreSQL 16 and Redis. Docker's legacy `just test` wrapper was unavailable. - `just ci`: passed, including repository checks, tests and client builds. - **Pre-correction behavior (superseded):** Real ES256 assertions through TCP root/audio upgrades: valid evidence upgrades; missing evidence denies in Off/Shadow, including submitted/inventoried/approved deletion stages. Restoring the old query makes the regression fail on an assertion-less Off/submitted/root HTTP 101. - **Pre-correction behavior (superseded):** Deterministic unarchive race: Notify barriers pause an actual archived-host policy miss; the same required row becomes serving before binding. Both root/audio deny missing assertions and admit valid ES256 assertions in Off/Shadow. Ignoring bound-row policy makes this test fail on an assertion-less HTTP 101. - The full rerun exposed an existing cancellation fixture whose synthetic five-second replica fence expired between probes. Refreshing that test fence per operation preserves its timeout assertions; the full suite now passes. - Local probe: 100 serial uncached primary policy reads took 12.33 ms. Production latency is unmeasured. No packaged-app, human or deployed-fleet acceptance run. Live activation awaits BUZZ-512 and the staging verification tickets. Published with the three-layer stack in one atomic push through normal hooks. The published stack includes main `1055f79d048ef22c878d02a7afb04f49c753564f`, including access logging, storage reporting and mobile read-state changes. Access-slot integration and early HTTP refusal attribution belong to BUZZ-511; activation callback refusal close owners belong to BUZZ-512. Every changed layer requires review at its published head and new-head CI. Fresh trunk, remote membership, queue and immutable-lease guards passed before publication. All nine normal pre-push hook lanes passed; no hooks were bypassed. Remote heads, chained bases, membership, ready/draft states and native tracking were verified after publication. Trunk’s merged 0081 is unchanged. The parent upgrade expansion is 0082 and activation is 0083, with contiguous migrator versions, updated documentation and a populated-upgrade fixture starting at 81. Trunk’s storage-test repair supersedes our earlier local repair. Before publication, main advanced three commits from `1055f79d048ef22c878d02a7afb04f49c753564f` to `ba7d7825ea` (desktop deep-link settling and mobile pairing/sign-in). The normal branch-skew guard passed. Those later trunk commits are outside this validated published head; current-trunk CI and delta review remain required before merge. Generated with Codex --------- Signed-off-by: Brad Seiler <seiler@squareup.com> Co-authored-by: Honor <e0bf479eeebe47dde1e35705212a4df70616a7bc834d907ddfb18cf0405f8b1b@seiler-test.communities.public.builderlabstaging.xyz>
The read answer says plainly that any member with a file's link can open it, even from a private channel or a DM, that removing someone from a channel doesn't take a file back, and that Buzz's setting to limit files to their channels starts off when it ships (block/buzz#8444). The push sentence names what Block's gateway sees, and a third note says the relay's log records who read what (block/buzz#8376, in the pin). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
File reads check community membership only (block/buzz media.rs get_blob and head_blob, at the pin). The fix, block/buzz#8444, is still open and starts in log-only mode, so the paragraph tells people to keep restricted files out of Buzz until a release has the setting and it's on. It also says the relay's log names each member with what they read (#8376, in the pin). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
## Summary Removed private-channel members can currently download attachments if they know the URL. With `BUZZ_MEDIA_READ_MODE=enforce`, fresh GET, HEAD and Range requests check current membership and readable references on the primary database. Removed members receive 404; remaining members retain access, and rejoining restores access. Thumbnail aliases use the parent attachment’s permissions. Publication checks also prevent copied private URLs from granting new access. They cover attachments, link previews, profile pictures, banners, group pictures and emoji. Persona creation and changed-avatar edits, plus team and individual snapshot imports, transfer animated posters and animations under the agent’s signer before profile publication. Failed persona transfers preserve existing local records; import failures appear in the results. Authorization uses bounded queries, a two-second statement deadline and transaction-local JIT suppression. An index on edit targets avoids broad scans. Publications have a 512-reference limit, and capacity waits up to 500 ms before refusal. Protection defaults off. Existing downloaded bytes cannot be recalled. All modes stop creating immutable HTTP cache entries. ### Related issue Related approaches: #7565, #5185, #8451 and #8444. This change evaluates live stored references directly. ### Testing Published head: `3d1aa7431cc495a001a0f13e95e2e422c5a77ab1`, rebased on main `2cd40badb85de059a6916206e133da7a0a88fb2e`. Push hooks were **SKIPPED** with explicit author approval. Current-head HTTP, full package, PostgreSQL migration/media and profile-filter mutation checks are pending; CI is pending. The earlier complete HTTP journey passed at `f20b449262` (identical source tree), with its evidence attributed separately. Earlier failing preview/report-fixture runs are preserved; they do not count as passes. The HTTP journey covers upload/publication, fresh GET/HEAD/Range revocation and rejoin, emoji replacement, edited previews, persona visibility and non-reference decoys. The migration regression forces an index timeout, verifies no 0086 history or media tables, then proves a migrator through the base's 0085 can restart. Human attachment removal/rejoin testing passed on the pinned `1f08a8e0fd2edf19dc94321f7258518149fe1bf6` trial. Native animated persona edits and team/individual import acceptance remain incomplete. That old-snapshot database stays intact and cannot run the reconciled binary; a new trial needs a separate fresh database. ### Rollout prerequisites - Build and verify the reference index before shadow mode. Small databases can auto-migrate through one atomic 0086 migration within the configured timeouts. It builds the index before creating the media tables, so failure records no new migration version. Large deployments must disable auto-migration and prebuild indexes concurrently. - **Enforce remains blocked unless the operator can rule out SDK/CLI attachment edits and the unverified agent-harness edit producer.** SDK/CLI edits still omit attachment metadata. That behavior is deferred, not fixed. Removing this blocker requires latest-effective attachment metadata across successive edits, no restoration of removed files, and an upload → successive edits → GET/HEAD regression. - Validate client signer compatibility and split emoji publications above 512 references before enforce mode. - Complete the human workflows above in an isolated community with enforce enabled. - Follow the [index deployment guide](https://github.com/block/buzz/blob/honor/private-channel-media/docs/private-media-index-deployment.md) and [operator guidance](https://github.com/block/buzz/blob/honor/private-channel-media/docs/operator-private-media.md) for rollout and rollback. The [prebuild timeout finding](#9077 (comment)) and [avatar-transfer finding](#9077 (comment)) are implemented and independently reviewed. ### Schema change checklist - [x] Additive expand and code deployment; no backfill or contract phase. Prebuild/verify before shadow under either schema deployment path. - [x] No SQL function, trigger, procedure, rule or foreign key added. Signed events are untouched. - [x] No migration mutates existing rows. - [x] Migration and desired-state schema match. Catalog parity, target parsing, future partition inheritance and upgrade refusal are covered. - [x] Old/new binaries coexist for serving. Suspend community deletion during expansion and while rolled back to a binary with the old deletion catalog. - [x] Keep auto-migration disabled on old binaries after schema expansion; sqlx rejects unknown versions. - [x] Migration lock timeout 1 s; statement timeout 5 s. Small populated databases build inline within those budgets; large deployments use concurrent child-index prebuild/attachment with auto-migration off. Missing, invalid or wrong indexes fail verification. Generated with Codex --------- Signed-off-by: Brad Seiler <seiler@squareup.com> Co-authored-by: Honor <e0bf479eeebe47dde1e35705212a4df70616a7bc834d907ddfb18cf0405f8b1b@seiler-test.communities.public.builderlabstaging.xyz>
Why?
GET/HEAD /mediaonly checks community membership, so anyone in a community who learns an attachment's hash — which is in every media URL — can download a file posted in a private channel or DM they cannot read.What?
BUZZ_MEDIA_READ_MODEcontrols file access independently of authentication and community checks:offskips the new check,log-only(default) counts outcomes without denying reads, andenforceapplies it. The setting loads at startup, so mode changes need a rolling restart but no code rebuild. Unknown values fail startup.In enforce mode, GET and HEAD require a live share or uploader preview permission. A private file is readable only through an accessible channel; deleted or purged source messages grant nothing. Edit grants also require the original message to remain live. Author deletion and moderator removal revoke that message’s grants, including retained and edit-added attachments. Other undeleted shares keep working. Unauthorized reads return 404; lookup failures return 503 and are logged.
Community owners and admins who can view the moderation queue may also review a file attached to a live message with an open report. Each exceptional GET or HEAD requires a durable audit entry before storage access. Closing the report, deleting the logical message, or removing the queue role ends that exception; edit attachments follow the same rule. Report notes, unrelated hashes, blob-only reports, and other communities grant no access; reports create no public shares. Disabled or failed auditing blocks the exception.
The elected usage-poller leader reclaims inactive grants in bounded pages. It removes obsolete public profile/emoji grants and compacts inactive channel history to at most one row per community and file. That last row preserves the uploader-preview restriction after message deletion; it grants no read access. Cleanup uses community admission and skips quiescing/deleting communities.
Rollout
Before the first backfill, explicitly accept grants from reconsidered references or defer until refusal-preserving behavior exists. Then deploy recording on every pod, run
buzz-admin backfill-media-shares, and inspectbuzz_media_read_access_total{mode="log-only",outcome="deny"}andoutcome="error"before enforcing. The backfill now restores historical uploader proof from valid retained upload audits; missing evidence and unparsed references still need repair before enforcement. See operator instructions for mode changes, counters, and rollback.Any backfill run, including the first, can grant previously refused references after their authors gain access. Those grants persist across read-mode changes and later prevention of reevaluation. Every invocation restarts the scan; unwanted existing grants need a provenance-based repair before enforcement.
The conformance table and NIP-FI captured-header note now describe this policy. Share creation already requires upload proof or a readable share in #8451.
Related issue
Closes the known limitation noted in
conformance_multitenant.rsmedia_blossom. Read-enforcement half of the file-access stack after #8451.Testing
On an isolated relay backed by PostgreSQL 17, Redis, and a disposable S3 HTTP fixture, ran
RELAY_HTTP_URL=http://localhost:13071 cargo test -p buzz-test-client --test e2e_media_extended --test e2e_media_review -- --ignored --nocapture --test-threads=1withBUZZ_MEDIA_READ_MODE=enforce: all 29 tests passed.Repeated
test_media_read_rollout_policywith the mode unset and withoff: both passed, and Prometheus recorded two would-deny reads in the default log-only mode. The report journey submits a real NIP-56 report, verifies moderator GET/HEAD and their audit entries, rejects an unrelated attachment and another member, then confirms report closure revokes access. It also covers reported edit attachments and deletion of the original message while the report remains open. Removing the open-status SQL condition made the regression fail; restoring it passed.The actual Desktop edit/upload/profile workflow smoke test remains pending: an isolated native demo build started, but UI automation returned
cgWindowNotFoundbefore the edit flow could be exercised.All 29 live media journeys passed again with a five-second usage-poller interval. Deletion and profile journeys waited for periodic cleanup: old public grants disappeared, and deleted channel attachments remained 404 after compaction. The private-channel journey added then removed a member using real signed membership events, verifying GET and HEAD changed from 200 to 404 while the owner retained access. Removing the membership tombstone check made that regression fail. Removing the live-source check or deleting the final channel boundary also failed authorization regressions.
The fresh security workflow failed before analysis because checkout refused fork code in a privileged
pull_request_targetjob; the protection remains enabled, and this run provides no security review result.Generated with Codex
Linear: BUZZ-358