Skip to content

test(agents): refuse a real relay workflow mention in the harness kit - #2444

Merged
loganj merged 1 commit into
mainfrom
fix/kit-test-mention-auth
Nov 4, 2026
Merged

loganj merged 1 commit into
mainfrom
fix/kit-test-mention-auth

Conversation

@loganj

@loganj loganj commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🤖

Summary

  • main fails the JavaScript unit tests at src/features/agent-harness/kit.test.ts ("refuses a mention from anyone else, and triggers that are not its"). This PR makes it pass again without changing what agents accept.
  • The cause is two PRs that merged in an order neither one expected. feat(agents): give plugin agent types the Buzz side of a turn #2364 added the harness kit, which plugin agent types use to decide which messages start a turn. Its test was written against the rule from fix(security): close workflow, Canvas, upload, and broker gaps #2360, which refused every message that carried a workflow tag. fix(agents): restore delegated workflow mentions #2399 merged before it and restored the older rule, which accepts any message the agent's owner signs. The test's "workflow" message was signed by the owner, so it was accepted.
  • The kit's behavior has not changed. It still accepts only messages from the owner or an agent the owner attests. A real relay workflow message is signed by the community's workflow key, so the kit still refuses it. Only the test fixture was out of date.

Details

  • The fixture is now a real relay workflow mention: signed by the community's workflow key, owned by the agent's owner, and aimed at this agent. The bundled harnesses (Claude Code, Codex, Pi, Goose) would run this message.
  • Before checking the refusal, the test first checks that the bundled workflow check accepts the fixture. Without that step the negative test could pass for the wrong reason again. Giving the kit the bundled workflow check makes the test fail.
  • The test also checks that an owner mention with only a buzz:workflow marker is still accepted: the marker neither grants authority nor removes the owner's own. This covers the same fix as test(agents): align harness fixture with owner authorization #2445, so this PR includes it.
  • docs/workflows.md listed only Claude Code and Codex as accepting workflow mentions. It now lists Pi and Goose too, and says that plugin agent types do not.

Deliberate divergence

A workflow that mentions an agent of a plugin-provided type is ignored, while the same workflow wakes a Claude Code or Codex agent. This is intentional. #2399 restored workflow mentions for the bundled harnesses only to keep existing behavior working, and it says the template/substitution issue from #2360 is still open. Plugin agent types have never accepted workflow mentions, so nothing regresses. Accepting them would give these agent types a known open issue. If the team wants plugin agent types to accept workflow mentions after the follow-up discussion on workflow authorization, that change belongs in its own PR.

🤖 Generated with Claude Code

@loganj
loganj marked this pull request as ready for review November 4, 2026 17:17
@loganj
loganj requested a review from a team as a code owner November 4, 2026 17:17
The kit test "refuses a mention from anyone else, and triggers that are
not its" fails on main. Its workflow fixture was a message signed by the
owner with a `buzz:workflow` tag. #2364 wrote it against #2360, whose
`fromOwner` refused any event with that tag. #2399 merged first and
restored the older `fromOwner`, which admits any event the owner signs,
so the fixture became an owner message and the kit admitted it.

The kit itself still calls `fromOwner` without a workflow check, so a
real relay workflow mention is refused. Make the fixture that real
mention: signed by the community's workflow key, owned by the owner and
aimed at this agent, with the workflow key set on the session. The
bundled harnesses admit this event; the kit must not. Wiring the
bundled workflow check into the kit makes the test fail.

Before the refusal, assert that the bundled check admits the fixture,
so the negative case cannot pass for the wrong reason again. Also assert that an owner mention carrying only a
`buzz:workflow` marker is still admitted: the marker neither grants
authority nor revokes the owner's own. Say at the
call that the missing workflow check is deliberate, and fix the workflow
docs, which named only Claude Code and Codex.

Signed-off-by: Logan Johnson <loganj@squareup.com>
Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@loganj
loganj force-pushed the fix/kit-test-mention-auth branch from 237e436 to 82a496b Compare November 4, 2026 17:21

@joahg joahg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Approving. I checked this against main:

  • Cause: #2364's fixture is signed by the owner (alice), and fromOwner accepts the owner before it looks at any workflow tag. Since #2399 restored that rule, the "workflow" message was admitted. This PR fixes the fixture, not the kit.
  • The new refusal case is the right one to pin. kit.conversation calls fromOwner(trigger.event, agent.owner) without a WorkflowCheck. A relay workflow mention signed by the community's workflow key never reaches workflowMentionOwner, so the kit refuses it. The extra fromOwner(workflow, alice, { agent, authority }) → true assertion proves the fixture is a valid delegated mention. Without it, the refusal could just be a bad signature.
  • Documentation: the docs/workflows.md line and the kit comment record that plugin agent types don't accept delegated workflow mentions. That keeps the kit's (a workflow's never is) doc comment accurate.
  • CI: green, including JavaScript (1/4).

Follow-up, not a blocker: whether plugin harnesses should eventually accept owner workflow mentions the way the bundled ones do is a product call. Refusing them is the safer default while the template/substitution issue that #2399 reopened is unresolved.

@loganj
loganj merged commit e121e52 into main Nov 4, 2026
76 of 79 checks passed
@loganj
loganj deleted the fix/kit-test-mention-auth branch November 4, 2026 18:23
tulsi-builder added a commit that referenced this pull request Nov 4, 2026
Picks up #2444, which fixes the harness-kit workflow-mention test that
was failing on main.

Signed-off-by: tulsi <tulsi@block.xyz>
Co-authored-by: BasilBot <399581b6165ab01343e3a1324774b1024d76942aea563468f93c975e5ee5193c@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
johnmatthewtennant added a commit that referenced this pull request Nov 4, 2026
Co-authored-by: Martha <73923046810b68ae182000e74ef7c444383a907df20f81f2ba1008870a6f45a1@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* origin/main:
  test(agents): refuse a real relay workflow mention in the harness kit (#2444)
  fix(agents): catch up every event watch on the kinds it watches (#1852)
  Keep local artifact collections current (#1507)
  feat(members): add several people to a chat in one write (#2010)

Signed-off-by: John Tennant <jtennant@block.xyz>
johnmatthewtennant added a commit that referenced this pull request Nov 4, 2026
Co-authored-by: Martha <73923046810b68ae182000e74ef7c444383a907df20f81f2ba1008870a6f45a1@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* origin/main: (96 commits)
  test(agents): refuse a real relay workflow mention in the harness kit (#2444)
  fix(agents): catch up every event watch on the kinds it watches (#1852)
  Keep local artifact collections current (#1507)
  feat(members): add several people to a chat in one write (#2010)
  feat(solo): let viewers join read-only spaces (#2301)
  [3/5] Add protection to the Claude harness plugin (#1878)
  fix(settings): keep the bottom padding on long sections (#2361)
  fix(mermaid): cap diagram output size (#2231)
  feat(agents): give plugin agent types the Buzz side of a turn (#2364)
  feat(plugins): pin buzz://plugin-install links to a path and commit (#2315)
  feat(plugins): tint currentColor SVG page icons like host icons (#2314)
  feat(notifications): add a soft notification sound (#1536)
  feat(agents): let agents propose a profile picture (#2267)
  fix(plugins): default reminders and workflows off for fresh installs (#2291)
  Show all reactors in compact, separate emoji capsules (#2179)
  fix(panels): remember page split widths per workspace (#2264)
  feat(search): browse open channels from a Channels tab in search (#2164)
  perf(relay): page combined replay filters after a reconnect (#2308)
  fix(composer): make native Mac text shortcuts work past link chips (#2390)
  perf(relay): read creator proofs for many channels per filter (#2312)
  ...

Signed-off-by: John Tennant <jtennant@block.xyz>
johnmatthewtennant added a commit that referenced this pull request Nov 4, 2026
…est-work-activity

* origin/main:
  test(agents): refuse a real relay workflow mention in the harness kit (#2444)
  fix(agents): catch up every event watch on the kinds it watches (#1852)
  Keep local artifact collections current (#1507)
  feat(members): add several people to a chat in one write (#2010)
  feat(solo): let viewers join read-only spaces (#2301)
  [3/5] Add protection to the Claude harness plugin (#1878)
  fix(settings): keep the bottom padding on long sections (#2361)
  fix(mermaid): cap diagram output size (#2231)
  feat(agents): give plugin agent types the Buzz side of a turn (#2364)
  feat(plugins): pin buzz://plugin-install links to a path and commit (#2315)
  feat(plugins): tint currentColor SVG page icons like host icons (#2314)
  feat(notifications): add a soft notification sound (#1536)
wpfleger96 added a commit that referenced this pull request Nov 4, 2026
…w-tab

* origin/main:
  test(agents): refuse a real relay workflow mention in the harness kit (#2444)
  fix(agents): catch up every event watch on the kinds it watches (#1852)
  Keep local artifact collections current (#1507)
  feat(members): add several people to a chat in one write (#2010)
  feat(solo): let viewers join read-only spaces (#2301)
tulsi-builder added a commit that referenced this pull request Nov 4, 2026
Picks up #2444, which fixes the harness kit test failing on main.

Co-authored-by: BasilBot <399581b6165ab01343e3a1324774b1024d76942aea563468f93c975e5ee5193c@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: tulsi <tulsi@block.xyz>
johnmatthewtennant added a commit that referenced this pull request Nov 4, 2026
Signed-off-by: John Tennant <jtennant@block.xyz>
Co-authored-by: Damon <8a08290d889f1c68921542bfe9631d4cd671be9bea5f1114951f386819e4c10a@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* origin/main:
  feat(agents): identify plugins in native hook output (#2299)
  [4/5] Add protection to the Codex harness plugin (#1879)
  feat(search): show member counts on channel results (#2333)
  Fix the mute icon flashing when selecting a channel (#2287)
  feat(agents): open a working agent's transcript from its activity (#2276)
  test(agents): refuse a real relay workflow mention in the harness kit (#2444)
  fix(agents): catch up every event watch on the kinds it watches (#1852)
  Keep local artifact collections current (#1507)
  feat(members): add several people to a chat in one write (#2010)
bradseiler added a commit that referenced this pull request Nov 4, 2026
Co-authored-by: Honor <e0bf479eeebe47dde1e35705212a4df70616a7bc834d907ddfb18cf0405f8b1b@seiler-test.communities.public.builderlabstaging.xyz>

* origin/main:
  feat(agents): identify plugins in native hook output (#2299)
  [4/5] Add protection to the Codex harness plugin (#1879)
  feat(search): show member counts on channel results (#2333)
  Fix the mute icon flashing when selecting a channel (#2287)
  feat(agents): open a working agent's transcript from its activity (#2276)
  test(agents): refuse a real relay workflow mention in the harness kit (#2444)
  fix(agents): catch up every event watch on the kinds it watches (#1852)
  Keep local artifact collections current (#1507)
  feat(members): add several people to a chat in one write (#2010)

Signed-off-by: Brad Seiler <seiler@squareup.com>
wpfleger96 added a commit that referenced this pull request Nov 4, 2026
…-service

* origin/main: (73 commits)
  fix(bestie): deleting an agent saves no Bestie record where there was none (#2420)
  fix(unread): never treat agents-audience messages as unread (#1552)
  feat(focus): add an optional Focus plugin that quiets everything except Solo chats (#2277)
  [5/5] Add protection to the Goose harness plugin (#839)
  feat(agents): identify plugins in native hook output (#2299)
  [4/5] Add protection to the Codex harness plugin (#1879)
  feat(search): show member counts on channel results (#2333)
  Fix the mute icon flashing when selecting a channel (#2287)
  feat(agents): open a working agent's transcript from its activity (#2276)
  test(agents): refuse a real relay workflow mention in the harness kit (#2444)
  fix(agents): catch up every event watch on the kinds it watches (#1852)
  Keep local artifact collections current (#1507)
  feat(members): add several people to a chat in one write (#2010)
  feat(solo): let viewers join read-only spaces (#2301)
  [3/5] Add protection to the Claude harness plugin (#1878)
  fix(settings): keep the bottom padding on long sections (#2361)
  fix(mermaid): cap diagram output size (#2231)
  feat(agents): give plugin agent types the Buzz side of a turn (#2364)
  feat(plugins): pin buzz://plugin-install links to a path and commit (#2315)
  feat(plugins): tint currentColor SVG page icons like host icons (#2314)
  ...

# Conflicts:
#	src-tauri/src/host_process.rs
#	src-tauri/src/lib.rs
#	src/bundled/claude-code/claude.ts
#	src/bundled/codex/rpc.ts
#	src/bundled/goose/goose.ts
#	src/bundled/goose/runtime.ts
#	src/bundled/goose/sessions.test.ts
#	src/bundled/goose/sessions.ts
#	src/features/agent-harness/acp/process.ts
gilgtm added a commit that referenced this pull request Nov 4, 2026
* origin/main: (22 commits)
  feat(mentions): quieter mention menu that marks agents and their owners (#2328)
  feat: add a Canvas View tab, History diffs and a 256 KiB limit (#2318)
  feat(plugins): let local agents write and reload plugins (#2435)
  feat(onboarding): offer Create a community after public Builderlab sign-in (#2397)
  Open any Inbox filter view by route (#2394)
  fix(bestie): deleting an agent saves no Bestie record where there was none (#2420)
  fix(unread): never treat agents-audience messages as unread (#1552)
  feat(focus): add an optional Focus plugin that quiets everything except Solo chats (#2277)
  [5/5] Add protection to the Goose harness plugin (#839)
  feat(agents): identify plugins in native hook output (#2299)
  [4/5] Add protection to the Codex harness plugin (#1879)
  feat(search): show member counts on channel results (#2333)
  Fix the mute icon flashing when selecting a channel (#2287)
  feat(agents): open a working agent's transcript from its activity (#2276)
  test(agents): refuse a real relay workflow mention in the harness kit (#2444)
  fix(agents): catch up every event watch on the kinds it watches (#1852)
  Keep local artifact collections current (#1507)
  feat(members): add several people to a chat in one write (#2010)
  feat(solo): let viewers join read-only spaces (#2301)
  [3/5] Add protection to the Claude harness plugin (#1878)
  ...

Signed-off-by: Gil Birman <gil@squareup.com>
Co-authored-by: Claude <noreply@anthropic.com>
kazemek21 added a commit that referenced this pull request Nov 4, 2026
## TL;DR
Solo chats retain explicit model and effort choices per chat and agent
until Reset, including choices made before the first message. Untouched
chats keep provider defaults.

Choices use a feature-owned IndexedDB transaction for
read/merge/compare/write. The existing composer catalog, menus and
scoped draft state remain their owners; no generic storage framework or
second authoritative localStorage copy is added. IndexedDB is the sole
authority: a missing row means no saved choice. This unreleased feature
does not import earlier development-only localStorage bytes; those bytes
remain untouched and non-authoritative. Scope cleanup shares the
save/transfer ownership lock so an already-queued save cannot recreate
deleted choices. Other windows are notified only after commit and reread
the authoritative store.

Send waits for initial loading and pending-save acknowledgement. Failed
reads offer **Retry saved choices** rather than silently using defaults;
failed saves keep local intent. First-send transfer and refused-start
restoration use ownership receipts, so a later edit or equal-value Reset
is not overwritten. The existing toast retains the refused prompt,
attachments and validated choices for **Retry restoring message**,
including repeated ownership/storage failure.

## Verification
Blox verified an uploaded content snapshot matching
`6f87b1101d21f3071cf0a64ab0197ae1442a4429`: typecheck, targeted lint,
**622 tests across the three affected full files**, and **14 browser
cases** across Chromium/WebKit (zero skipped/flaky). All six final
changed-file hashes match the tested snapshot. Historical `a004b3cd`
verified 709 tests across seven files plus design checks; those broader
results are not claimed as a new-head run. These are content-snapshot
checks, not claims that the candidate commit was checked out remotely.
Pre-commit hooks ran; only duplicate types/tests/design/clippy pre-push
jobs were skipped because required CI runs them. The verification box
was deleted.

The latest feedback regressions fail against `a004b3cd` and pass with
these fixes: a refused prompt-write reports the text-save cause,
compensates prepared choices, and succeeds through Retry; fresh IDB
initialization remains available when the unrelated localStorage read
throws; and scope purge waits behind already-queued saves in both
browser engines.

The [failed WebKit
job](https://github.com/block/buzz-app/actions/runs/41872987261/job/126725503435)
was not dismissed as a flake: an owned amd64 trace showed the second Web
Lock holder reading stale localStorage after the first writer committed,
then replacing the durable map. The transactional repair preserves both
agents in the unchanged two-page regression. Mounted tests cover
pending/failed edits, Send commit barriers, unavailable types,
navigation and refused-start recovery. The latest regressions preserve
corrective edit errors during a saved-choice read failure and retain
valid first-send choices when the catalog needs no selector. Both fail
with their previous production behavior and pass with the fixes.
Removing the later-Reset ownership guard produces the intended
regression failure; the restored source passes.

**Browser cases: four added, none removed** versus the merge base; this
feedback batch adds one and updates one. They cover
keyboard/Escape/Reset/focus, actual cross-page commits/notifications,
real IDB initialization/abort/first-send compare-clear custody, and Web
Lock queue ordering for purge. These browser boundaries cannot be
established by jsdom; lifecycle and failure permutations remain in
mounted tests. Shared menus, Button, Toast and LoadingRegion are reused
without new styling. Prior picker checks covered both themes, 360/1280px
widths, normal/24px text, keyboard and pointer: [before
selection](https://d24qwcpro867f5.cloudfront.net/repos/block/buzz-app/prs/2342/solo-choices-before-03b33b1.png),
[selected
choices](https://d24qwcpro867f5.cloudfront.net/repos/block/buzz-app/prs/2342/solo-choices-selected-03b33b1.png).

**Boundaries:** Cooperation is within the same storage bucket, not older
clients or separate profiles. Retry retention ends when this window
reloads or its notice is dismissed; it is not crash recovery. Native
simultaneous WKWebView/provider execution and full-chat runtime proof
remain unverified. Earlier failed attempts and test-contamination
evidence remain retained; passing repetitions do not establish that
every browser failure is fixed. The separate JavaScript fixture failure
was repaired on main by
[#2444](#2444); this branch
includes that merged source, not an unmerged human patch.

## Related
Fixes NEXUS-4155

This is native host Solo composer behavior, separate from plugin task
setup and shared-channel default routing. It does not depend on
[#2306](#2306) or
[#1909](#1909).

Part of [Nexus & Agent Tooling
status](https://g2.sqprod.co/apps/kylez-nexus-buzz-plugins-status).

_Generated with Codex (gpt-6.1-sol, high) on behalf of kylez._

Signed-off-by: Kyle Zemek <kylez@squareup.com>
Co-authored-by: Codex <noreply@openai.com>
bradseiler added a commit that referenced this pull request Nov 4, 2026
## Why

Buzz offers direct merge actions even when a pull request’s target
branch requires GitHub’s merge queue, so choosing Squash and merge
fails.

## What

Show Queue for merge and its confirmation on queue-required branches,
then display Queued for merge after GitHub accepts the request. Other
branches retain their existing merge options.

## How

Read GitHub’s target-branch queue flag and current queue membership with
the existing checklist. Enqueue the confirmed head commit, hide direct
merge methods for queue-required branches, and invalidate confirmations
when the head, base, or queue requirement changes. GitHub validates
queue admission; rejection stays visible for retry.

## Risk

Limited to the bundled GitHub checklist and merge actions. No live
GitHub enqueue was performed during agent testing.

## Testing

Exercised the existing GitHub actions fixture in Chromium and WebKit
through queue confirmation and queued status, in both themes at 1100px
and 390px with enlarged text, using pointer and keyboard activation.
Read-only GitHub GraphQL introspection and a PR query confirmed the
queue fields and expectedHeadOid input.

Try `/tests/fixtures/github-actions.html?ready&queue` on this worktree’s
dev server: choose Queue for merge, confirm, and expect Queued for
merge. Native app behavior and human acceptance remain deferred. The
full local Vitest run was stopped after failures and timeouts outside
the GitHub plugin, including Settings, Inbox, message reactions, profile
navigation and relay discovery; broad validation remains incomplete. The
first pre-push run timed out in app page composition while that broad
run was active; the unchanged page-composition test passed in 8 seconds
after the broad run stopped. No automated browser cases added or
removed; queue state/failure/confirmation regressions are covered in
colocated component tests.

Updated with main’s agent-harness test fix from #2444 to clear the
existing baseline CI failure. The merge-queue feature diff is unchanged;
CI is rerunning on the updated branch.

Generated with Codex

Signed-off-by: Brad Seiler <seiler@squareup.com>
Co-authored-by: Honor <e0bf479eeebe47dde1e35705212a4df70616a7bc834d907ddfb18cf0405f8b1b@seiler-test.communities.public.builderlabstaging.xyz>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants