[lukemelia]

What this does

The dashboard's Your Workspaces section lists the realms a user has a permission row of their own for: realms they own, and realms shared with them. Realms that anyone can read stay reachable by URL, but they don't appear there.

The Catalogs section shows two kinds of realm:

• Operator catalogs. These are public realms an operator has flagged, and every signed-in user sees them. The Boxel catalog is flagged, and it always leads the section.
• The user's own picks. These are public realms the user added from a gear beside the "Catalogs" heading.

Full-quality video (MP4)

How it works

Permissions enumeration. fetchUserPermissions returns the user's own permission rows by default. Pass includePublicRealms: true to add the '*'-readable realms. Every caller that authorizes, renders or indexes on a user's behalf passes it: multi-realm authorization, download, publish, skill validation, run-command, capture, the indexer, prerender-html and definition lookup. Callers that pass userId: '*' read the public rows directly and are unaffected.

POST /_realm-auth takes ?include-public=true|false. With false it mints sessions only for the caller's own realms, which is what the host's dashboard list asks for. With true it also mints sessions for every world-readable realm, which is what boxel-cli's realm-token fetch and realm list, the host's login-time token fetch and the load harness ask for. Without the parameter it includes public realms, so boxel-cli versions already installed keep their public-realm sessions.

Operator catalogs are the realms flagged realm_metadata.show_as_catalog = true (the existing column):

• POST /_grafana-set-realm-catalog?realm=…&catalog=true|false sets the flag. It's behind Grafana-secret auth, and it refuses realms that aren't publicly readable, published realms, and URLs that aren't realms on this server. It clears the realm's cached _info locally and on peer instances.
• The Grafana Realms dashboard has an operator panel that shows the selected realm's public and catalog status, with Add and Remove buttons.
• /_catalog-realms selects catalogs by the flag in SQL, so it fetches _info only for the flagged realms. It caches the list for 60 seconds, so a change reaches every instance within a minute without a restart. Requests that find the cache expired share one refresh, and a refresh that a reset overtakes doesn't write its stale list back.

A user's picks:

• GET /_public-realms (signed-in users only) lists the active, world-readable source realms with their names and icons. The names and icons come from each realm's indexed realm.json, so listing doesn't mount any realm.
• The picks persist in app.boxel.catalog-realms Matrix account data, like workspace favorites. They're loaded at boot after the server catalogs and the user's own realms, so no realm is listed twice. The account-data event keeps them in sync across tabs and devices.
• The picker is a Pret UI Dialog with a multi-select List. It offers every public realm except the base realm, the operator catalogs and the user's own realms, and checks the current picks. Users can add and remove their own picks; operator catalogs always show.
• The host keeps the raw pick list apart from the realms it displays. A pick that another list also shows (an operator catalog, or one of the user's own realms) is listed there, and it comes back as a pick when that list stops showing it. Saving the picker never drops such a pick.
• At boot, a favorite that no dashboard list shows but that is a public realm becomes a catalog pick, so the favorite stays visible.
• With nothing to show, the Catalogs section says how to add catalogs.

Compatibility

No client breaks: a request without include-public still gets sessions for public realms. This PR's title releases a boxel-cli that sends include-public=true explicitly.

Out of scope: making own-realms-only the parameterless default. Every client in this repo names the parameter, so that's a server-only change. It's safe once installed CLIs send the parameter.

Tests

• realm-server:
  • queries-test: fetchUserPermissions with and without includePublicRealms.
  • realm-auth-test: _realm-auth without include-public, and with it set to true and to false.
  • server-endpoints/user-and-catalog-test: /_public-realms listing and auth, plus the Grafana catalog endpoint's add and remove, validation and secret check, its effect on /_catalog-realms, and a change made during a refresh.
• host:
  • Workspace-chooser acceptance: adding a realm, removing it, cancelling, a pick surviving an overlap with another list, adopting a favorited public realm, and the empty Catalogs section.
  • Boot-assembly integration: a public-only realm isn't listed as the user's own but still gets a session.
  • A unit test for the rule that the Boxel catalog leads the Catalogs section.
• software-factory: the entrypoint integration test's realm-server mock matches /_realm-auth by path, so the CLI's query string reaches it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PWtpTuSscLnbf84a2c96R9

[github-actions]

Observability diff (vs staging)

Show diff

diff --git a/tmp/remote-canon.drN0ri/dashboards/boxel-status/realms.json b/tmp/committed-canon.gWm9jb/dashboards/boxel-status/realms.json
index 0f281ca..a121833 100644
--- a/tmp/remote-canon.drN0ri/dashboards/boxel-status/realms.json
+++ b/tmp/committed-canon.gWm9jb/dashboards/boxel-status/realms.json
@@ -306,6 +306,10 @@
           "uid": "cef5v5sl9k7i8f"
         },
         "description": "Operator action: grant the chosen permission level to a Matrix user on this realm. Pick from existing users (sourced from the `users` table) or type a new Matrix ID like `@user:matrix.host`. POSTs to the realm-server with `Authorization: Bearer ${grafana_secret}` (token from SSM at apply time, CS-10929).",
+        "fieldConfig": {
+          "defaults": {},
+          "overrides": []
+        },
         "gridPos": {
           "h": 8,
           "w": 24,
@@ -758,6 +762,10 @@
           "uid": "cef5v5sl9k7i8f"
         },
         "description": "Operator action: queue a from-scratch reindex for the realm currently selected in the **Realm** picker (template variable `${realm_url}`). Live blast-radius (pending / in-flight / oldest pending) is fetched every refresh; the button disables itself while an indexing job is already in flight. Click POSTs to `/_grafana-reindex?realm=...` with `Authorization: Bearer ${grafana_secret}` (substituted from SSM at apply time, CS-10929). System-wide \"Reindex ALL realms\" lives on the Indexing dashboard.",
+        "fieldConfig": {
+          "defaults": {},
+          "overrides": []
+        },
         "gridPos": {
           "h": 8,
           "w": 24,
@@ -937,6 +945,183 @@
         "title": "Operator Actions: Reindex this realm",
         "type": "volkovlabs-form-panel"
       },
+      {
+        "datasource": {
+          "type": "grafana-postgresql-datasource",
+          "uid": "cef5v5sl9k7i8f"
+        },
+        "description": "Operator action: add the realm selected in the **Realm** picker (template variable `${realm_url}`) to the Catalogs section of every signed-in user's dashboard, or take it out. Only public realms (a `*` read grant) can be catalogs; every other public realm stays reachable by URL without being listed. The choice persists in `realm_metadata.show_as_catalog`. Clicks POST to `/_grafana-set-realm-catalog?realm=...&catalog=true|false` with `Authorization: Bearer ${grafana_secret}`. Other realm-server instances serve the change within a minute.",
+        "fieldConfig": {
+          "defaults": {},
+          "overrides": []
+        },
+        "gridPos": {
+          "h": 7,
+          "w": 24,
+          "x": 0,
+          "y": 34
+        },
+        "id": 101,
+        "options": {
+          "buttonGroup": {
+            "orientation": "center",
+            "size": "md"
+          },
+          "confirmModal": {
+            "body": "Please confirm the action.",
+            "cancel": "Cancel",
+            "columns": {
+              "include": [
+                "name",
+                "newValue"
+              ],
+              "name": "Field",
+              "newValue": "Value",
+              "oldValue": "Previous"
+            },
+            "confirm": "Confirm",
+            "elementDisplayMode": "modified",
+            "title": "Confirm operator action"
+          },
+          "elements": [
+            {
+              "fieldName": "public_readable",
+              "id": "public_readable",
+              "labelWidth": 28,
+              "queryField": {
+                "refId": "A",
+                "value": "public_readable"
+              },
+              "section": "current",
+              "title": "Public",
+              "tooltip": "Whether the realm has a `*` read grant. Only public realms can be catalogs.",
+              "type": "disabled"
+            },
+            {
+              "fieldName": "in_catalog",
+              "id": "in_catalog",
+              "labelWidth": 28,
+              "queryField": {
+                "refId": "A",
+                "value": "in_catalog"
+              },
+              "section": "current",
+              "title": "In Catalogs",
+              "tooltip": "Whether realm_metadata.show_as_catalog is set, which lists the realm in every signed-in user's Catalogs section.",
+              "type": "disabled"
+            },
+            {
+              "buttonLabel": "Add ${realm_url:text} to Catalogs",
+              "customCode": "const realm = ${realm_url:doublequote};\nif (!window.confirm('Add to Catalogs: ' + realm + '?')) { return; }\n(async () => {\ntry {\n  const r = await fetch(${realm_server:doublequote} + '_grafana-set-realm-catalog?realm=' + encodeURIComponent(realm) + '&catalog=true', { method: 'POST', headers: { 'Authorization': 'Bearer ' + ${grafana_secret:doublequote} } });\n  if (r.ok) {\n    context.grafana.notifySuccess(['Added to Catalogs', (await r.json()).message]);\n    if (typeof context.grafana.refresh === 'function') { context.grafana.refresh(); }\n  } else {\n    const txt = await r.text();\n    context.grafana.notifyError(['Added to Catalogs failed', 'HTTP ' + r.status + ': ' + txt]);\n  }\n} catch (err) {\n  context.grafana.notifyError(['Added to Catalogs failed', String(err)]);\n}\n\n})().catch(function (err) { console.error('operator action handler error:', err); });",
+              "disableIf": "return (context.panel.elements.find(function(e){return e.id==='public_readable';})||{}).value !== 'yes' || (context.panel.elements.find(function(e){return e.id==='in_catalog';})||{}).value === 'yes';",
+              "id": "btn_add_to_catalog",
+              "labelWidth": 28,
+              "section": "actions",
+              "show": "form",
+              "size": "md",
+              "title": "",
+              "tooltip": "POST /_grafana-set-realm-catalog?realm=${realm_url}&catalog=true. Disabled unless the realm is public and not already in Catalogs.",
+              "type": "button",
+              "value": "",
+              "variant": "primary"
+            },
+            {
+              "buttonLabel": "Remove ${realm_url:text} from Catalogs",
+              "customCode": "const realm = ${realm_url:doublequote};\nif (!window.confirm('Remove from Catalogs: ' + realm + '?')) { return; }\n(async () => {\ntry {\n  const r = await fetch(${realm_server:doublequote} + '_grafana-set-realm-catalog?realm=' + encodeURIComponent(realm) + '&catalog=false', { method: 'POST', headers: { 'Authorization': 'Bearer ' + ${grafana_secret:doublequote} } });\n  if (r.ok) {\n    context.grafana.notifySuccess(['Removed from Catalogs', (await r.json()).message]);\n    if (typeof context.grafana.refresh === 'function') { context.grafana.refresh(); }\n  } else {\n    const txt = await r.text();\n    context.grafana.notifyError(['Removed from Catalogs failed', 'HTTP ' + r.status + ': ' + txt]);\n  }\n} catch (err) {\n  context.grafana.notifyError(['Removed from Catalogs failed', String(err)]);\n}\n\n})().catch(function (err) { console.error('operator action handler error:', err); });",
+              "disableIf": "return (context.panel.elements.find(function(e){return e.id==='in_catalog';})||{}).value !== 'yes';",
+              "id": "btn_remove_from_catalog",
+              "labelWidth": 28,
+              "section": "actions",
+              "show": "form",
+              "size": "md",
+              "title": "",
+              "tooltip": "POST /_grafana-set-realm-catalog?realm=${realm_url}&catalog=false. Disabled unless the realm is in Catalogs.",
+              "type": "button",
+              "value": "",
+              "variant": "secondary"
+            }
+          ],
+          "initial": {
+            "code": "",
+            "contentType": "application/json",
+            "getPayload": "return {};",
+            "highlight": false,
+            "method": "query",
+            "payload": {}
+          },
+          "layout": {
+            "orientation": "horizontal",
+            "padding": 10,
+            "sectionVariant": "default",
+            "sections": [
+              {
+                "id": "current",
+                "name": "Current state"
+              },
+              {
+                "id": "actions",
+                "name": "Actions"
+              }
+            ],
+            "variant": "split"
+          },
+          "reset": {
+            "backgroundColor": "purple",
+            "foregroundColor": "yellow",
+            "icon": "process",
+            "text": "Refresh",
+            "variant": "hidden"
+          },
+          "resetAction": {
+            "code": "",
+            "confirm": false,
+            "contentType": "application/json",
+            "getPayload": "return {};",
+            "method": "-",
+            "mode": "initial",
+            "payload": {}
+          },
+          "saveDefault": {
+            "icon": "save",
+            "text": "Save Default",
+            "variant": "hidden"
+          },
+          "submit": {
+            "backgroundColor": "purple",
+            "foregroundColor": "yellow",
+            "icon": "cloud-upload",
+            "text": "Submit",
+            "variant": "hidden"
+          },
+          "sync": false,
+          "update": {
+            "code": "",
+            "confirm": false,
+            "contentType": "application/json",
+            "getPayload": "return {};",
+            "method": "-",
+            "payload": {},
+            "payloadMode": "all"
+          },
+          "updateEnabled": "disabled"
+        },
+        "pluginVersion": "6.2.0",
+        "targets": [
+          {
+            "datasource": {
+              "type": "grafana-postgresql-datasource",
+              "uid": "cef5v5sl9k7i8f"
+            },
+            "editorMode": "code",
+            "format": "table",
+            "rawQuery": true,
+            "rawSql": "SELECT\n  '${realm_url}' AS realm,\n  CASE WHEN EXISTS (\n    SELECT 1 FROM realm_user_permissions\n     WHERE realm_url = '${realm_url}' AND username = '*' AND read = true\n  ) THEN 'yes' ELSE 'no' END AS public_readable,\n  CASE WHEN COALESCE(\n    (SELECT show_as_catalog FROM realm_metadata WHERE url = '${realm_url}'),\n    false\n  ) THEN 'yes' ELSE 'no' END AS in_catalog;",
+            "refId": "A"
+          }
+        ],
+        "title": "Operator Actions: Catalogs section",
+        "type": "volkovlabs-form-panel"
+      },
       {
         "datasource": {
           "type": "grafana-postgresql-datasource",
@@ -1022,7 +1207,7 @@
           "h": 10,
           "w": 24,
           "x": 0,
-          "y": 34
+          "y": 41
         },
         "id": 8,
         "options": {
@@ -1327,7 +1512,7 @@
           "h": 10,
           "w": 24,
           "x": 0,
-          "y": 44
+          "y": 51
         },
         "id": 9,
         "options": {

(Run: https://github.com/cardstack/boxel/actions/runs/38337865588)

[github-actions]

Preview deployments

• Host staging preview

• Host production preview

Host Test Results

    1 files  ±0      1 suites  ±0   1h 35m 45s ⏱️ - 15m 4s
5 428 tests ±0  5 417 ✅ ±0  11 💤 ±0  0 ❌ ±0 
5 443 runs  ±0  5 432 ✅ ±0  11 💤 ±0  0 ❌ ±0 

Results for commit e27eebb. ± Comparison against earlier commit da52d31.

Realm Server Test Results

    1 files  ±0    319 suites  ±0   1h 51m 46s ⏱️ + 1m 38s
5 171 tests ±0  5 171 ✅ ±0  0 💤 ±0  0 ❌ ±0 
5 230 runs  ±0  5 230 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit e27eebb. ± Comparison against earlier commit da52d31.

[lukemelia]

[Claude Code 🤖] Self-review. Fixes for each comment below are coming in follow-up commits on this PR, and each thread will get a reply naming its commit.

[lukemelia]

[Claude Code 🤖] Race: a refresh that's already in flight when /_grafana-set-realm-catalog calls resetCatalogRealms() writes its now-stale list back here with a fresh fetchedAt. The instance that just confirmed "Added" can then serve the old list for up to 60s. The write needs to be skipped when the cache was reset after the refresh started.

[lukemelia]

[Claude Code 🤖] Fixed in 74a0d17. Every reset bumps a generation, and a refresh only writes its list to the cache if no reset happened while it ran. A new test holds a refresh at its _info call, removes the realm, and checks that the next read leaves it out. That test fails against the previous handler.

[lukemelia]

[Claude Code 🤖] Concurrent refreshes: when the cache expires, every request in that moment re-runs the DB query and the per-realm _info calls. /_catalog-realms is hit on every host boot, so this should share one in-flight refresh promise. That also gives a natural place to fix the race at line 60.

[lukemelia]

[Claude Code 🤖] Fixed in 74a0d17. Requests that find the cache expired wait on one shared in-flight refresh, so each expiry reads the database and the catalogs' _info once per instance.

[lukemelia]

[Claude Code 🤖] Picks can be lost. The user's picks only exist as user-catalog entries in availableRealms, and they're dropped whenever they overlap another list:

• A pick that an operator also makes a catalog is never stored as user-catalog. The picker doesn't offer it, so the next Save writes the picks without it, and it's gone once the operator un-catalogs it.
• A pick that's shared with the user is spliced out by setAvailableRealmIdentifiers. When the share is revoked, it shows nowhere until a reload.

Keeping the raw pick list separately and deriving the displayed entries from it fixes both. Save should also keep picks that are only hidden because another list shows them.

[lukemelia]

[Claude Code 🤖] Fixed in da52d31. RealmServerService keeps the raw pick list separately and rebuilds the user-catalog entries from it whenever the user's realms or the server catalogs change, so a pick hidden by an overlap comes back when the overlap ends. The picker's Save keeps picks it didn't offer because another list shows them, and drops only picks that are no longer public. A new acceptance test covers the share-then-revoke case.

[lukemelia]

[Claude Code 🤖] Empty section: the Catalogs section now always renders, but with no catalogs and no picks it's a bare heading over an empty list. It needs an empty-state line that points at the gear.

[lukemelia]

[Claude Code 🤖] Fixed in da52d31. With nothing to show, the section reads "No catalogs yet. Add public realms with the gear." The hosted-only filter keeps its "No matching results". A new acceptance test covers it.

[lukemelia]

[Claude Code 🤖] Favorites disappear. Existing users lose the non-catalog public realms that _realm-auth used to put in Your Workspaces. A favorited one also drops out of Favorites, because favoriteRealmIdentifiers filters against the displayed lists. Favorites that are public realms not otherwise listed should be migrated into the user's catalog picks once, so they stay visible.

[lukemelia]

[Claude Code 🤖] Fixed in da52d31. At boot, a favorite that no dashboard list shows but that /_public-realms lists becomes a catalog pick, so it stays in Catalogs and in Favorites. The check makes a request only when such a favorite exists, and favorites that aren't public are left alone. A new acceptance test covers both cases.

[lukemelia]

[Claude Code 🤖] Duplicate type: this redefines the PublicRealmSummary that runtime-common exports alongside fetchPublicRealmSummaries, so the /_public-realms row shape is declared twice. Import the runtime-common type instead.

[lukemelia]

[Claude Code 🤖] Fixed in da52d31. The host imports PublicRealmSummary from runtime-common.

[lukemelia]

[Claude Code 🤖] Duplicated SQL: this branch repeats the UNION's first arm word for word, so a future filter change has to be made twice. Build the own-rows fragment once, and add the public arm only when includePublicRealms is set.

[lukemelia]

[Claude Code 🤖] Fixed in 8e9e56a. The own-rows arm is built once, covering onlyOwnRealms too, and the public arm is appended only when includePublicRealms is set.

[richardhjtan]

[Claude Code 🤖] Reviewed the permission-enumeration default flip, /_realm-auth consumers, catalog flagging and the pick lifecycle; the host UI was read, not exercised.

Two blocking items: CI is red because of this change, and the /_realm-auth default breaks every published boxel-cli.

1. Software Factory shard 1 fails on this change (regression, blocking). packages/software-factory/tests/factory-entrypoint.integration.test.ts mocks the realm server with request.url === '/_realm-auth'. boxel-cli's getRealmTokens now requests /_realm-auth?include-public=true, so the mock answers 404 and factory:go --debug prints a structured JSON summary fails with Realm auth lookup failed: 404 Unexpected request: POST /_realm-auth?include-public=true. Match on the pathname instead (new URL(request.url, 'http://x').pathname === '/_realm-auth').
2. Default direction of include-public (decision, blocking). See the inline comment on handle-realm-auth.ts.