Skip to content

feat(website): the .t27 catalog syncs itself from GitHub, nightly - #1021

Merged
gHashTag merged 4 commits into
mainfrom
site/catalog-github-sync
Sep 15, 2026
Merged

gHashTag merged 4 commits into
mainfrom
site/catalog-github-sync

Conversation

@gHashTag

@gHashTag gHashTag commented Sep 15, 2026

Copy link
Copy Markdown
Owner

The catalog behind the Queen map and the Spec Explorer had been frozen at 856 specs since 2026-09-12 while GitHub held far more. Three things had to be true at once, and all three are fixed here: the founding sync read a laptop, the nightly workflow never ran it and died at an unrelated gate, and its push to main is refused by the work-report ruleset.

1407 specs across 10 repositories, from GitHub, with no machine of anyone's in the path.

{
  "version": 1,
  "head_sha": "604c0186d2c7bc40e8bf6790ff0ec99912d6274a",
  "summary": "The .t27 catalog now refreshes itself from GitHub nightly: the founding sources are pulled as tarballs instead of read from a developer's working copy, the scheduled workflow runs that sync and lands its result through a pull request, and the gate that had been failing every night since 2026-09-13 now asserts the invariant it states.",
  "changes": [
    "scripts/sync-t27-specs.mjs pulls gHashTag/t27 from a GitHub tarball at a named ref instead of reading a working copy at T27_ROOT, which CI does not have; the local checkout is now opt-in through T27_LOCAL=1 for compiler work.",
    "The teaching course, whose nine specs live on a branch never merged to master, is vendored through a named overlay ref that contributes only the paths the main ref lacks, so the overlay contributes nothing once the course is merged.",
    "The compiler wasm is reused from the copy already vendored beside the corpus when no cargo build is present, so a spec refresh no longer requires a Rust toolchain and no longer silently swaps the compiler.",
    "Files a spec names in SOURCES or BODY_EN that are not .t27 are vendored from the same tarball at the same commit, which restores the seven docs/system markdown documents a docs chapter renders.",
    "qa/tools-spec-contract.mjs asserts that every MCP server lists tools or is marked external, per its own message, instead of summing the two counts as though they were disjoint; mcp/inngest-dev is truthfully both.",
    ".github/workflows/t27-world-scan.yml runs the founding sync as its first step, and commits to a branch, generates a work report from the artifacts it wrote, opens or updates a pull request and enables auto-merge instead of pushing to main, which ruleset 23148303 refuses.",
    "Duplicate copies are recorded as {path, repo, sameAs} pairs by both vendor paths and carried through the manifest rebuild, and the Spec Explorer shows a spec's other homes.",
    "The embedded Spec Explorer lifts its course filter whenever an address names a spec, so the homepage frame no longer reads 9 specs beside a map reading 856.",
    "The .spec-x-brief box no longer shrinks below its content in either variant: measured at 404px it was a 0px box around 636px of content drawing over the tab strip and the code.",
    "The apex deploy workflow copied a fixed list of four data directories while the build produces ten, so the six generated catalogs behind the Explorer tabs were live only because a person had copied them; it now iterates what the build produced.",
    "A failed universe-atlas.json fetch used to replace the whole Queen shell, taking all thirteen tabs down with the map; the shell now mounts without the atlas, as it already does for a connected repository, and the failure is reported over it with a retry.",
    "The catalog itself moved from 856 specs across 8 repositories to 1407 across 10, adding tt-trinity-euler and turbobaby-user-bot and attributing chips/euler to the repository that owns it."
  ],
  "tests": [
    {
      "command": "npm run check:discovery check:shared-core check:spec-catalog check:queen-atlas check:queen-catalog check:queen-continuous check:queen-spec-mirrors check:skills-catalog check:crons-catalog check:agents check:tools check:docs",
      "result": "all twelve catalog gates pass against this tree",
      "status": "passed",
      "evidence": "run with BOT_ROOT and T27_ROOT pointed at a nonexistent path so the sibling-checkout branches behave as they do on a runner"
    },
    {
      "command": "node scripts/sync-t27-specs.mjs with T27_ROOT unset",
      "result": "the founding corpus was vendored from GitHub with no local checkout",
      "status": "passed",
      "evidence": "manifest generatedFrom.source reads github tarball gHashTag/t27@master, wasmSource reads the copy already vendored here, and the overlay contributed 11 paths not on master"
    },
    {
      "command": "npm run discover -- scan && npm run discover -- vendor && npm run core -- index && npm run atlas -- scan && npm run atlas -- build",
      "result": "the full nightly chain ran end to end and produced 1407 specs across 10 repositories",
      "status": "passed",
      "evidence": "atlas reports 10 worlds, 1167 issues, 167 shared opportunities; shared-core reports 1407 specs and 91 collisions"
    },
    {
      "command": "npm run typecheck:ratchet",
      "result": "179 errors across 26 files, unchanged from the baseline",
      "status": "passed",
      "evidence": "the runner printed: no file gained type errors"
    },
    {
      "command": "npx vite build",
      "result": "the production build completed without errors",
      "status": "passed",
      "evidence": "built in 21.57s after the final gate run"
    },
    {
      "command": "browser measurement of the Spec Explorer at 404 and 1280 CSS pixels",
      "result": "the brief no longer draws over the tab strip at either width",
      "status": "passed",
      "evidence": "at 404px the brief is 405px with its 636px of content scrolling and the tab strip at y=583; at 1280px it is 308px and uncapped, as before"
    },
    {
      "command": "npm run lint",
      "result": "ESLint reports 180 problems, the same count as origin/main",
      "status": "failed",
      "evidence": "the identical 180 problems appear with this branch's source reverted, so the count is pre-existing and untouched here"
    }
  ],
  "limitations": [
    "The duplicate-pair list ships empty in this commit: the pairs were recorded by the founding sync and then dropped by the vendor step, and that ordering was fixed after the corpus had already been rebuilt. The next scheduled refresh writes them.",
    "Health is what the vendored compiler said at scan time, not a claim that a spec is correct or that its generated code was run.",
    "Eleven t27 specs, the teaching course among them, reach the site only through a named overlay ref; a reader cloning master alone will not find them.",
    "The scan reads public repositories only, so a private repository writing .t27 specs stays invisible to the map.",
    "Auto-merge on the nightly pull request depends on two repository settings that were enabled for this, and it will fall back to leaving the pull request open if either is turned off again."
  ],
  "tags": ["t27", "catalog", "automation", "specs", "website"],
  "blog": {
    "title": "The catalog that could not refresh itself",
    "summary": "A nightly job had been running for three days, rebuilding the spec catalog correctly every time and throwing the result away, while the repository holding two thirds of the corpus was the one repository it could not read at all.",
    "outline": [
      "The symptom was that the map and the library disagreed with GitHub, and the first instinct is to look for a data divergence between the two artifacts the two screens read; they agreed exactly, and shared a manifest hash, so the divergence was between both of them and the source.",
      "The founding sources were pulled from a working copy at a hard-coded path on one developer's machine, which meant that no runner could ever refresh them, and that whatever branch the checkout happened to be standing on became the corpus the world saw.",
      "The nightly job did rebuild the discovered worlds correctly on each of its three runs, and each time it aborted at a gate about MCP servers that had nothing to do with the scan, before it reached the step that commits.",
      "That gate asserted a sum where its own message stated a disjunction, which held only for as long as no external server listed any tools, and stopped holding the day one truthfully did both.",
      "Fixing all of that still left the job unable to push, because a ruleset now requires a work report on the branch it was pushing to, so the job takes the route everybody else takes and opens a pull request with a report generated from the artifacts it just wrote."
    ]
  }
}

gHashTag and others added 2 commits September 15, 2026 22:29
The catalog behind the Queen map and the Spec Explorer had been frozen since
2026-09-12 at 856 specs while GitHub held far more, and three separate things
had to be true at once for that to happen.

1. sync-t27-specs.mjs read gHashTag/t27 from a working copy at /Users/playom/t27,
   so the repository holding two thirds of the corpus was the one repository CI
   could not refresh. That checkout sat on a feature branch 704 commits behind
   master, which is how the site came to be missing 540 specs that were on
   master all along -- specs/tools (94), bootstrap/tests (90), specs/ternary
   (60), specs/igla (44), specs/crons (33), specs/agents (27), specs/skills (26).
   It now pulls a tarball like the other four founding sources, reuses the
   vendored compiler wasm when cargo is not available, and brings the teaching
   course -- which lives on a branch never merged to master -- through a named
   overlay ref that contributes only the paths master lacks, so the overlay
   retires itself the day the course is merged.

2. The nightly workflow never ran that sync at all, only the discovery scan; and
   every run since 2026-09-13 died at check:tools before reaching its commit
   step. That gate asserted mcpWithTools + mcpExternal == mcp.length while its
   own message states "either lists tools or is external" -- an OR written as a
   sum, which held only while no external server listed any tools.
   mcp/inngest-dev is both, truthfully, so the gate now asserts the invariant it
   names.

3. Even green, its `git push` to main is rejected by ruleset 23148303, which
   requires the "T27 work report" check and lists no bypass actors. It now
   commits to bot/t27-world-scan, generates a report from the artifacts the run
   itself wrote, opens or updates a pull request and enables auto-merge.

Result: 1407 specs across 10 repositories, from GitHub, with no laptop in the
path. Two worlds the frozen catalog never saw -- tt-trinity-euler (46) and
turbobaby-user-bot (12) -- are on the map, and chips/euler is now attributed to
tt-trinity-euler, the repository that owns it.

Also here, all found while proving the above:
- Duplicate copies are recorded as {path, repo, sameAs} pairs instead of only
  counted, and the Spec Explorer shows a spec's other homes. The bytes are
  discarded at vendor time, so nothing downstream could reconstruct this.
- The embedded Explorer lifted its course filter only for a spec outside the
  course. The homepage frame opens hello_world, which is lesson 0, so it read
  "9 specs" beside a map saying 856. An address that names a spec now shows the
  whole library.
- .spec-x-brief is a flex item with nothing to give: measured at 404px it was a
  0px box around 636px of content, which drew over the tab strip and the code.
  No variant shrinks now; an open one takes up to 45vh and scrolls.
- A spec's SOURCES and BODY_EN name files that are not .t27, so a GitHub-sourced
  refresh deleted the seven docs/system/*.md a chapter renders. They are
  vendored from the same tarball, at the same commit, as the spec naming them.
- The Chip panel's "of 676 specs, 361 produce Verilog" now says when it was
  measured and that the corpus has grown since.

Каталог за картой Королевы и обозревателем спек стоял с 12 сентября на 856
спеках, хотя на GitHub их было куда больше, и для этого должны были совпасть
три вещи: основной синк читал gHashTag/t27 с ноутбука (ветка на 704 коммита
позади master), ночной workflow этот синк вообще не запускал и падал на
неродственном гейте check:tools, а даже зелёным не смог бы запушить в main
из-за правила о рабочем отчёте. Теперь источник -- GitHub, курс приходит через
именованный overlay-ref, гейт проверяет то, что заявляет, а workflow открывает
pull request с отчётом и включает авто-мердж. Итог: 1407 спек в 10
репозиториях без участия чьей-либо машины.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
sync-t27-specs.mjs records which file was dropped for which, and then
mergeWorld rebuilt the manifest from a fixed list of fields and dropped the
list itself -- so the one place that knows a spec lives in more than one
repository was thrown away one step later. It is carried through now, a
scanned world contributes its own pairs the same way it contributes entries,
and the contract's manifest key order names the new field rather than failing
on it.

Синк записывает, какой файл отброшен в пользу какого, а шаг vendor пересобирал
манифест по фиксированному списку полей и терял этот список. Теперь он
переносится, найденный мир добавляет свои пары так же, как свои записи, а
контракт порядка ключей знает о новом поле.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@gHashTag
gHashTag enabled auto-merge (squash) September 15, 2026 15:34
gHashTag and others added 2 commits September 15, 2026 22:39
The copy loop named t27, queen, wasm and fonts while dist also holds agents,
crons, docs, functions, skills and tools -- the six generated catalogs the
Explorer tabs read. They are live on t27.ai only because a person copied them
by hand; the next clean run of this workflow would have 404ed all six. The
loop's own comment claimed that copying directories rather than named files
meant the next data file would arrive without anyone remembering to add a line,
which was true only inside the four it named. It now iterates what the build
actually produced.

Цикл копирования перечислял t27, queen, wasm и fonts, тогда как в dist лежат
ещё agents, crons, docs, functions, skills и tools -- шесть сгенерированных
каталогов, которые читают вкладки обозревателя. На t27.ai они живы только
потому, что их скопировали руками. Теперь цикл идёт по тому, что реально
собралось.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The shell was rendered only once universe-atlas.json had arrived: on failure
the branch returned an error paragraph in place of <Runtime/>, so one
unreachable JSON file took Kanban, the Spec Explorer, Agents, Tools, Crons,
Skills, Functions, Project, TRI, Factory, Research and the Mission Map down
with it -- none of which read the atlas at all.

Queen already takes sharedCatalog as optional and falls back to the older hive
without it; that is the path a connected repository has always used. So the
shell mounts either way and the failure is reported over the map instead of in
place of it, with the same Retry, which restores the catalog hive and keeps
whichever tab you were on.

Verified with the file 404ing: all thirteen tabs mount and switch, the address
follows (?tab=kanban, ?tab=specs, ?tab=tools, ?tab=agents, ?tab=project,
?tab=research), the banner clears the command rail, and Retry brings the map
back on ?tab=research without leaving it.

Оболочка рисовалась только после загрузки universe-atlas.json, а при ошибке
ветка возвращала абзац с ошибкой вместо <Runtime/> -- один недоступный JSON
уносил с собой все тринадцать вкладок, включая те, что атлас вообще не читают.
Queen принимает sharedCatalog как необязательный, поэтому оболочка теперь
поднимается в любом случае, а сбой показывается поверх карты, а не вместо неё.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@gHashTag
gHashTag merged commit eab253b into main Sep 15, 2026
15 of 19 checks passed
github-actions Bot added a commit that referenced this pull request Sep 15, 2026
feat(website): the .t27 catalog syncs itself from GitHub, nightly (#1021)

* feat(website): the .t27 catalog syncs itself from GitHub, nightly

The catalog behind the Queen map and the Spec Explorer had been frozen since
2026-09-12 at 856 specs while GitHub held far more, and three separate things
had to be true at once for that to happen.

1. sync-t27-specs.mjs read gHashTag/t27 from a working copy at /Users/playom/t27,
   so the repository holding two thirds of the corpus was the one repository CI
   could not refresh. That checkout sat on a feature branch 704 commits behind
   master, which is how the site came to be missing 540 specs that were on
   master all along -- specs/tools (94), bootstrap/tests (90), specs/ternary
   (60), specs/igla (44), specs/crons (33), specs/agents (27), specs/skills (26).
   It now pulls a tarball like the other four founding sources, reuses the
   vendored compiler wasm when cargo is not available, and brings the teaching
   course -- which lives on a branch never merged to master -- through a named
   overlay ref that contributes only the paths master lacks, so the overlay
   retires itself the day the course is merged.

2. The nightly workflow never ran that sync at all, only the discovery scan; and
   every run since 2026-09-13 died at check:tools before reaching its commit
   step. That gate asserted mcpWithTools + mcpExternal == mcp.length while its
   own message states "either lists tools or is external" -- an OR written as a
   sum, which held only while no external server listed any tools.
   mcp/inngest-dev is both, truthfully, so the gate now asserts the invariant it
   names.

3. Even green, its `git push` to main is rejected by ruleset 23148303, which
   requires the "T27 work report" check and lists no bypass actors. It now
   commits to bot/t27-world-scan, generates a report from the artifacts the run
   itself wrote, opens or updates a pull request and enables auto-merge.

Result: 1407 specs across 10 repositories, from GitHub, with no laptop in the
path. Two worlds the frozen catalog never saw -- tt-trinity-euler (46) and
turbobaby-user-bot (12) -- are on the map, and chips/euler is now attributed to
tt-trinity-euler, the repository that owns it.

Also here, all found while proving the above:
- Duplicate copies are recorded as {path, repo, sameAs} pairs instead of only
  counted, and the Spec Explorer shows a spec's other homes. The bytes are
  discarded at vendor time, so nothing downstream could reconstruct this.
- The embedded Explorer lifted its course filter only for a spec outside the
  course. The homepage frame opens hello_world, which is lesson 0, so it read
  "9 specs" beside a map saying 856. An address that names a spec now shows the
  whole library.
- .spec-x-brief is a flex item with nothing to give: measured at 404px it was a
  0px box around 636px of content, which drew over the tab strip and the code.
  No variant shrinks now; an open one takes up to 45vh and scrolls.
- A spec's SOURCES and BODY_EN name files that are not .t27, so a GitHub-sourced
  refresh deleted the seven docs/system/*.md a chapter renders. They are
  vendored from the same tarball, at the same commit, as the spec naming them.
- The Chip panel's "of 676 specs, 361 produce Verilog" now says when it was
  measured and that the corpus has grown since.

Каталог за картой Королевы и обозревателем спек стоял с 12 сентября на 856
спеках, хотя на GitHub их было куда больше, и для этого должны были совпасть
три вещи: основной синк читал gHashTag/t27 с ноутбука (ветка на 704 коммита
позади master), ночной workflow этот синк вообще не запускал и падал на
неродственном гейте check:tools, а даже зелёным не смог бы запушить в main
из-за правила о рабочем отчёте. Теперь источник -- GitHub, курс приходит через
именованный overlay-ref, гейт проверяет то, что заявляет, а workflow открывает
pull request с отчётом и включает авто-мердж. Итог: 1407 спек в 10
репозиториях без участия чьей-либо машины.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(website): the duplicate list survives the vendor step

sync-t27-specs.mjs records which file was dropped for which, and then
mergeWorld rebuilt the manifest from a fixed list of fields and dropped the
list itself -- so the one place that knows a spec lives in more than one
repository was thrown away one step later. It is carried through now, a
scanned world contributes its own pairs the same way it contributes entries,
and the contract's manifest key order names the new field rather than failing
on it.

Синк записывает, какой файл отброшен в пользу какого, а шаг vendor пересобирал
манифест по фиксированному списку полей и терял этот список. Теперь он
переносится, найденный мир добавляет свои пары так же, как свои записи, а
контракт порядка ключей знает о новом поле.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(ci): the apex deploy copies every directory the build produced

The copy loop named t27, queen, wasm and fonts while dist also holds agents,
crons, docs, functions, skills and tools -- the six generated catalogs the
Explorer tabs read. They are live on t27.ai only because a person copied them
by hand; the next clean run of this workflow would have 404ed all six. The
loop's own comment claimed that copying directories rather than named files
meant the next data file would arrive without anyone remembering to add a line,
which was true only inside the four it named. It now iterates what the build
actually produced.

Цикл копирования перечислял t27, queen, wasm и fonts, тогда как в dist лежат
ещё agents, crons, docs, functions, skills и tools -- шесть сгенерированных
каталогов, которые читают вкладки обозревателя. На t27.ai они живы только
потому, что их скопировали руками. Теперь цикл идёт по тому, что реально
собралось.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(queen): a failed atlas loses the map, not all thirteen tabs

The shell was rendered only once universe-atlas.json had arrived: on failure
the branch returned an error paragraph in place of <Runtime/>, so one
unreachable JSON file took Kanban, the Spec Explorer, Agents, Tools, Crons,
Skills, Functions, Project, TRI, Factory, Research and the Mission Map down
with it -- none of which read the atlas at all.

Queen already takes sharedCatalog as optional and falls back to the older hive
without it; that is the path a connected repository has always used. So the
shell mounts either way and the failure is reported over the map instead of in
place of it, with the same Retry, which restores the catalog hive and keeps
whichever tab you were on.

Verified with the file 404ing: all thirteen tabs mount and switch, the address
follows (?tab=kanban, ?tab=specs, ?tab=tools, ?tab=agents, ?tab=project,
?tab=research), the banner clears the command rail, and Retry brings the map
back on ?tab=research without leaving it.

Оболочка рисовалась только после загрузки universe-atlas.json, а при ошибке
ветка возвращала абзац с ошибкой вместо <Runtime/> -- один недоступный JSON
уносил с собой все тринадцать вкладок, включая те, что атлас вообще не читают.
Queen принимает sharedCatalog как необязательный, поэтому оболочка теперь
поднимается в любом случае, а сбой показывается поверх карты, а не вместо неё.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant