Skip to content

fix(brain-ci): restore the brain test steps that exist, drop the ones that test nothing - #1333

Merged
gHashTag merged 15 commits into
mainfrom
fix/brain-ci-unit-tests
Oct 4, 2026
Merged

gHashTag merged 15 commits into
mainfrom
fix/brain-ci-unit-tests

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

#1328 is merged (squash 292d827), so the diff now contains only this change. The commit list still shows #1328's four original commits because this branch was cut from #1328 before the squash; their content equals the squash and they add nothing to the Files tab.

Closes #1331

What was wrong

#517 (42490a2) deleted every test-<region>, test-brain and test-brain-stress step from build.zig, but brain-ci.yml kept calling them. Each Unit Tests leg died on no step named 'test-basal-ganglia', Integration, Stress and CLI Smoke were skipped behind it, and the Report failed. With the steps back, three more things surfaced: stress_test.zig no longer compiled, metrics_dashboard.zig crashed and leaked under its own tests, and the integration binary needed libc on Linux. Several remaining steps also "passed" without testing anything.

Per-job decision

job decision reason evidence
Brain Health Check kept, one step removed tri stress --record is a TODO stub that exits 0, so "Health snapshot recorded to Hippocampus" was never true. The scored health step from #1328 is unchanged. run 37181206224: Score: 100.0/100 (17 of 17 probes passed)
Build Check unchanged zig build -Dci=true is a real compile gate. run 37181206224: success
Unit Tests: basal-ganglia restored test-basal-ganglia is back in build.zig with src/brain/basal_ganglia.zig as the test root. 51/51; CI NC run 37181508280: 50/51, red
Unit Tests: reticular-formation restored test-reticular-formation, root src/brain/reticular_formation.zig. 48/48; CI NC: 47/48 expected 3, found 2, red
Unit Tests: locus-coeruleus restored test-locus-coeruleus, root src/brain/locus_coeruleus.zig. 52/52; CI NC: 51/52 expected 1001, found 1000, red
Unit Tests: intraparietal removed intraparietal_sulcus.zig wraps the hslm library that moved out. What is left is stubs (GF16 = f32, TF3 = f32, PHI: f32 = 3.0) and an undeclared hslm; zig test stops at 11 compile errors, and its 31 tests assert behaviour of the moved library. Making it compile means re-implementing GF16/TF3 here or importing from another repo. That is not a small, honest fix, so the leg is dropped and build.zig says why. zig test src/brain/intraparietal_sulcus.zig: 11 errors, first use of undeclared identifier 'hslm'
Unit Tests: hslm removed hslm lives in gHashTag/trinity-training and is tested there; nothing named hslm is in this tree. no hslm source under src/
Integration Test restored test-brain runs two binaries: the src/brain/brain.zig aggregator (13 tests) and src/brain/integration_test.zig (138 tests, including metrics_dashboard.zig by path). It needed the metrics_dashboard fixes below and libc for state_recovery. 151/151; CI NC run 37181253809: signal 6, red
Stress Test restored, gate rewritten test-brain-stress runs src/brain/stress_test.zig (261 tests). The old gate parsed a Score: line out of 300 that nothing ever printed, turned its absence into 0, compared it with a threshold of 270, and posted a hard-coded 100/100 per-region table. Now the step reads zig's own Build Summary: a missing or zero count is NOT MEASURED (exit 1, not 0); a non-zero exit, any failed test, or any leak fails. STRESS_TEST_THRESHOLD and the --record call are gone. 261/261, 0 leaked; CI NC runs 37181281587 (255/261, red) and 37181306269 (NOT MEASURED, red)
CLI Smoke Test changed Only tri stress --health is left. It exits 1 on any failed probe, which is stricter than the health job's threshold of 80. Dropped: tri task stats, tri task list and tri event stats (each || true around a TODO stub that exits 0) and tri stress --scan (a stub that prints "not implemented" and exits 0). No scan measurement was implemented here. run 37181206224: 17 of 17 probes, HEALTHY
Export Brain Metrics removed tri stress --metrics is a TODO stub. The job ran under if: always(), so on #1328 it reported success while every job above it failed. The file it uploaded is written only by health_history.zig, which no workflow command calls; the --record stub that was meant to write it does nothing. #1328 run 37176676345: Export Brain Metrics success, Unit Tests failed
Brain Health Report kept strict, table fixed The verdict still requires every job to be success. The table now prints each job's real result (success, failure, skipped, cancelled) instead of mapping everything else to FAIL, so a job skipped behind a failure no longer reads as a second failure. NC runs: report failure when one upstream job fails
Merge Gate changed It needs the report and had no always(), so a failed report made it skipped (grey) instead of red. On #1328 run 37176676345 the report failed and the gate showed skipped. With always() the gate runs, and its existing check fails it for any report result other than success. PR run 37182290330: the gate ran and passed. Its red path was not exercised in CI, because the negative-control branches had no PR and the gate is PR-only.
Weekly Brain Trend Report changed Removed the --record/--history stub calls and an unused build step. The stress run uses set -o pipefail and --summary all, the report is generated with if: always(), and the artifact name carries the run id. schedule-only; actionlint clean

Code fixes the restored steps forced

  • metrics_dashboard.zig, RegionMetrics.deinit, use after deinit. It read self.raw_metrics.allocator after raw_metrics.deinit(), and Zig 0.15's HashMap.deinit sets the map to undefined, so the test binary segfaulted at 0xaaaaaaaaaaaaaac2. The allocator is now read first.
  • metrics_dashboard.zig, setMetric overwrite leaked. HashMap.put on an existing key keeps the old key, so the fresh key copy and the old value both leaked. A new putOwnedValue uses getOrPut, frees the old value, and dupes the key only for a new entry.
  • metrics_dashboard.zig tests. Four tests that fill the global basal_ganglia and reticular_formation singletons now reset them with defer. Two tests now free what they allocate. The formatAscii long-name test now asserts the 20-byte cut that formatAscii makes (name[0..20]); the old expectation matched a 23-byte prefix.
  • stress_test.zig. The registry was sharded (shards: [16]Shard) but the stress tests still read registry.claims, which no longer exists, so the file did not compile. They now use registry.count() and a findClaim helper that takes the shard's read lock.
  • build.zig, target names the capability checker can read. The S01 checker (external/t27/tools/trinity_manifest.py) finds a test target only as const X = b.addTest(.{ .root_module = b.createModule(.{ .root_source_file = b.path(...) }) }) and a step only as b.step("literal", ...). The first version built the roots through a helper and the region steps in a loop, so the checker saw one anonymous test:None and missed three steps. 291ac8b writes each root and step out. The same files compile with the same imports, and the test counts are unchanged.
  • build.zig, libc. state_recovery.captureState() calls std.c.getpid(). macOS links libc implicitly, so test-brain passed locally, but on ubuntu-latest it failed with dependency on libc must be explicitly specified (run 37180861619). The state_recovery module now sets link_libc = true. A cross-compile for x86_64-linux-gnu reproduces the error before the change and compiles cleanly after it.

Negative controls

Each restored step was broken, seen red, and restored.

Local, zig 0.15.2, macOS arm64:

step break broken restored
test-basal-ganglia Registry.count() returns + 1 exit 1, 48/51, 3 failed 51/51
test-reticular-formation publish counts fetchAdd(2) exit 1, 43/48, 5 failed, 1 leaked 48/48
test-locus-coeruleus nextDelay fast path adds 1 ms exit 1, 47/52, 5 failed 52/52
test-brain deinit reads the allocator after raw_metrics.deinit() again exit 1, segfault at 0xaaaaaaaaaaaaaac2; the summary still says 151/151 passed 151/151
test-brain-stress Registry.count() returns + 1 exit 1, 248/261, 13 failed 261/261

CI, on throwaway branches (fix/brain-ci-nc*). Each branch was one breaking commit on top of this branch. All branches have been deleted.

branch, run break Unit Tests Integration Stress Report
nc2-units, 37181508280 one wrong expected value in a test of each region file 3 legs red: BG 50/51, RF 47/48, LC 51/52 skipped skipped failure
nc2-integration, 37181253809 RegionMetrics.deinit use-after-deinit 3 green red: signal 6 in RegionMetrics alert allocation and cleanup; summary reads 151/151 passed, exit 1 skipped failure
nc2-stress, 37181281587 telemetry percentile insertion sort flipped 3 green green red: 255/261, 6 failed, brain stress FAILED failure
nc2-stress-unmeasured, 37181306269 stress_test.zig back to registry.claims (does not compile) 3 green green red: brain stress was NOT MEASURED, exit 1 failure

A first units control (run 37181016114) broke the region implementations. The Brain Health Check caught it first (70.6/100, 5 failed probes), so the unit legs were skipped and never exercised. That is why nc2-units breaks test expectations instead: test blocks are not compiled into tri, so health stays green and the unit legs have to catch the break.

CI on this branch

  • Push run 37180861619 (f8d82f8): units green; integration red with the Linux libc error. Fixed in f4170dd.
  • Push run 37181206224 (f4170dd): every job green: health 100.0/100, units 51/51, 48/48 and 52/52, integration 151/151, stress 261/261 with 0 leaked, CLI Smoke 17 of 17, report success. Merge Gate skipped, as it is PR-only.
  • Push run 37181938721 and PR run 37182290330 (1cdca6a, after merging main): every brain-ci job green; on the PR run the Merge Gate ran and passed.
  • Push run 37184075728 (291ac8b): every brain-ci job green: health 100.0/100, units 51/51, 48/48 and 52/52, integration 151/151, stress 261/261 with 0 leaked, CLI Smoke 17 of 17, report success; Merge Gate skipped as it is PR-only.
  • PR run 37184077352 (291ac8b): every brain-ci job green with the same counts, and the Merge Gate ran and passed. In the same PR, reproduce.yml run 37184077325 was red on the 11 UNASSIGNED_TARGET findings that the capability card and the re-vendor below resolve.
  • At 40a5315 (merge of main and the re-vendor): t27 contracts, CI, Codegen Validation, Build paths, Pre-commit and both Headless profile jobs passed; the work report failed only because the PR body still named 291ac8b.
  • Push run 37193596465 (5604063): every brain-ci job green: health 100.0/100, units 51/51, 48/48 and 52/52, integration 151/151, stress 261/261 with 0 leaked, CLI Smoke 17 of 17, report success; Merge Gate skipped as it is PR-only.
  • PR run 37193597519 (5604063): every brain-ci job green with the same counts, and the Merge Gate ran and passed. In the same PR, t27 contracts (run 37193597515) and both Headless profile jobs (run 37193597584) passed.

The capability card and the re-vendor

"Headless profile from a clean clone" was red here because no vendored t27 capability card owned the 11 restored brain targets. The S01 checker reported each one as UNASSIGNED_TARGET. This is now fixed where the cards live; the finding was not tolerated.

  • spec(trinity): a capability card for the brain region tests, one ahead of the pin (Closes #5953) t27#5965 (squash 39e06aa0, closes t27#5953) adds the card trinity/brain.regions.
    • It owns the five steps and the six src/brain test roots.
    • Its acceptance is the five zig build test-<region> -Dci=true commands.
    • It removes src/brain from research.unreferenced-sources.
  • 2113ee7 merges main into this branch. Main was at a6ec8dc, which includes feat(website): home gets a Get started door; frames boot on a click; a 404 wire is asked once #1310 (76f8975). Nothing was rebased or force-pushed.
  • 40a5315 runs tools/contracts.py vendor against t27 39e06aa0. The vendor now holds 76 files instead of 75; the extra one is the new card. The pin moves from de2a1aca, which brings two earlier upstream changes:
    • t27#5474 changes the S01 checker, tools/trinity_manifest.py (+310 lines).
      • It no longer reads build.zig comments, so mcp.needle-mcp and mcp.trinity-mcp stop owning needle-mcp and trinity-mcp. Those steps exist only in commented-out lines. The cards' acceptance becomes zig build -Dci=true && test -x zig-out/bin/<name>.
      • It counts the vendored t27 copies separately.
      • project.t27, the README and research.unreferenced-sources restate the corrected counts.
    • t27#5632 adds agent bindings to specs/tools/mcp/needle.t27 and trinity.t27.
  • specs/reproduce/capabilities.t27:
    • trinity/brain.regions goes into RUN, because the profile's own test step compiles none of its roots.
    • mcp.needle-mcp and mcp.trinity-mcp leave KNOWN_BLOCKED. Their new acceptance is the install the profile already makes, and keeping them on the list would fail the index as BLOCKED_NOW_MEASURED.
    • registry.commands stays blocked, with its reason unchanged.
  • 5604063 raises TESTS_RUN_MIN in specs/reproduce/contracts.t27 from 77 to 78.
    • The new card's single test is the only test this vendor adds.
    • t27c built at 39e06aa0 measures 58 specs and 78 tests, all of them passing.
    • The other ledgers were re-checked and still hold:
      • the 11 TEST_BLOCKED specs each fail with the same first error the ledger records;
      • queen/lotus.t27 still drops 48 top-level tokens;
      • the two specs/tools/mcp specs still declare no test.
capability index cards measured blocked excluded
main a6ec8dc, run 37189617187 (both arches) 51 21 3 27
this PR at 40a5315, run 37190239982 (x86_64 and aarch64) 52 24 1 27
this PR at 5604063, run 37193597584 (x86_64 and aarch64) 52 24 1 27

At the new pin, the S01 checker reports only the drift that is already tolerated: PIN_MISMATCH and nine COUNT_MISMATCH findings. It reports no UNASSIGNED_TARGET.

Local runs, and what was not run locally:

  • tools/contracts.py check --upstream --t27c used t27c built at 39e06aa0 and zig 0.16.0 on macOS arm64.
    • The first run reported four findings. test-report --all did not list llm.igla-chat, mcp.needle-mcp or mcp.trinity-mcp, each of which passes 1/1 when run alone, so only 75 tests ran against a floor of 77.
    • A direct --all run listed all 58 specs and 78 tests. So did two later full checks, one with the floor at 77 and one at 78; both passed.
    • CI's contracts job passed on 40a5315.
    • The transient is not explained. Other sessions were building and running compilers on the same host at the time. The check reported the transient as a finding rather than passing it.
  • The headless profile and tools/capabilities.py index were not run locally. The host had 6 to 9 GB free and is shared with other sessions; a fresh-cache profile build plus the index's acceptance runs would have used much of that. CI ran both, with the results above.
  • Pre-existing on main as well: the test blocks of specs/reproduce/contracts.t27 do not compile under t27c test-report. The compiler emits [1]u32 = [48] as the Zig [48] and fails with spec.zig:29:46: expected type expression. So no tool executes the updated assert TESTS_RUN_MIN == 78; tools/contracts.py reads the constant directly.

Not fixed here (out of scope, recorded)

  • tri task stats|list|claim, tri event stats|publish and tri stress --scan|--record|--metrics|--history are TODO stubs that exit 0 (src/tri/tri_commands.zig). CI no longer calls them as tests. Implementing them is separate work.
  • The stress test "maximum TTL" reads the claim with if (entry) |claim|, so it passes silently if the claim is missing. This weakness predates this PR.
  • intraparietal_sulcus.zig stays uncompilable in-tree. It should either import hslm from trinity-training or be deleted, which is a decision for its owner.
  • trinity/registry.commands stays a known-blocked capability: zig build export-registry writes the time it ran into generated_at, so git diff --exit-code .trinity/registry.json cannot pass.
  • The pr-opened workflow's gh: Bad credentials (HTTP 401) failure is unrelated to brain-ci.
{
  "version": 1,
  "head_sha": "56040636efd3b645809d1e9c439851bed53fed31",
  "summary": "brain-ci called build steps that #517 deleted, so its unit, integration and stress jobs never ran. The steps that compile are restored and green on Linux, the ones that cannot be are removed with reasons, stub commands are no longer counted as tests, and every restored step was shown to go red when broken. A t27 capability card now owns the restored targets, and the re-vendor makes the headless capability index green.",
  "changes": [
    "build.zig restores test-basal-ganglia, test-reticular-formation, test-locus-coeruleus, test-brain and test-brain-stress, and links libc for state_recovery.",
    "metrics_dashboard.zig: RegionMetrics.deinit no longer reads the allocator after deinit, setMetric overwrite no longer leaks, and its tests clean up after themselves.",
    "stress_test.zig reads claims through the sharded registry (count and a read-locked findClaim) and compiles again.",
    "brain-ci.yml drops the intraparietal and hslm legs, the Export Brain Metrics job and every call to a tri stub, and gates stress on zig's own test count, treating a missing count as NOT MEASURED.",
    "brain-ci.yml report prints real job results and the merge gate runs under always() so it fails red instead of skipping.",
    "build.zig writes each brain test root and step out literally so the S01 capability checker names every restored target precisely.",
    "Re-vendored gHashTag/t27 at 39e06aa0: the trinity/brain.regions card (t27#5965), the S01 checker that skips build.zig comments (t27#5474) and MCP agent bindings (t27#5632).",
    "specs/reproduce/capabilities.t27 measures trinity/brain.regions in RUN and drops mcp.needle-mcp and mcp.trinity-mcp from KNOWN_BLOCKED.",
    "specs/reproduce/contracts.t27 raises TESTS_RUN_MIN from 77 to the 78 tests that run at the new pin."
  ],
  "tests": [
    {
      "command": "zig build test-basal-ganglia / test-reticular-formation / test-locus-coeruleus -Dci=true --summary all (zig 0.15.2)",
      "result": "51/51, 48/48 and 52/52 tests pass locally and on ubuntu-latest in push run 37181206224.",
      "status": "passed",
      "evidence": "Build Summary lines in the three Unit Tests job logs of run 37181206224."
    },
    {
      "command": "zig build test-brain -Dci=true --summary all",
      "result": "151/151 tests pass (13 aggregator + 138 integration) after the metrics_dashboard and libc fixes, locally and in run 37181206224.",
      "status": "passed",
      "evidence": "Integration Test job log: Build Summary: 5/5 steps succeeded; 151/151 tests passed."
    },
    {
      "command": "zig build test-brain-stress -Dci=true --summary all",
      "result": "261/261 stress tests pass with 0 leaked; the step prints brain stress: 261/261 tests passed, 0 leaked.",
      "status": "passed",
      "evidence": "Stress Test job log of run 37181206224; stress_test.zig compiles only after the sharded-registry fix."
    },
    {
      "command": "brain-ci push run 37180861619 on f8d82f8b8",
      "result": "Units passed but the integration binary failed to compile on Linux: dependency on libc must be explicitly specified (state_recovery std.c.getpid).",
      "status": "failed",
      "evidence": "Fixed by f4170dde1; a cross-compile for x86_64-linux-gnu reproduces the error before and compiles after."
    },
    {
      "command": "brain-ci push run 37181206224 on f4170dde1",
      "result": "Every job succeeded: health 100.0/100, three unit legs, integration, stress, CLI Smoke 17 of 17 probes, and the report.",
      "status": "passed",
      "evidence": "gh run view 37181206224; Merge Gate skipped because it runs only on pull_request events."
    },
    {
      "command": "local negative controls via a break-run-restore script for each restored step",
      "result": "Every break went red: BG 48/51, RF 43/48 plus a leak, LC 47/52, test-brain segfault with 151/151 printed, stress 248/261; every restore went green.",
      "status": "passed",
      "evidence": "Local negative-control table in the PR description."
    },
    {
      "command": "CI negative-control branches fix/brain-ci-nc2-units, -integration, -stress, -stress-unmeasured",
      "result": "Each run went red at exactly the targeted job with upstream jobs green: runs 37181508280, 37181253809, 37181281587 and 37181306269.",
      "status": "passed",
      "evidence": "CI negative-control table in the PR description; the throwaway branches are deleted."
    },
    {
      "command": "stress step parser fed simulated pass, fail, leak-only, compile-error and empty outputs",
      "result": "Pass gives RESULT=pass; failures and leaks give fail; compile error and empty output give not-measured with exit 1, never a zero score.",
      "status": "passed",
      "evidence": "Matches CI: run 37181306269 printed brain stress was NOT MEASURED for a compile error."
    },
    {
      "command": "zig test src/brain/intraparietal_sulcus.zig",
      "result": "Fails with 11 compile errors starting at use of undeclared identifier hslm; the leg is dropped rather than faked.",
      "status": "failed",
      "evidence": "Stubs GF16 = f32, TF3 = f32, PHI: f32 = 3.0 remain after hslm moved to trinity-training."
    },
    {
      "command": "actionlint -ignore SC2086 -ignore SC2129 .github/workflows/brain-ci.yml; zig fmt --check build.zig and touched sources",
      "result": "Both clean on the final workflow and the touched Zig files.",
      "status": "passed",
      "evidence": "Run after the merge of main, which had produced a duplicated permissions block that is now removed."
    },
    {
      "command": "S01 checker (external/t27/tools/trinity_manifest.py inventory + check) on a clean tree at 291ac8b24",
      "result": "11 UNASSIGNED_TARGET findings, all brain targets: five steps and six src/brain test roots that no vendored t27 capability card owns. Before 291ac8b24 they read as test:None plus two steps. The other findings are the tolerated PIN_MISMATCH and COUNT_MISMATCH drift.",
      "status": "failed",
      "evidence": "Resolved by gHashTag/t27#5965 (39e06aa0) and the re-vendor in 40a531560; the next entries record the check at the new pin."
    },
    {
      "command": "tools/contracts.py vendor --t27 <t27 clone> --revision 39e06aa0, then tools/contracts.py check",
      "result": "The vendor holds 76 files of gHashTag/t27 at 39e06aa0 (one more: the brain.regions card), and the offline check says the consumed contracts are what the lock says.",
      "status": "passed",
      "evidence": "Commit 40a531560; contracts --self-check and capabilities --self-check also ok."
    },
    {
      "command": "S01 checker and tools/capabilities.py index in reproduce.yml, runs 37190239982 (40a531560) and 37193597584 (56040636e)",
      "result": "52 cards: 24 measured, 1 blocked (registry.commands), 27 excluded; every capability is what the index says. Main has 51 cards: 21 measured, 3 blocked. No UNASSIGNED_TARGET; only PIN_MISMATCH and COUNT_MISMATCH drift.",
      "status": "passed",
      "evidence": "Headless profile job logs; the test step reports 135/135 steps succeeded on both arches."
    },
    {
      "command": "tools/contracts.py check --upstream --t27c <t27c at 39e06aa0>, zig 0.16.0, macOS arm64, first run",
      "result": "4 findings: test-report --all did not list llm.igla-chat, mcp.needle-mcp and mcp.trinity-mcp (each 1/1 alone), so 75 tests ran under the floor of 77.",
      "status": "failed",
      "evidence": "Not reproduced by a direct --all run or two later full checks; the cause is unknown; the host was shared with other compiler builds."
    },
    {
      "command": "tools/contracts.py check --upstream --t27c <t27c at 39e06aa0>, re-runs with TESTS_RUN_MIN 77 and then 78",
      "result": "Both exit 0: 58 specs measured, 78 tests run and passing, 11 blocked, 2 with no tests; the consumed contracts are what the lock says.",
      "status": "passed",
      "evidence": "CI contracts job passed on 40a531560 (run 37190239971)."
    },
    {
      "command": "t27c test-report on each of the 11 TEST_BLOCKED specs at 39e06aa0",
      "result": "All 11 are still blocked, and each first error is identical to the one TEST_BLOCKED_ERRORS records.",
      "status": "passed",
      "evidence": "Compared string for string with the regex tools/contracts.py uses."
    },
    {
      "command": "t27c test-report specs/reproduce/contracts.t27",
      "result": "BLOCKED: spec.zig:29:46 expected type expression, found ';' because [1]u32 = [48] is emitted as Zig [48]. The same happens on main a6ec8dcdd.",
      "status": "failed",
      "evidence": "Pre-existing compiler lowering; tools/contracts.py reads TESTS_RUN_MIN directly, so the floor is still enforced."
    },
    {
      "command": "brain-ci push run 37184075728 and pull_request run 37184077352 on 291ac8b24",
      "result": "Every brain-ci job green in both runs: health 100.0/100, units 51/51, 48/48 and 52/52, integration 151/151, stress 261/261 with 0 leaked, CLI Smoke 17 of 17.",
      "status": "passed",
      "evidence": "gh run view 37184075728 and 37184077352 --repo gHashTag/trinity; the Merge Gate ran and passed on the PR run."
    },
    {
      "command": "brain-ci push run 37193596465 on 56040636e",
      "result": "every brain-ci job green: health 100.0/100, units 51/51, 48/48 and 52/52, integration 151/151, stress 261/261 with 0 leaked, CLI Smoke 17 of 17, report success; Merge Gate skipped as it is PR-only",
      "status": "passed",
      "evidence": "gh run view 37193596465 --repo gHashTag/trinity"
    },
    {
      "command": "brain-ci pull_request run 37193597519 on 56040636e",
      "result": "every brain-ci job green with the same counts, and the Merge Gate ran and passed. In the same PR, t27 contracts (run 37193597515) and both Headless profile jobs (run 37193597584) passed",
      "status": "passed",
      "evidence": "gh run view 37193597519 --repo gHashTag/trinity"
    }
  ],
  "limitations": [
    "tri task, tri event and tri stress --scan/--record/--metrics/--history remain stubs that exit 0; CI no longer calls them as tests.",
    "intraparietal_sulcus.zig still does not compile; its tests target the hslm library now in gHashTag/trinity-training.",
    "The stress maximum-TTL test still passes silently if the claim is missing; that weakness predates this change.",
    "trinity/registry.commands stays known-blocked: the registry export writes the time it ran into generated_at.",
    "The headless profile and capability index were measured in CI only, not locally, because the shared host had 6 to 9 GB free.",
    "One local contracts run missed three specs from t27c test-report --all; three later runs did not reproduce it and the cause is unknown.",
    "The test blocks of specs/reproduce/contracts.t27 do not compile under t27c test-report, on main as well."
  ],
  "tags": [
    "brain_ci",
    "zig",
    "negative_control",
    "honest_metrics"
  ],
  "blog": {
    "title": "Bringing back the brain tests, and only the ones that test something",
    "summary": "Brain CI was red because it called build steps deleted in April. Restoring them exposed a stress suite that no longer compiled, a use-after-deinit crash and leaks in the metrics dashboard, and a Linux-only libc link error. Jobs that only ran stubs were removed, not turned green.",
    "outline": [
      "Why every unit leg failed: #517 removed the test steps from build.zig while brain-ci kept calling them, so nothing behind the unit job ever ran.",
      "What came back: three region test steps, an integration step over the aggregator and integration suite, and the 261-test stress suite, all green on ubuntu-latest.",
      "What restoring them uncovered: the stress suite still used the pre-sharding registry, and RegionMetrics.deinit read its allocator after HashMap.deinit made it undefined.",
      "What was removed instead of faked: the intraparietal and hslm legs, the metrics export job, and every tri command that prints a not-implemented notice and exits zero.",
      "How we know the gates bite: each restored step was broken on purpose locally and on throwaway CI branches, and each one went red at exactly that job."
    ]
  }
}

🤖 Generated with Claude Code

gHashTag and others added 11 commits October 4, 2026 10:47
poll(since, allocator, max_events) scanned only the first max_events
buffered events and then filtered them by `since`. Whenever those oldest
events were at or before `since`, it returned nothing even though newer
events matched: with three old events and two new ones,
poll(boundary, a, 1) returned 0 events instead of 1.

The scan now walks the whole buffer and stops once max_events events
have passed the filter, which is what the doc comment already promised
("Returns events with timestamp > since ... Limited by max_events").
poll(0, a, n) is unchanged.

Found by the new `tri stress --health` probe limit-after-filter. A module
test pins it: it fails on the old code (0 events) and passes on this one.

Refs #1326

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`tri stress --health` was a stub that printed "not implemented yet" and
no Score: line, so the brain-ci Brain Health Check failed on every run
as "not measured".

It now runs 17 probes against the three brain regions linked into tri:
basal_ganglia (8: claim, refuse, heartbeat, complete, abandon, TTL
expiry, shard accounting, an 8-thread one-winner-per-task race),
reticular_formation (5: string ownership, FIFO order, since filter,
limit-after-filter, bounded ring) and locus_coeruleus (4: exponential,
capped and monotone, linear and constant, jitter bounds).

Each probe uses fresh private instances (never the process globals) and
its own leak-checking allocator; a leak fails the probe. The output
lists every probe as PASS or FAIL with its claim, then one plain-ASCII
line

  Score: <n.n>/100 (<passed> of <total> probes passed)

where n = 100 * passed / total, then "Status: HEALTHY" or
"Status: UNHEALTHY". The command exits 1 if any probe failed. Nothing
in the score is a constant: removing the basal_ganglia live-claim check,
the locus_coeruleus cap or the poll fix drops it to 88.2, 82.4 and 94.1.

Not covered: other src/brain regions (not linked into tri), load and
soak behaviour. The other stress modes still only print a notice.

Refs #1326

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Brain Health Check parsed the first number anywhere on the Score:
line, so a colour escape such as ESC[32m before "Score:" read as 32.
It now reads only the number directly after "Score:" on the first
Score: line, integer or decimal.

The HEALTHY check used `grep -q "HEALTHY"`, which also matches
"UNHEALTHY", so every run with a Status line reported healthy. It now
matches the whole line "Status: HEALTHY".

A run above the threshold with failed probes now gets a warning
annotation instead of passing silently. The threshold (80), the
not-measured and unparseable branches, and `|| true` are unchanged.

The comment block describes what the score measures and states that the
CLI Smoke Test, which fails on any failed probe, only runs after
brain-unit, brain-integration and brain-stress succeed.

Closes #1326

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The repository default GITHUB_TOKEN is read-only and brain-ci.yml had no
permissions block, so "Comment Health on PR" failed with 403 "Resource
not accessible by integration" on the first PR run where the health
check passed (run 37176287100). The same applies to the stress comment
and the Critical State Notification. Those steps were unreachable while
the health check failed first.

Grant contents: read and pull-requests: write at workflow level. A PR
from a fork still receives a read-only token.

Refs #1326

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
stress_test.zig still read `registry.claims`, the single map the registry
had before it was split into 16 shards, so the file stopped compiling
(13 errors) and test-brain-stress could not run. Count with
Registry.count() and look a claim up in the one shard
Registry.getShardIndex names, under that shard's read lock. The tests
themselves are unchanged: 261/261 pass on zig 0.15.2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…or leak

Restoring test-brain ran the dashboard's own tests for the first time in
months (integration_test.zig imports the file by path). They found:

- RegionMetrics.deinit freed the alert through raw_metrics.allocator
  after raw_metrics.deinit() had set the map to undefined: a segfault at
  0xaaaa... whenever a region carried an alert. Read the allocator first.
- setMetric/setMetricOwned on an existing key leaked the new key copy and
  the replaced value, because HashMap.put keeps the stored key. Copy the
  key only on first insert and free the replaced value.
- Five tests leaked memory they owned (the singletons collect() creates,
  buffers setMetric copies) and one asserted a 23-byte prefix of a name
  formatAscii truncates to 20 bytes. The assertions now match the code.

test-brain: 151/151 pass, no leaks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#517 (42490a2) removed test-basal-ganglia, test-reticular-formation,
test-locus-coeruleus, test-brain and test-brain-stress from build.zig while
brain-ci.yml kept calling them, so every Unit Tests leg failed with "no
step named ..." and nothing behind it ran. Point the steps back at the
source files, which were never removed.

Not restored: test-intraparietal (intraparietal_sulcus.zig is stubs around
the hslm library that left this repo and does not compile) and test-hslm
(hslm is tested in gHashTag/trinity-training; no such step ever existed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Unit Tests: basal-ganglia, reticular-formation, locus-coeruleus only;
  intraparietal and hslm dropped with the reason in the file.
- Stress: pipefail-safe exit code plus zig's own "N/M tests passed"
  count. A missing count is NOT MEASURED and fails, never 0. The 270/300
  threshold and the hard-coded 100/100 PR table are gone; nothing ever
  produced them.
- CLI Smoke: only `tri stress --health`, no `|| true`. task stats, task
  list, event stats and stress --scan are unimplemented and exit 0.
- Export Brain Metrics job and the --record/--history steps removed: they
  call unimplemented subcommands, and the health job wrote "snapshot
  recorded" whether or not anything was.
- Report shows each job's real result instead of FAIL for skipped; the
  verdict still requires every job to succeed. Weekly artifact name is
  now unique (`with:` never expanded $(date)).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
captureState() calls std.c.getpid(). macOS links libc implicitly, so
test-brain passed locally, but on ubuntu-latest the integration binary
failed to compile with 'dependency on libc must be explicitly specified'
(brain-ci run 37180861619). A cross-compile for x86_64-linux-gnu
reproduces the error before this change and compiles cleanly after it.

Refs #1331

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gate needed brain-health-report without always(), so whenever the
report failed the gate was skipped. A skipped check renders grey, not
red: on #1328 (run 37176676345) the report failed and the Merge Gate
showed skipped. With always() the gate runs and its existing check
fails it for any report result other than success.

Refs #1331

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1328 landed on main as 292d827. This branch carried its four original
commits; the merge is clean and the diff against main now holds only the
#1331 change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the status:in-progress 🔵 Agent working label Oct 4, 2026
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

X Brain Health Check

Score: 100.0/100
Status: 🟢 HEALTHY
Threshold: 80/100

X Brain is above merge threshold

@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

🧪 Brain Stress Test Results

Tests: 261/261 passed, 0 leaked
Status: 🟢 PASS
Gate: every test in zig build test-brain-stress passes and none leaks

The capability index (external/t27/tools/trinity_manifest.py) finds build
targets only as `const X = b.addTest(.{ .root_module = b.createModule(.{
.root_source_file = b.path(...) }) })` and `b.step("literal", ...)`. The
helper-built roots and the region-step loop showed up as `test:None` and
hid three steps. Each brain test root and step is now written out, so
every target has a precise name a capability card can own. No test
changes: the same files are compiled with the same imports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag added a commit to gHashTag/t27 that referenced this pull request Oct 4, 2026
…d of the pin (Closes #5953) (#5965)

gHashTag/trinity#1333 puts five brain test steps back into trinity's
build.zig (test-basal-ganglia, test-reticular-formation,
test-locus-coeruleus, test-brain, test-brain-stress) over six src/brain
test roots. Trinity's capability index, which vendors
tools/trinity_manifest.py and these cards byte for byte, found the
eleven targets with no owning card, failed on each as UNASSIGNED_TARGET,
and turned the "Headless profile from a clean clone" job red.

specs/trinity/capabilities/brain.regions.t27 owns exactly those eleven
targets: executable, zig, headless (the steps sit outside every
!ci_mode block), work package S03. Its acceptance is the five
`zig build <step> -Dci=true` commands. Its evidence is measured, from
brain-ci runs 37184075728 (push) and 37184077352 (pull_request) at
gHashTag/trinity@291ac8b24, each step's exit code gated, with the same
counts in both runs: 51/51, 48/48, 52/52, 151/151, and 261/261 with 0
leaked. No canonical spec is claimed: src/brain is handwritten Zig, and
specs/brain/brain.t27 only lists it in a comment table.

research.unreferenced-sources no longer names src/brain; its NOTE says
why, and records that 43 of the 46 .zig files under src/brain stay
unreachable at the pin (37 at 291ac8b24).

The order problem stays visible. None of the eleven targets exists at the
pinned gHashTag/trinity@976df517, so `trinity_manifest.py check` now
reports eleven UNKNOWN_TARGET findings, and the regenerated
conformance/trinity/report.json records them until S01 re-pins. The
card header, the README and the docs/now entry say so. Against an
inventory of 291ac8b24 the card has no finding and none of the eleven is
unassigned. --self-check passes.

Both cards are sealed with t27c built from this branch; seal --verify
matches both.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag and others added 3 commits October 4, 2026 15:45
Brings in #1310 (76f8975) and the other main commits since 291ac8b
before re-vendoring t27 at 39e06aa0e.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ain.regions

The brain test steps this PR restores (test-basal-ganglia,
test-reticular-formation, test-locus-coeruleus, test-brain,
test-brain-stress) and their six src/brain test roots had no S01 card,
so the capability index failed on each as UNASSIGNED_TARGET.
gHashTag/t27#5965 (39e06aa0, Closes gHashTag/t27#5953) adds the card
trinity/brain.regions that owns exactly those eleven targets.

- tools/contracts.py vendor --revision 39e06aa0: external/t27 and the
  lock move from de2a1aca; 76 files (the new card is the 76th). It also
  brings gHashTag/t27#5474: the S01 checker skips build.zig comments and
  counts the vendored copies apart, and the acceptance of mcp.needle-mcp
  and mcp.trinity-mcp becomes zig build -Dci=true && test -x
  zig-out/bin/<name>.
- specs/reproduce/capabilities.t27: trinity/brain.regions is in RUN. The
  profile's test step does not compile its roots, so its five zig build
  test-<region> commands are its only measure in the index.
  mcp.needle-mcp and mcp.trinity-mcp leave KNOWN_BLOCKED: they were
  blocked on steps that exist only in commented-out lines, and their
  new acceptance is the install the profile already makes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… t27@39e06aa0

The vendoring of gHashTag/t27@39e06aa0 added one spec test, the
card_is_bound_to_its_package test of specs/trinity/capabilities/
brain.regions.t27; no other vendored spec changed its number of tests.
t27c built at 39e06aa0 with zig 0.16.0 measures 58 specs and 78 tests,
all passing, so a floor left at 77 would let one test vanish unseen.

The other ledgers were re-checked against the new vendor and stand as
they are: the eleven TEST_BLOCKED specs fail with the same first error
each, lotus.t27 still drops 48 top-level tokens, and the two
specs/tools/mcp specs still declare no test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

brain-ci: Unit Tests call build steps removed in #517

1 participant