Skip to content

fix(catalog-gate): private scratch dir per regeneration, so concurrent runs stop failing each other (Closes #6058) - #6059

Merged
gHashTag merged 3 commits into
masterfrom
fix/catalog-gate-scratch-race
Oct 4, 2026
Merged

gHashTag merged 3 commits into
masterfrom
fix/catalog-gate-scratch-race

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 4, 2026

Copy link
Copy Markdown
Owner

Closes #6058

What

generate_emitted in bootstrap/src/catalog_gate.rs regenerated the numeric catalog into one fixed $TMPDIR/t27-catalog-emitted, removed and recreated by every run. Two runs at once removed each other's output, and the loser reported emitted-unreadable for a catalog that is fine (a false Phase 7 failure, seen 2026-10-04 with two concurrent t27c suite runs).

  • Each run now creates its own directory: pid plus a process-wide counter, create_dir, so an existing directory is never adopted or removed. A Scratch guard removes it after the comparison.
  • The emitted line names that directory and says it is removed. The old generated into ... text was overwritten on every path and never reached the report.

Proof, built and run on GitHub's runners (not on a workstation)

The branch has two commits, pushed one at a time on purpose:

  1. 9cebc3968 (control): two of the new tests on master's code, unfixed. They are expected to FAIL in Bootstrap Test Ratchet: concurrent_regenerations_do_not_remove_each_other (8 threads through a Barrier) and the_emitted_line_names_the_regeneration_and_its_removal.
  2. 16cf34854 (fix): the same tests plus a_regenerated_directory_is_private_and_removed_by_its_owner, which needs the new type and so cannot be a control. Expected to pass, with no test newly failing against the baseline.

The run links for both heads are added to this PR as comments once they finish.

Boundary

bootstrap/src/catalog_gate.rs only. bootstrap/ is the owner's to merge.

🤖 Generated with Claude Code

…er (control, fails on master)

Two tests on master's code, before the fix, so CI shows them failing:
8 threads released together through a Barrier into check_emitted_at,
and the emitted line that never says where the catalog was regenerated.

Refs #6058

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-04 12:39:42 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 44
PRs with All Checks Green 6
READY 5
FAILING 44
PENDING 0
NO CHECKS YET 0

These columns do not partition: 5 + 44 + 0 + 0 = 49, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=f668b2a441df != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

gHashTag and others added 2 commits October 4, 2026 20:14
…on master's code in CI (Refs #6058)

Control run 37202843986 on 9cebc39: the_emitted_line_names_the_regeneration_and_its_removal
FAILED as expected, but concurrent_regenerations_do_not_remove_each_other passed: one
round of 8 threads finished before any two overlapped. A regression test that passes on
the bug is not a control. Each thread now regenerates 25 times back to back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d fixed path

generate_emitted wrote into $TMPDIR/t27-catalog-emitted, removed and
recreated by every run: two runs at once removed each other's output
and the loser reported emitted-unreadable for a catalog that is fine.
Each run now creates its own directory (pid plus a process-wide counter,
create_dir so an existing one is never adopted or removed) and removes it
when the comparison is done. The emitted line names that directory and
says it is gone; the old "generated into" text was overwritten on every
path. Adds the third test (a directory is private and removed by its
owner), which needs the new type and so cannot run as a control.

Closes #6058

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-04 13:16:21 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 40
PRs with All Checks Green 10
READY 10
FAILING 40
PENDING 0
NO CHECKS YET 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=001c24236320 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag

gHashTag commented Oct 4, 2026

Copy link
Copy Markdown
Owner Author

Control on master's code, built and tested by Bootstrap Test Ratchet on GitHub's runners:

  • 9cebc3968 (run 37202843986): the_emitted_line_names_the_regeneration_and_its_removal FAILED; concurrent_regenerations_do_not_remove_each_other passed -- one round of 8 threads finished before any two overlapped, so that test was not yet a control.
  • 786ad16be (run 37204999537): each thread now regenerates 25 times. Both tests FAIL on master's code; the race showed up as the comparison did not run: unparsable: EOF while parsing a value (a reader got a file another run was rewriting), not only as emitted-unreadable.

The fix e33358ada is pushed on top; its run link follows when it finishes. duplicate-bodies names ordinary_if / wide_tail, which this branch does not touch -- checked against master on the fix head.

@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-04 13:42:01 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 40
PRs with All Checks Green 10
READY 10
FAILING 40
PENDING 0
NO CHECKS YET 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=a47525693295 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag

gHashTag commented Oct 4, 2026

Copy link
Copy Markdown
Owner Author

Fix head e33358ada: Bootstrap Test Ratchet passes (no test newly failing against the baseline), so concurrent_regenerations_do_not_remove_each_other, the_emitted_line_names_the_regeneration_and_its_removal and a_regenerated_directory_is_private_and_removed_by_its_owner pass on the fix. Both controls fail on master's code (runs 37202843986, 37204999537).

duplicate-bodies is red on master too, from 0886f014c (13:09Z; b1e3c159f at 13:06Z was green); ordinary_if / wide_tail are not in this branch's diff. spec-guards and fpga-conformance fail as they do on master.

This was referenced Oct 4, 2026
@gHashTag
gHashTag merged commit b1b85e6 into master Oct 4, 2026
31 of 34 checks passed
@gHashTag
gHashTag deleted the fix/catalog-gate-scratch-race branch October 4, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

catalog-gate: concurrent runs remove each other's regenerated catalog (shared $TMPDIR/t27-catalog-emitted)

1 participant