fix(codegen): give each .ll compile a private temp dir so it can be deleted again (#7144)

[proggeramlug]

Fixes #7144.

compile_ll_to_object stopped unlinking its temp .ll in #7135, on purpose:
that PR had just made the name a pure function of the IR (#7131 — clang records
a translation unit's basename into the ELF object), so two workers holding
identical IR now shared the path and a per-call unlink could race a sibling
that had computed the path but not yet handed it to clang.

Nothing else deleted them. Because the name is content-addressed, the leftovers
are bounded by distinct IR ever compiled on the machine — which sounds benign
until you notice that working on the compiler changes the IR on essentially every
rebuild. 1627 files / 951.8 MB after one day on one box; the temp dir of the Mac
this was written on still held 1069 files / 635 MB from before the fix.

The fix: stop sharing, rather than delete more carefully

Every .ll → .o compile gets a directory it owns:

$TMPDIR/perry_llvm_scratch_<pid>_<counter>/
    perry_llvm_<fnv1a64(ir)>.ll          <- content-addressed, unchanged
    perry_llvm_<hash>_<pid>_<counter>.o  <- uniquifiers kept (#7140/#509)

and remove_dir_alls it once the object bytes are in memory.

Why this survives the identical-IR race, where a delete-after-use does not.
The race needs a shared name. There isn't one any more: the directory carries
every uniquifier, so no two calls — in one process or across processes — are ever
handed the same .ll path, and unlinking is unobservable to anyone else. There
is no window to lose rather than a narrow one, which matters because a narrow
window is not testable: sabotaged to the naive shape (one flat shared .ll,
unlinked after use), the 8-way concurrent test in this PR went red in one
full-suite run and green in the next three. The structural property is what is
asserted; the concurrency test is the end-to-end complement, and says so in its
own comment.

Why it does not undo #7131. The directory carries the uniquifiers, the
basename carries none — and the object records the basename and nothing else.
That was measured by hand once, on a Pi, in #7140. It is now a test
(the_ll_directory_is_not_recorded_in_the_object_but_the_basename_is): compile
one .ll under the same basename from two directories, for both Linux ELF
targets, assert the objects are identical, with a live control that a different
basename does change them. Cross-compiling because the embedding is a property of
the ELF writer, not of the host — so this runs on the Mac where the defect class
is otherwise invisible.

PERRY_DEBUG_SYMBOLS is not an exemption — the premise was false

The issue asks whether -g builds should keep the file, since they "embed the
absolute .ll path in DWARF". They do not. Measured on a real Perry module
(Apple clang 21, -target x86_64-unknown-linux-gnu and aarch64-unknown-linux-gnu):

-g object vs no--g object	byte-identical
.debug_* sections in the -g object	none
DICompileUnit/DIFile/!dbg in Perry's IR	none (grep, whole crate)

clang -g on a .ll lowers debug info that is in the IR; it does not
synthesise a compile unit for the input file. So nothing records where the .ll
was, and nothing needs to outlive the compile.

I had first implemented the exemption — a second DebugShared layout — and then
deleted it, because a mode justified by something that does not happen is exactly
the "unexercised mode nobody has verified" CLAUDE.md says to remove. One layout
now. Both halves of the measurement are tests, including a direct assertion that
the -g object contains no .debug_ section name (an equality assertion can be
defeated by an edit to itself; a claim about the artifact cannot). Not measured
on COFF/Windows — stated in the code.

Failure policy

Unchanged and now pinned: a failed compile keeps its IR. The error already
names the file; failed_compile_keeps_the_ll_for_diagnosis asserts exactly one
.ll survives anywhere under the temp root and that the message names it. Same
for PERRY_LLVM_KEEP_IR, which now keeps .ll, .o, .clang-stderr and the
compile plan together in one directory instead of scattered across the temp root.

Red then green

census-temp-hygiene (new): compile the corpus with TMPDIR pointed at an empty
directory, then look in it. Two real arms, built sequentially from one target dir,
distinct binary hashes:

arm	54 compiles (27 workloads x 2, concurrent)
df7214b0d + main's linker.rs	exit 1 — 27 entries left behind
this branch	exit 0 — 0 left behind

Note the shape of the number: 27, not 54. Repeats share a content hash and so
share a filename. "No growth run-over-run" would have been green on the broken
compiler — which is why CI never saw this and developer machines filled up. The
gate asserts the absolute property instead.

The other two acceptance criteria

Determinism holds — census-determinism --repeat 3 --jobs 4: 27/27 workloads
byte-identical; --repeat 4 --jobs 8 twice more, same. #7140's concurrency
property holds — 24 concurrent perry processes compiling one identical
source, three runs: 0/24 failures, one distinct object hash each time.

No behavioural change: across all 27 census workloads the objects emitted by
main and by this branch are byte-identical, 0/27 differences — including the
raced-repeat fixture, whose hash 94f09ed6782d4917 is the same on both arms.

Sabotage-verified, both directions

Every check was made to fail on purpose before being believed:

sabotage	result
never delete the scratch dir (the #7135 shape)	3 lifecycle tests RED
flat shared .ll, unlinked after use (the naive fix)	5 tests RED
pid back in the .ll basename (re-break #7131)	3 tests RED, incl. both #7135/#7140 tests
clean up on the failure path too	failure-policy test RED
PERRY_LLVM_KEEP_IR deletes anyway	keep test RED
ELF probe loses its -target (falls back to Mach-O)	directory-recording test RED
assert the -g object does have .debug_	-g test RED
harness verdict always green / accepts a 0-compile run / truncates silently	self-test RED (x3)

Found on the way, filed not fixed: #7167

The hygiene gate went red on its first real run — on someone else's files.
run_pipeline.rs creates a perry-objs-<pid>-<nanos>/ object staging directory
and removes it on both link exits; --no-link returns before either, so every
--no-link compile leaks the directory and its objects. Unlike #7144 that is
unbounded in compiles, not in distinct IR, and the objects are much larger.
Filed as #7167 and left alone: it is crates/perry/src/commands/compile/, not
this module.

So this gate fails on the clang driver's own temp names (perry_llvm_*,
perry_cgu_*, perry_bc_*) and reports anything else without failing — a gate
that goes red for another module's defect gets muted rather than fixed. Widening
OWNED_PREFIXES to "nothing at all" is a one-line change once #7167 lands, and
the comment there says so.

What I could not verify

• No Linux host. The Pi was not reachable from this machine (no ssh entry, no
  mDNS), and the Mini belongs to another agent. So census-determinism was not
  re-run on real ELF hardware. What that check would add over what is here: the
  ELF-specific risk this change introduces is "does the new per-call directory
  reach the object", and that is measured directly above, by cross-compiling to
  both Linux targets — the embedding is a property of the ELF writer, not of the
  host. The corpus-level Linux run is still worth doing when a Pi is free.
• COFF/Windows -g behaviour is not measured, only ELF and Mach-O.
• The pre-existing garbage is not reclaimed. I deliberately did not sweep old
  perry_llvm_*.ll files from the compile path: an age-based sweep can unlink a
  file a concurrently running old binary is about to reuse via its
  "already exists, same content" fast path, and several worktrees on this machine
  are doing exactly that. One-time reclaim, when the box is quiet:
  find "$TMPDIR" -maxdepth 1 -name 'perry_llvm_*.ll' -delete.
• main is independently red: manifest_consistency (missing API manifest
  entries for iovalkey, dgram::sendto, sqlite::{serialize,deserialize}) and
  the addr-class ratchet (child_process/value_util.rs, fs/dirent.rs). Neither
  file is in this diff. Judge CI against main's, not against green.
• Not re-run: the gap/parity suite. This change cannot reach it — the
  byte-identical-objects result is the direct evidence.

Note on the file split

linker.rs crossed the 2,000-line cap, so the new tests live in
linker_temp_lifecycle_tests.rs via #[path] (repo precedent:
object_cache_tests.rs, type_analysis_tests.rs). linker.rs is 1,803 lines.

[coderabbitai]

Warning

Review limit reached

@proggeramlug, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 17 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 556705c7-fa08-46a7-84fe-76b48151773a

📥 Commits

Reviewing files that changed from the base of the PR and between 5f5006a and 8bd0d30.

📒 Files selected for processing (7)

• .github/workflows/test.yml
• benchmarks/repsel_census/README.md
• changelog.d/7168-ll-temp-file-lifecycle.md
• crates/perry-codegen/src/linker.rs
• crates/perry-codegen/src/linker_temp_lifecycle_tests.rs
• scripts/compiler_output_harness/cli.py
• scripts/compiler_output_harness/repsel_temp_hygiene.py

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/7144-ll-temp-lifecycle

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}