perf(runtime): cache the hot thread-locals so one allocation pays one _tlv_get_addr

[proggeramlug]

Workstream A of #7469, items 1 and 2: cut the number of thread-local resolutions
on the allocation path. Workstream B (per-object footprint) is not in this PR.

The mechanism

On Darwin every thread_local! access is an out-of-line _tlv_get_addr call in
libdyld. Unlike ELF's local-exec / initial-exec it is not inlined and not
cached across accesses — and, critically, LLVM can CSE repeated accesses to the
same thread-local but two different thread-locals are two different
descriptors. So N distinct thread-locals on one path cost N calls however well
that path inlines. A single {v, w} object literal touched about a dozen.

crates/perry-runtime/src/tls_hot.rs caches those addresses in one
const-initialised thread-local. The storage does not move: every slot is the
address of the existing thread_local! in its owning module, so init order,
lazy init and destructor registration are all unchanged. Slots are untyped
*mut u8 so each owning module keeps its storage type private — which means a
mis-wired fill() would hand out a well-typed reference to the wrong object,
so tls_hot::tests::cached_addresses_match_thread_locals asserts every
address/accessor pairing, and every_slot_is_populated asserts fill() skipped
nothing.

What profiling the remainder turned up

Symbolicated sample over churn.ts, attributing _tlv_get_addr to its
immediate caller, found four things worth fixing beyond the consolidation:

• incremental_mark_barrier_value read a thread-local on every heap-pointer
  store — 91 of 653 attributed samples, all spent proving a null pointer was
  still null. It now consults the existing process-global
  PERRY_INCREMENTAL_MARK_BARRIER_ACTIVE_COUNT first, which is the same
  authority generated code already trusts for exactly this question.
  incremental_mark_barrier_enable now arms that count before installing
  the thread-local pointer (and disable still clears the pointer before
  decrementing), so there is no window where the pointer is live while the count
  reads idle. That ordering was merely tidy before; this PR makes it
  load-bearing, so it is written down at the site.
• js_array_length probed the Map and Set registries on every call — an
  arr.length loop condition paying two thread-local hash lookups. Monotone
  "has anything ever been registered" flags answer for programs that use
  neither. Monotone deliberately: a maintained count is a count that can be got
  wrong, and this one only ever needs to prove nothing exists.
• The page-generation cache had one entry, while the write barrier
  classifies at least two unrelated addresses per store (child, then parent) —
  they evicted each other and it missed on essentially every call, 71 self
  samples in the map lookup the cache exists to avoid. Now 4-way. It lives
  behind an UnsafeCell rather than a Cell because Cell::get returns a
  copy, and copying the set on every classification cost more than the lookup
  (caught as a 2% retain.ts regression while bisecting my own arms).
• layout_forget_object takes one borrow_mut per map instead of a
  borrow to test emptiness plus a second borrow_mut to remove.

gc/hot_tls.rs exists because barrier.rs crossed the 2000-line cap; the
accessor/provider pairs sit together there so the casts are reviewable side by
side.

Results

gc-handoff/bench, interleaved A/B (arms alternate round by round so load drift
hits both equally), best-of-5 user CPU, quiet host, PERRY_NO_AUTO_OPTIMIZE=1:

bench	main	this PR	speedup	RSS main	RSS PR
churn	4.32 s	3.71 s	1.16x	24.5 MB	24.6 MB
cycles	1.09 s	0.99 s	1.10x	29.5 MB	29.7 MB
retain1	1.15 s	1.06 s	1.08x	167.4 MB	168.9 MB
retain	4.49 s	4.18 s	1.07x	422.5 MB	425.3 MB
deeplist	1.76 s	1.65 s	1.07x	163.8 MB	162.9 MB
tree	10.12 s	9.85 s	1.03x	193.0 MB	192.8 MB

Program output is byte-identical on all six. _tlv_get_addr 27.9% -> 23.5% of
leaf samples; attributed callers 653 -> 542.

Where the acceptance criteria stand

Being explicit, because this does not close the ticket:

criterion	status
1. _tlv_get_addr < 5%	not met — 27.9% -> 23.5%. See below.
2. churn >= 2x	not met — 1.16x.
3. tree/retain improve, no RSS regression	met — 1.03x / 1.07x, RSS within 0.9%
4. no GC regressions	met — see below
5. ratchet probes hold	met — see below
6. workstream B, bytes/record	not attempted (separate workstream)
7. cargo test green	met for perry-runtime (1715 tests)

Why 1 and 2 are not reachable by consolidation alone. The floor of this
design is one resolution per runtime FFI call, and generated code makes ~12 of
them per object literal: js_gc_temp_root_push / _get / _truncate are alone
206 of the 542 remaining samples, three separate calls around a single
allocating expression. Driving this below 5% needs the call count to drop, not
the per-call cost — i.e. workstream A item 4 (codegen emits the bump inline),
which the ticket already anticipates as "likely its own ticket once 1 lands".
The other large remaining item on this profile is not TLS at all:
js_typed_feedback_class_field_set_guard + class_field_fast_contract are ~12%
of self time.

Validation

• cargo test --release -p perry-runtime: 1715 pass.
• gc_ratchet vs clean main: the pinned artifact currently fails on main
  itself (28 regressions, identical set on both arms — it predates gc: adaptive tenuring + young-scoped scavenge cap (fixes the large-live-set scavenge regression) #7432/gc: old-generation hole free list — swept holes become reusable capacity (#7437) #7443/
  arena: recycled-block pool — block round-trips through the allocator were the tree.ts RSS term (#7438) #7449), so I measured clean main with the same harness on the same host and
  diffed. All 108 compared metrics agree except heap_used_bytes on 2 of 12
  probes, <=0.8% — and it moves on different probes per build, so it is
  allocation-boundary jitter, not behaviour. Every metric in the gc family
  (minor_cycles, step_cycles, copied_objects, copied_bytes,
  promoted_objects, promoted_bytes, freed_bytes) is identical on all 12,
  and correctness passes everywhere.
• Per-cycle PERRY_GC_TRACE: churn 105 cycles / 0.03 s total pause / 2.x ms
  max, tree 43, retain 11 — unchanged. tree copying volume stays at the
  post-gc: adaptive tenuring + young-scoped scavenge cap (fixes the large-live-set scavenge regression) #7432 level, 0.017 GB / 0.2 M object-copies.
• cargo fmt --all -- --check, scripts/check_file_size.sh clean.

One thing to flag. A single run of the suite showed
gc::tests::root_words::bare_address_in_shadow_slot_survives_a_real_collection
failing (a budgeted cycle stopping early), under load average 17 with another
build saturating the machine. It has not recurred in 33 subsequent runs on this
branch, and clean main is also 33/33 clean, so I could not attribute it either
way. Recording it rather than dropping it.

Refs #7469.

Summary by CodeRabbit

• Performance

  • Improved allocation and garbage-collection runtime performance by reducing overhead in frequently used memory-management paths.
  • Optimized page metadata lookups and registry checks for faster execution.
  • Benchmarks show fewer runtime address-resolution operations and improved runtimes.

• Reliability

  • Preserved existing collector metrics, output, allocation behavior, and memory-management results while applying these optimizations.

• Documentation

  • Added changelog documentation describing the performance improvements and benchmark results.

[coderabbitai]

📝 Walkthrough

Walkthrough

The runtime adds a per-thread hot-TLS address cache and routes allocation, page metadata, GC state, temporary roots, and registry checks through cached accessors. Barrier activation ordering and page-generation caching also change.

Changes

Hot TLS runtime paths

Layer / File(s)	Summary
Hot TLS cache and accessors crates/perry-runtime/src/tls_hot.rs, crates/perry-runtime/src/gc/hot_tls.rs, crates/perry-runtime/src/lib.rs, crates/perry-runtime/src/gc/{mod.rs,roots.rs}, changelog.d/...	The runtime resolves supported TLS addresses once per thread, stores them in HotTls, and exposes typed cached accessors. Tests verify pointer wiring, initialization, and thread isolation.
Arena allocation and page-generation caching crates/perry-runtime/src/arena/{allocators.rs,block.rs,mod.rs,page_meta.rs}	Arena allocation and GC free-list access use cached TLS pointers. Page-generation classification uses a four-entry round-robin cache with full invalidation.
GC barrier, layout, and root access crates/perry-runtime/src/gc/{barrier.rs,hot_tls.rs,layout.rs,roots/temp_roots.rs}	GC barriers use cached state and skip TLS resolution when globally idle. Layout maps and temporary-root operations use cached accessors. Barrier activation publishes the global count before thread-local state.
Map and Set registry short-circuits crates/perry-runtime/src/{map.rs,set.rs}	Map and Set lookups return early when their registries have never been used, based on monotonic atomic flags.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Runtime
  participant HotTls
  participant ArenaAllocator
  participant GCBarrier
  Runtime->>HotTls: resolve and cache thread-local addresses
  ArenaAllocator->>HotTls: read cached arena state
  HotTls-->>ArenaAllocator: return typed arena accessors
  GCBarrier->>HotTls: read cached barrier state
  HotTls-->>GCBarrier: return typed barrier accessors

Loading

Possibly related issues

• perf(runtime): allocation path spends 34% of self time in _tlv_get_addr — 24× behind Node on object churn with the collector already idle #7469 — Directly covers the TLS allocation-path optimization implemented here.

Possibly related PRs

• PerryTS/perry#6811 — Both modify incremental-mark barrier TLS handling and the global active-barrier count.
• PerryTS/perry#6972 — Both modify the temporary-root TLS access path.
• PerryTS/perry#7050 — Both modify the arena allocation hot path.

Suggested labels: run-extended-tests

Suggested reviewers: thehypnoo

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly states that the runtime caches hot thread-locals to reduce _tlv_get_addr calls on the allocation path.
Description check	✅ Passed	The description provides detailed summary, changes, issue reference, benchmarks, validation, and test results, although it omits the template headings and checklist.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch perf/7469-tls-alloc-path

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/perry-runtime/src/arena/page_meta.rs`:
- Around line 551-552: Update the cache-miss documentation above
classify_heap_generation to describe re-priming the four-way
PageGenerationCacheSet via classify_heap_generation_uncached, replacing the
incorrect “one-entry cache” wording.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 48d90df3-6236-4996-bc1b-28ecb92150e9

📥 Commits

Reviewing files that changed from the base of the PR and between a8dee4b and 20b716f.

📒 Files selected for processing (15)

• changelog.d/7474-hot-tls-alloc-path.md
• crates/perry-runtime/src/arena/allocators.rs
• crates/perry-runtime/src/arena/block.rs
• crates/perry-runtime/src/arena/mod.rs
• crates/perry-runtime/src/arena/page_meta.rs
• crates/perry-runtime/src/gc/barrier.rs
• crates/perry-runtime/src/gc/hot_tls.rs
• crates/perry-runtime/src/gc/layout.rs
• crates/perry-runtime/src/gc/mod.rs
• crates/perry-runtime/src/gc/roots.rs
• crates/perry-runtime/src/gc/roots/temp_roots.rs
• crates/perry-runtime/src/lib.rs
• crates/perry-runtime/src/map.rs
• crates/perry-runtime/src/set.rs
• crates/perry-runtime/src/tls_hot.rs

[coderabbitai]

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Correct the cache description.

Line 552 says “one-entry cache”. classify_heap_generation_uncached now inserts into the four-way PageGenerationCacheSet.

Proposed fix

-/// re-prime the one-entry cache.
+/// re-prime the multi-way cache.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	/// Cache-miss arm of [`classify_heap_generation`]: consult the page map and
	/// re-prime the one-entry cache.
	/// Cache-miss arm of [`classify_heap_generation`]: consult the page map and
	/// re-prime the multi-way cache.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/perry-runtime/src/arena/page_meta.rs` around lines 551 - 552, Update
the cache-miss documentation above classify_heap_generation to describe
re-priming the four-way PageGenerationCacheSet via
classify_heap_generation_uncached, replacing the incorrect “one-entry cache”
wording.

[proggeramlug]

Reviewed and merged. This got the deepest look of today's batch because it touches the write-barrier classification path, where a stale answer is a lost object. What I verified independently on a rebased build:

1. Every cached slot is the address of a thread-local's own storage (KEY.with(|c| c.get() as *mut u8)), const-init, no Drop — stable for the thread's life. The Vec cases cache the address of the Vec struct, not its data pointer, so buffer reallocation is unaffected. Nothing heap-managed is cached, so no root scanner is owed.
2. The 4-way page-generation cache keeps the pre-existing invalidation invariant: generation flips only happen through register_block_space/unregister_block_generation, both of which clear all ways; register_old_object_pages changes the object index, not the generation, so its lack of invalidation is correct. The diff removes no invalidation call.
3. The barrier fast-path veto is semantically additive: when the global count is nonzero it falls through to the exact pre-existing TLS check, and a thread only ever consults its own barrier state, so Relaxed is sufficient; the arm-before-install / clear-before-decrement ordering closes the only window.
4. Battery: 1716 lib tests, all three tls_hot tests (including per-thread isolation), fmt, raw-handle debt gate, file-size, dev build — all green. 92-test gap sample (json/array/map/gc): 90 pass, 2 failures both proven pre-existing (one reproduced on a pre-PR main build, one the known node_fail).

The honest "acceptance criterion 1 not met — 27.9% → 23.5%" framing is appreciated and correct to leave #7469 open. Worth noting for the remainder: the measured wins (1.03–1.16×) line up with the temp-root/TLS share I saw on bench_array_ops, so the residual is likely the per-store classification itself rather than TLS resolution — the 4-way cache hit rate under real barrier traffic would be the next number to pull.