perf(runner-shared): decode memtrack frames for module events in parallel

[not-matthias]

TLDR: Finding the module events decoded the whole memtrack artifact on one thread through a streaming deserializer. Every frame is a self-contained zstd frame, so frames are now decoded in parallel from an in-memory buffer (stacked on #576).

• decode_module_events splits the artifact into its zstd frames and decodes them on the rayon pool. Each worker decompresses a frame into a reused buffer and deserializes MemtrackEvent from that slice, so values are read in place instead of copied out of a stream first. Results keep artifact order.
• The runner maps the artifact file instead of streaming it.
• The Mapping/Fork/Exec filter moves to MemtrackEventKind::is_module_event.
• A new test checks the decoder against a filtered full decode over every event kind, several frames and a truncated last frame.
• The bench gets back its 10M and 100M sizes, now fast enough to run. The 100M artifact with stacks is about 4.65 GB on disk.

memtrack_reader bench, local walltime on a 32-thread machine:

Events	without stacks	with stacks
1M	258 → 15.4 ms	267 → 17.1 ms
10M	2.59 s → 101 ms	2.73 s → 114 ms
100M	26.0 s → 1.00 s	27.1 s → 1.05 s

Review notes

• decode_module_events now takes &[u8] and returns a Vec.
• A truncated last frame is still streamed, so the events before the cut are found as before. A frame in the middle that fails to decompress now returns an error; the streamed decoder stopped there silently.
• CodSpeed simulation runs threads one at a time, so it only shows the single-threaded gain from decoding a buffer instead of a stream. The 10M and 100M cases will still take several minutes there, mostly generating the artifact.

[codspeed]

Merging this PR will improve performance by 29.44%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 8 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

• find_module_events[1000000]
• find_module_events_with_stacks[1000000]
• encode_events_realistic[8]
• encode_events_realistic[2]
• encode_events_realistic[1]
• encode_events_realistic[16]
• encode_events_realistic[4]
• sleep 1

ℹ️ 6 benchmarks have no flame graph

The flamegraph could not be generated because we failed to resolve the symbols in the profiling trace. This happens when we fail to harvest information about language frames.

• find_module_events[1000000]
• find_module_events_with_stacks[1000000]
• find_module_events_with_stacks[10000000]
• find_module_events_with_stacks[100000000]
• find_module_events[10000000]
• find_module_events[100000000]

⚡ 2 improved benchmarks
❌ 2 (👁 2) regressed benchmarks
✅ 33 untouched benchmarks
🆕 4 new benchmarks
⏩ 6 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡ ℹ️	WallTime	find_module_events[1000000]	1,269.3 ms	186.8 ms	×6.8
⚡ ℹ️	WallTime	find_module_events_with_stacks[1000000]	1,305.6 ms	194.4 ms	×6.7
🆕 ℹ️	WallTime	find_module_events_with_stacks[10000000]	N/A	1.1 s	N/A
🆕 ℹ️	WallTime	find_module_events_with_stacks[100000000]	N/A	10.9 s	N/A
🆕 ℹ️	WallTime	find_module_events[10000000]	N/A	1 s	N/A
🆕 ℹ️	WallTime	find_module_events[100000000]	N/A	10.4 s	N/A
👁	Memory	find_module_events_with_stacks[1000000]	1.1 MB	4.6 MB	-75.18%
👁	Memory	find_module_events[1000000]	1.1 MB	4.4 MB	-75.2%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing cod-3819-parallel-module-event-scan (5ff86d7) with cod-3819-memtrack-investigate-8-minutes-spent-in-teardown (2f1f983)}

Footnotes

1. 6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no blocking issue was found.

Summary

The PR finds module events by decoding separate zstd frames in parallel. It keeps artifact order, streams a shortened final frame, maps artifact files, and restores larger walltime benchmarks.

• Both callers use the new slice input and returned Vec.
• The new test compares complete and shortened artifacts with a filtered streamed decode.
• No new actionable issues were found.
• The previous RAM thread was manually resolved by not-matthias without an explanatory reply. The current code streams each frame instead of expanding the whole frame into a buffer.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Map artifact file] --> B[Split complete zstd frames]
  B --> C[Stream complete frames in parallel]
  C --> D[Keep Mapping, Fork, and Exec events]
  D --> E[Combine results in frame order]
  B --> F[Stream remaining tail]
  F --> G[Append module events before the cut]
  E --> G
  G --> H[Rebuild process mappings]

Loading

_{Reviews (2) · Last reviewed commit: "perf(runner-shared): stream each memtrac..." · Reviewed by Greptile}

[GuillaumeLagrange]

quite neat, olgtm