Investigate & benchmark cold TCG boot-time optimizations (virtio-rng, random.trust_cpu, snapshot/restore)

[Xiddoc]

First boot of the binder: vm TCG path took ~8 minutes on a no-KVM host (accel: auto correctly fell back to TCG). The QEMU launcher (vm/qemu.py ~252–276) uses -accel tcg,thread=multi,tb-size=1024 -cpu max and appends console=ttyS0 root=/dev/vda rw init=/init panic=1 mitigations=off — notably no RNG device and no random.trust_cpu=on, so Android init may stall on entropy (amplified under TCG).

This is an investigation, not a confirmed fix: benchmark cold-boot wall time with vs. without -device virtio-rng-pci and with random.trust_cpu=on, on the same host, several runs each. Separately, document/validate snapshot/restore (cli.py:snapshot/restore) as the warm-start path — boot once, snapshot the booted state, restore for near-instant subsequent starts.

Do not add vCPUs: the design log (vm-rnd-log §B.5) and resolve_smp already found -smp 4 optimal and -smp 8 regressing under MTTCG.

Acceptance: a short report with numbers on the RNG options, and a documented snapshot/restore warm-start workflow.

[Xiddoc]

Report — benchmarked on a binderless, KVM-less host under pure TCG

Investigated in #89. All numbers are real boots using the committed prebuilt 6.12.9 kernel (vm-kernel-6.12.9-bb5e50f8d3e1, the artifact beetroot build --vm-kernel ships). Minimal busybox rootfs + a static blocking-getrandom() probe; 4 runs per cell.

RNG options — negative result

cmdline / device	crng init done	getrandom() blocked
production (-cpu max, no RNG flags)	~0.13 s	0.000 s
random.trust_cpu=on	~0.1 s	0.000 s
+ virtio-rng-pci	~0.1 s	0.000 s
random.trust_cpu=off	~4.5 s	~0.35–0.39 s
control: -cpu qemu64 (no RDRAND)	~3.2 s	~0.39 s

Entropy is not the bottleneck. -cpu max exposes RDRAND and the pinned kernel's defconfig sets CONFIG_RANDOM_TRUST_CPU=y, so the CRNG is seeded at ~0.13 s before userspace — getrandom() never blocks. virtio-rng-pci is inert (rng_current=none): CONFIG_HW_RANDOM_VIRTIO is =m in defconfig and the guest builds with CONFIG_MODULES off, so it's compiled out.

Decision:

• ✅ Added random.trust_cpu=on to the guest cmdline — zero cost, boot-neutral on -cpu max, but explicit insurance against the measured ~3–4.5 s slip if RDRAND is hidden or the defconfig default flips.
• ❌ Did not add virtio-rng / the kernel driver: it would change the kernel.config fingerprint → fetch_prebuilt misses the release → every build --vm-kernel falls back to a ~7-min source compile, for zero boot benefit (net regression). Same shape as the Stage D mitigations=off finding.

Snapshot/restore warm-start — the real lever (validated)

The cold cost is the ~100 s of CPU-bound ART/Zygote boot, not entropy. QEMU savevm/loadvm (#49) skips it:

phase	measurement
savevm booted (110 MiB RAM → qcow2)	~5.5 s
loadvm booted → guest live	1.91 s

The resumed guest's /proc/uptime and wall clock were identical to pre-save — a true RAM restore, not a re-boot. Documented the snapshot-(cold)-vs-savevm-(warm) workflow in the sandbox quickstart and the full harness + numbers in R&D log §E.

---

Generated by Claude Code

[Xiddoc]

Report: cold TCG boot-time investigation + warm-start

Benchmarked on a binderless, KVM-less host (4 physical cores, 15 GiB RAM, no /dev/kvm) against the committed recipe — prebuilt bzImage 6.12.9 + build_rootfs, Android 14, pure TCG, -smp 4, MTTCG, 8 GiB. Metric: host wall to first ADB interaction (adb getprop sys.boot_completed=1), fresh qcow2 overlay per boot. Implemented in #93.

1. RNG options — benchmarked, boot-neutral ❌

Two interleaved rounds, guest boot_seconds (×2) and host wall (×2):

Arm	guest boot_seconds	host wall (s)
baseline (shipped)	188, 192	223.5, 228.5
random.trust_cpu=on	196, 193	232.0, 230.6
-device virtio-rng-pci	193, 192	223.6, 224.0
both	223, 186	258.3, 221.9
both + cache=unsafe	194, 181	232.2, 214.2

Every arm clusters at 181–196 s with no arm consistently faster — the lone high sample (both=223 in round 1) is exposed as host-contention noise by round 2, where the same flags gave 186 s, the fastest run of the sweep.

Root cause (from the serial log):

• CRNG is already seeded at ~0.19 s every boot (random: crng init done). x86_64 defconfig ships CONFIG_RANDOM_TRUST_CPU=y and -cpu max exposes RDRAND even under TCG, so random.trust_cpu=on is redundant — there's no entropy stall to remove.
• virtio-rng-pci is inert here: the guest logs prng_seeder: … Unable to open hwrng /dev/hw_random even with the device attached, because CONFIG_HW_RANDOM_VIRTIO is =m in defconfig and the module-less guest ships no modules. (That service failing also doesn't gate boot_completed.)
• cache=unsafe is neutral too — the boot is TCG-CPU-bound (emulating ART/Zygote/system_server), not entropy- or disk-I/O-bound.

→ The cold-path QEMU argv is left unchanged (no RNG flags added). Full write-up + serial evidence: docs/design/vm-rnd-log.md Stage E.

2. Snapshot/restore warm-start — ~22x, shipped ✅

Since the cold boot is irreducibly CPU-bound under TCG, the real lever is to not boot on repeat starts. Boot once, QEMU savevm the running machine state into a qcow2 overlay, then -loadvm to resume:

Phase	wall
cold first boot → first host ADB	~222 s
savevm checkpoint write	~1 s (overlay ~2.2 GiB)
warm resume (-loadvm) → first host ADB	~10 s

This shipped in #93 as the opt-in vm.boot_cache flag. Validated end-to-end through the real beetroot CLI (not just a harness):

COLD up: 237s → "cached boot snapshot for 'vmcache'"; overlay 2.2G; boot_completed=1
WARM up: "resuming cached boot snapshot (warm start)" → boot_completed=1, ro.build.version.release=14

The e2e surfaced (and the PR fixes) two real integration bugs a unit test alone missed: the HMP monitor drain broke on the connect banner before savevm ran, and the checkpoint wasn't gated on a real sys.boot_completed poll (plain adb connect succeeds the instant QEMU's hostfwd binds). Both fixed; savevm now reliably captures a fully-booted guest.

Note this is distinct from beetroot snapshot (which packs cold /data and still requires a full boot to restore). Resume reverts to the checkpoint each time — a fast known-good boot, not persistence.

Bottom line for Claude-Cloud-class hosts

No cold-boot micro-lever helps (CPU-bound under TCG). The win is vm.boot_cache: true — after the first ~3-4 min boot, every subsequent up resumes in seconds. Docs: config § warm-start boot cache and the sandbox quickstart.

---

Generated by Claude Code