Repository navigation
Restore snapshots with another scratch size - #162
Merged
Merged
Conversation
danbugs
added
the
ci/arm64
There was a problem hiding this comment.
🔵 Needs a closer look
Snapshot memory-layout changes depend on unmerged host and kernel work and require final human validation.
No open issues 🎉
What changed in this PR
Adds smaller-scratch snapshots so guests can boot with ample memory and restore with less, reducing restore costs.
Changes:
- Adds
snapshot_with_scratch_mband documents sizing and recovery behavior. - Tests resized snapshots, persistence, and failure recovery.
- Updates snapshot compatibility and temporarily pins unmerged host and kernel dependencies.
| File | Description |
|---|---|
| tests/resize.rs | Tests resizing, persistence, limits, and recovery. |
| src/lib.rs | Adds the resized-snapshot API and saturating size conversion. |
| docs/execution.md | Documents smaller-scratch restores. |
| CHANGELOG.md | Records the API and compatibility changes. |
| Cargo.toml | Pins experimental Hyperlight dependencies. |
| Cargo.lock | Locks the pinned Hyperlight revision. |
| build.rs | Bumps the snapshot contract to 6. |
| .gitmodules | Selects the scratch-resize kernel fork. |
| .github/workflows/arm64.yml | Includes resize tests in arm64 CI. |
Review effort: Balanced
danbugs
force-pushed
the
scratch-resize
branch
from
October 29, 2026 19:05
719453b to
583163e
Compare
danbugs
changed the title
danbugs
force-pushed
the
scratch-resize
branch
from
October 29, 2026 19:30
583163e to
7ab54b6
Compare
Pin hyperlight-host and hyperlight-common to hyperlight main at a4563dbd, which has scratch resize (#1947), here and in the crates that build hyperlight-unikraft from this tree (integrations/clickhouse, demos/pptx-gen). crates.io takes no git dependency, so hyperlight-unikraft is not published until a hyperlight release replaces the pin. hyperlight main builds bindgen 0.73, which needs syn 3.0.9 or later and prettyplease 0.3. Signed-off-by: danbugs <danilochiarlone@gmail.com>
Bump the kernel (plat-hyperlight-v2 0d9edd33 -> 9f414729), and rebuild the kernels and the native-kernel fixture: - The allocator stopped two pages below the top of scratch, but Hyperlight reserves twelve there, and the early fault handlers had no limit. Allocation stops at the reserved pages, or at the transport arena when the host places it below them. - A guest that ran out of scratch inside a copy-on-write fault crashed again reporting it and triple-faulted the VM. It aborts. - On x86, an unhandled early page fault halted the guest, which the host takes as a normal end. It aborts. - The scratch base follows the scratch size, so the kernel runs from a snapshot restored with another scratch size. The kernel runs on hyperlight 0.18.0 as well as on hyperlight main. Signed-off-by: danbugs <danilochiarlone@gmail.com>
Booting needs room to unpack the rootfs; guest function calls need far less, or sometimes more. snapshot_with_scratch_mb takes a snapshot that a sandbox restores with `mb` MiB of scratch, through hyperlight's SnapshotOptions::scratch_size, so every restore of a warm image costs less, or guest function calls get more room than the boot had. The snapshot records `mb` as its guest's size, so snapshot_scratch_mb and a restore go by the size it restores with. It errors on a size above the maximum memory size or too small for the transport and page tables; a size too small for the kernel fails the restore, and a restore of a snapshot that fits recovers the sandbox. The host's scratch layout changed (the transport arena sits at a fixed address below the top of scratch), so SNAPSHOT_CONTRACT goes to 8 and older snapshots are refused. tests/resize.rs covers shrinking and growing, the record, the error paths and guest function calls that outgrow a snapshot; the arm64 job runs it too. Signed-off-by: danbugs <danilochiarlone@gmail.com>
danbugs
force-pushed
the
scratch-resize
branch
from
October 30, 2026 00:14
7ab54b6 to
17c4fde
Compare
danbugs
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
AppSandbox::snapshot_with_scratch_mb, which takes a snapshot that restores with less or more scratch than the guest booted with. hyperlight has not released scratch resize (#1947) yet, so this pins hyperlight-host and hyperlight-common to hyperlight main by rev, which keeps hyperlight-unikraft off crates.io until a hyperlight release replaces the pin.plat-hyperlight-v29f414729. It stops allocating at the host's reserved pages and the transport arena, and aborts cleanly when scratch runs out in a copy-on-write fault or an early x86 page fault. It runs from a resized restored scratch, and works on hyperlight 0.18.0 too.snapshot_with_scratch_mbgoes through hyperlight'sSnapshotOptions::scratch_sizeand records the target size in the snapshot's metadata.SNAPSHOT_CONTRACTis 8, so snapshots saved by v0.20.1 and earlier are refused.tests/resize.rscovers shrinking, growth and the record, and also runs in the arm64 job.