Restore snapshots with another scratch size

.github/workflows/arm64-jobs.yml

@@ -201,8 +201,8 @@ jobs:
       SMALL_GUESTS: >-
         --lib --bins --test bash --test cli --test compiled --test cpio --test fs_backend
         --test native_kernel --test networking --test php --test project
-        --test python --test python_shell --test quickjs --test resolv
-        --test snapshot_mounts --test step --test wasmtime
+        --test python --test python_shell --test quickjs --test resize
+        --test resolv --test snapshot_mounts --test step --test wasmtime
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 

CHANGELOG.md

@@ -4,6 +4,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ## [Prerelease] - Unreleased
 
+### Added
+
+- `AppSandbox::snapshot_with_scratch_mb` takes a snapshot that restores with another scratch size: boot with what unpacking the rootfs needs, then run guest function calls with what they need, less or more. The snapshot records the size (`snapshot_scratch_mb`). A size too small for the kernel makes the restore fail (restoring a snapshot that fits recovers the sandbox) and a boot from the snapshot fail. A guest function call that runs out of scratch ends with its process killed (status 137 with a single-process driver, such as python's) or the guest aborted, depending on what runs out first. A restore recovers the sandbox.
+
+### Changed
+
+- The host places the transport arena at a fixed address below the top of scratch, so a snapshot keeps it when restored with a different scratch size. Snapshots saved by an earlier release are refused: save them again. A native Unikraft kernel (`--kernel`, `SandboxBuilder::from_kernel`) built against an earlier `plat/hyperlight` can allocate over the arena: rebuild it against this kernel tree.
+
+### Fixed
+
+- The kernel allocated scratch pages the host reserves at the top of scratch. It now stops below them.
+- A guest that ran out of scratch during a copy-on-write fault crashed the VM (an unexpected exit) instead of aborting.
+- On x86, a page fault the kernel could not handle early in boot or in its fault handler ended the guest as if it had finished: a guest function call that hit one looked complete. The guest aborts.
+
 ## [v0.20.1]
 
 ### Changed

Cargo.toml

@@ -103,3 +103,10 @@ tracing-subscriber = { version = "0.3", default-features = false, features = ["r
 # The FlatBuffer decoder's external-value source, mirrored in the tests.
 anyhow = "1"
 bytes = "1"
+
+# hyperlight main, for scratch resize (#1947). crates.io takes no git
+# dependency, so hyperlight-unikraft is not published until a hyperlight
+# release replaces this.
+[patch.crates-io]
+hyperlight-host = { git = "https://github.com/hyperlight-dev/hyperlight", rev = "a4563dbd585d47bfc0664d7b9559fbf0cb51d2c1" }
+hyperlight-common = { git = "https://github.com/hyperlight-dev/hyperlight", rev = "a4563dbd585d47bfc0664d7b9559fbf0cb51d2c1" }

build.rs

@@ -15,7 +15,7 @@ use sha2::{Digest, Sha256};
 /// the guest MSR list, and the hyperlight-host release that reads a
 /// snapshot.  Bump it when any of those changes in a way a snapshot would
 /// notice; a kernel change rolls the key by itself.
-const SNAPSHOT_CONTRACT: u32 = 7;
+const SNAPSHOT_CONTRACT: u32 = 8;
 
 /// The embedded kernel, whose code and host-call protocol a snapshot
 /// carries in its memory: one per architecture, named as Unikraft names