feat(aarch64): add GDB debugging for Windows ARM64 WHP by cshung · Pull Request #1907 · hyperlight-dev/hyperlight

cshung · 2026-10-08T17:36:47Z

Summary

Windows ARM64 WHP guests support GDB register and memory access, two hardware breakpoints, four-byte BRK software breakpoints, and hardware single-step.

Guest synchronous exception vectors forward debug stops through HVC. Interrupted register context and breakpoint configuration survive cache synchronization and vCPU reset. Asynchronous vectors retain their normal exception paths.

Recovery

Poisoned sandbox restore discards software breakpoint and pending-step state.
Crash detach disables backend debugging even when breakpoint memory is unavailable.
Snapshot checkpoints reject installed software breakpoints before replacing memory.
The cache-sync trampoline is read-only to guests. Exception-vector tests use a separate writable stack.
Host cache flushing resolves layered snapshot memory through the guest memory view.

Validation

Local Windows ARM64/WHP:

Five real GDB client tests and 14 backend/recovery tests pass on this branch.
Formatting, standard Clippy, and GDB-enabled Clippy pass.
The full standard test suite and build pass on the combined WHP/KVM source tree. Its contents are unchanged by the stack rebase.

Copilot

🟡 Changes recommended

Breakpoint state can survive crash or cancellation recovery, and the hidden cache-sync mapping conflicts with the public mapping contract.

4 open findings

Clean up debugger breakpoints after crash detach · New
Cancel ARM64 software steps for all cancellation sources · New
Reserve hidden GPA 0x1000 in the mapping contract · New
Restore corrupted mathematical symbols in comments · New

What changed in this PR

Adds Windows ARM64 WHP guest debugging and generalizes the GDB implementation across architectures.

Changes:

Adds ARM64 registers, software breakpoints, stepping, and cache synchronization.
Extracts shared debugger logic from x86_64 code.
Updates tests, documentation, and platform configuration.

File	Description
`CHANGELOG.md`	Records ARM64 GDB support.
`docs/how-to-debug-a-hyperlight-guest.md`	Documents architecture-specific capabilities.
`src/hyperlight_host/build.rs`	Enables GDB on Windows ARM64.
`src/hyperlight_host/examples/guest-debugging/main.rs`	Adds ARM64 debugging tests.
`src/hyperlight_host/src/hypervisor/gdb/arch.rs`	Dispatches architecture-specific logic.
`src/hyperlight_host/src/hypervisor/gdb/arch/aarch64.rs`	Implements ARM64 breakpoint and step decoding.
`src/hyperlight_host/src/hypervisor/gdb/arch/x86_64.rs`	Houses extracted x86_64 logic.
`src/hyperlight_host/src/hypervisor/gdb/event_loop.rs`	Maps ARM64 stop reasons.
`src/hyperlight_host/src/hypervisor/gdb/mod.rs`	Extends shared debugging interfaces.
`src/hyperlight_host/src/hypervisor/gdb/target.rs`	Adds architecture-specific GDB registers.
`src/hyperlight_host/src/hypervisor/hyperlight_vm/aarch64.rs`	Connects GDB to ARM64 VMs.
`src/hyperlight_host/src/hypervisor/hyperlight_vm/debug.rs`	Centralizes breakpoint and stepping state.
`src/hyperlight_host/src/hypervisor/hyperlight_vm/mod.rs`	Handles ARM64 debug exits.
`src/hyperlight_host/src/hypervisor/hyperlight_vm/x86_64.rs`	Removes extracted debugger logic.
`src/hyperlight_host/src/hypervisor/virtual_machine/mod.rs`	Extends VM exits and errors.
`src/hyperlight_host/src/hypervisor/virtual_machine/whp/aarch64.rs`	Implements WHP debugging and cache synchronization.
`src/hyperlight_host/src/sandbox/initialized.rs`	Guards snapshots with active breakpoints.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

syntactically · 2026-10-12T15:50:39Z

@cshung I guess you are using software breakpoints here because WHPX aarch64 does not yet support setting MDCR_EL2.TDE. I think this is very divergent from the other implementations and especially the emulation of software step looks very fragile in its current state.

I wonder if we can make things a little bit more congruent with the other implementations by using the same hardware debug approach as other targets, but on WHP set MDSCR_EL1.KDE and have little assembly stubs catch this case and make hvcs (or, even make an hvc on every exception entry, when in gdb mode)? Admittedly this will not work very well if VBAR_EL1 or virtual memory configuration for the exception handlers are messed up, but in that case I think given the current guest architecture it is extremely unlikely that guest code would continue to execute and produce any useful results.

Adds GDB stub support for guests running on the ARM64 WHP backend. * Split the x86_64 debug code out of `hyperlight_vm/x86_64.rs` into a shared `hyperlight_vm/debug.rs`. * Rename `gdb/x86_64_target.rs` to `gdb/target.rs` and add per-architecture register handling in `gdb/arch/`. * Give `DebuggableVm` default methods so backends without hardware breakpoints or single step report `Unsupported`. * Implement `DebuggableVm` for the aarch64 WHP VM, using the batched register helpers. * Flush the host instruction cache after guest memory writes so software breakpoints take effect. * Document ARM64 debugging in `docs/how-to-debug-a-hyperlight-guest.md` and `CHANGELOG.md`. On Windows ARM64 (WHP): * `just build`, `just clippy`, `just guests`, `just test` and `just fmt-check` pass. --------- Signed-off-by: cshung <3410332+cshung@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

## Changes * Run the full disable path when the debugger detaches after a crash, so software breakpoints and pending steps are cleaned up. * Cancel pending software steps for any cancellation source, not only debugger interrupts. * Reject `map_region` requests that overlap the hidden cache sync page at GPA 0x1000, and add a test. * Restore the `≡` and `×` characters in `hyperlight_vm/x86_64.rs` comments. ## Verification On Windows ARM64 (WHP): * `just clippy`, `just fmt-check` and the `hypervisor::` and `map_region` lib tests with the `gdb` feature pass. --------- Signed-off-by: cshung <3410332+cshung@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Guest synchronous exception vectors forward debug stops through HVC. Hardware single-step supports branches, returns, and self-loops. Two hardware breakpoint slots and BRK software breakpoints are available. Interrupted register context and breakpoint configuration survive cache synchronization and vCPU reset. Asynchronous vectors retain their normal exception paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>

Poisoned sandbox restore discards software breakpoint state. Crash detach disables backend debugging even when breakpoint memory is unavailable. Snapshot checkpoints reject installed software breakpoints before replacing memory. The WHP cache-sync page permits guest reads and execution. Exception-vector tests use a separate writable stack. Cache flushing resolves layered snapshot memory through the guest memory view. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>

hyperlight-gh-bot · 2026-11-03T20:10:21Z

Benchmark Results

Measured commit: 00b72128ef72
Baseline commit: fa13116cd998

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`decode_vec_bytes_copy`	`encode_control`
	`373.34 µs` (➖ 1.00x slower)
`byte_chunks`		`749.91 ns` (➖ 1.05x faster)
`vec_bytes`		`585.31 ns` (➖ 1.01x slower)

payload_allocation

	`slot_pool_segmented`
`65536`	`149.07 ns` (➖ 1.05x slower)
`262144`	`793.44 ns` (➖ 1.51x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`13.16 ms` (➖ 1.02x slower)

slot_pool

	`alloc_dealloc_4096`	`alloc_dealloc_128`	`alloc_dealloc_1500`
	`8.00 ns` (➖ 1.04x slower)	`7.78 ns` (➖ 1.01x slower)	`7.72 ns` (➖ 1.00x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`101.21 µs` (➖ 1.02x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`65536`	`1.90 µs` (➖ 1.11x faster)	`2.09 µs` (➖ 1.10x slower)
`262144`	`7.00 µs` (➖ 1.04x faster)	`7.11 µs` (➖ 1.01x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`8192`	`1.04 µs` (➖ 1.03x slower)	`1.11 µs` (➖ 1.09x slower)
`65536`	`6.24 µs` (➖ 1.02x faster)	`6.18 µs` (➖ 1.05x faster)
`262144`	`26.73 µs` (➖ 1.02x faster)

kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`decode_vec_bytes_copy`	`encode_control`
	`697.96 µs` (➖ 1.07x faster)
`byte_chunks`		`751.58 ns` (➖ 1.06x faster)
`vec_bytes`		`561.48 ns` (➖ 1.07x faster)

payload_allocation

	`slot_pool_segmented`
`65536`	`162.18 ns` (➖ 1.05x faster)
`262144`	`516.89 ns` (➖ 1.04x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`15.20 ms` (➖ 1.06x slower)

slot_pool

	`alloc_dealloc_4096`	`alloc_dealloc_128`	`alloc_dealloc_1500`
	`7.28 ns` (➖ 1.07x slower)	`7.29 ns` (➖ 1.08x slower)	`7.31 ns` (➖ 1.08x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`54.20 µs` (➖ 1.01x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`65536`	`2.31 µs` (➖ 1.09x slower)	`2.25 µs` (➖ 1.02x slower)
`262144`	`8.02 µs` (➖ 1.07x slower)	`8.01 µs` (➖ 1.08x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`8192`	`754.90 ns` (➖ 1.07x slower)	`767.43 ns` (➖ 1.04x slower)
`65536`	`7.50 µs` (➖ 1.06x slower)	`7.58 µs` (➖ 1.07x slower)
`262144`	`31.56 µs` (➖ 1.09x slower)

mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`decode_vec_bytes_copy`	`encode_control`
	`268.99 µs` (➖ 1.33x faster)
`byte_chunks`		`938.74 ns` (➖ 1.01x slower)
`vec_bytes`		`716.56 ns` (➖ 1.01x faster)

payload_allocation

	`slot_pool_segmented`
`65536`	`193.08 ns` (➖ 1.34x faster)
`262144`	`727.21 ns` (➖ 1.00x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`61.40 ms` (➖ 1.03x faster)

slot_pool

	`alloc_dealloc_4096`	`alloc_dealloc_128`	`alloc_dealloc_1500`
	`9.68 ns` (➖ 1.03x faster)	`9.51 ns` (➖ 1.00x faster)	`9.73 ns` (➖ 1.08x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`105.05 µs` (➖ 1.02x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`65536`	`2.27 µs` (➖ 1.01x slower)	`2.44 µs` (➖ 1.00x faster)
`262144`	`10.05 µs` (➖ 1.03x slower)	`10.97 µs` (➖ 1.01x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`8192`	`1.26 µs` (➖ 1.05x slower)	`1.25 µs` (➖ 1.01x faster)
`65536`	`8.24 µs` (➖ 1.18x faster)	`8.41 µs` (➖ 1.15x faster)
`262144`	`36.34 µs` (➖ 1.06x faster)

mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`decode_vec_bytes_copy`	`encode_control`
	`670.34 µs` (➖ 1.10x faster)
`byte_chunks`		`929.33 ns` (➖ 1.02x slower)
`vec_bytes`		`650.73 ns` (➖ 1.02x faster)

payload_allocation

	`slot_pool_segmented`
`65536`	`255.04 ns` (➖ 1.03x faster)
`262144`	`663.75 ns` (➖ 1.12x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`77.96 ms` (➖ 1.07x slower)

slot_pool

	`alloc_dealloc_4096`	`alloc_dealloc_128`	`alloc_dealloc_1500`
	`8.68 ns` (➖ 1.02x faster)	`8.73 ns` (➖ 1.00x faster)	`8.80 ns` (➖ 1.01x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`65.48 µs` (➖ 1.06x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`65536`	`2.28 µs` (➖ 1.00x slower)	`2.27 µs` (➖ 1.00x faster)
`262144`	`7.93 µs` (➖ 1.02x faster)	`8.03 µs` (➖ 1.01x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`8192`	`893.31 ns` (➖ 1.02x faster)	`903.26 ns` (➖ 1.11x faster)
`65536`	`7.59 µs` (➖ 1.01x slower)	`7.48 µs` (➖ 1.02x faster)
`262144`	`41.12 µs` (➖ 1.09x slower)

hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`decode_vec_bytes_copy`	`encode_control`
	`1.87 ms` (➖ 1.00x faster)
`byte_chunks`		`1.17 µs` (➖ 1.02x faster)
`vec_bytes`		`783.63 ns` (➖ 1.00x faster)

payload_allocation

	`slot_pool_segmented`
`65536`	`311.81 ns` (➖ 1.35x slower)
`262144`	`787.38 ns` (➖ 1.00x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`10.77 ms` (➖ 1.08x faster)

slot_pool

	`alloc_dealloc_4096`	`alloc_dealloc_128`	`alloc_dealloc_1500`
	`10.11 ns` (➖ 1.02x faster)	`10.52 ns` (➖ 1.01x slower)	`10.51 ns` (➖ 1.04x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`959.38 µs` (➖ 1.16x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`65536`	`2.34 µs` (➖ 1.11x faster)	`2.35 µs` (➖ 1.01x slower)
`262144`	`9.12 µs` (➖ 1.11x faster)	`8.94 µs` (➖ 1.06x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`8192`	`1.31 µs` (➖ 1.00x faster)	`1.33 µs` (➖ 1.03x slower)
`65536`	`8.98 µs` (➖ 1.02x faster)	`9.03 µs` (➖ 1.02x slower)
`262144`	`40.11 µs` (➖ 1.08x faster)

hyperv-ws2025 / intel (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`decode_vec_bytes_copy`	`encode_control`
	`2.90 ms` (➖ 1.21x faster)
`byte_chunks`		`1.18 µs` (➖ 1.00x slower)
`vec_bytes`		`761.50 ns` (➖ 1.00x slower)

payload_allocation

	`slot_pool_segmented`
`65536`	`230.28 ns` (➖ 1.49x faster)
`262144`	`756.52 ns` (➖ 1.01x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`13.27 ms` (➖ 1.01x faster)

slot_pool

	`alloc_dealloc_4096`	`alloc_dealloc_128`	`alloc_dealloc_1500`
	`10.31 ns` (➖ 1.01x faster)	`9.88 ns` (➖ 1.02x faster)	`10.07 ns` (➖ 1.00x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`648.42 µs` (➖ 1.32x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`65536`	`2.46 µs` (➖ 1.05x slower)	`2.31 µs` (➖ 1.00x slower)
`262144`	`7.54 µs` (➖ 1.02x faster)	`7.64 µs` (➖ 1.01x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`8192`	`1.08 µs` (➖ 1.02x slower)	`1.16 µs` (➖ 1.09x slower)
`65536`	`9.43 µs` (➖ 1.11x slower)	`9.35 µs` (➖ 1.12x slower)
`262144`	`43.06 µs` (➖ 1.15x faster)

_{Reported by cargo ci bench-report --candidate run:41702936690 --config-file bench_report.toml.}

cshung added the kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. label Oct 8, 2026

This comment has been minimized.

Sign in to view

cshung marked this pull request as ready for review October 8, 2026 19:33

Copilot AI balanced review requested due to automatic review settings October 8, 2026 19:33

cshung requested review from andreiltd, danbugs, dblnz, devigned, jprendes, jsturtevant, ludfjig, simongdavies, squillace and syntactically as code owners October 8, 2026 19:33

Copilot started reviewing on behalf of cshung October 8, 2026 19:37

View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

cshung force-pushed the cshung/whp-aarch64-gdb branch from 8c0c245 to 39ea4cb Compare October 9, 2026 16:54

This comment has been minimized.

Sign in to view

cshung force-pushed the cshung/whp-aarch64-gdb branch from 7238539 to 6c35092 Compare October 19, 2026 14:56

This comment has been minimized.

Sign in to view

cshung force-pushed the cshung/whp-aarch64-gdb branch from dd86b49 to 698b851 Compare October 20, 2026 17:59

This comment has been minimized.

Sign in to view

cshung force-pushed the cshung/whp-aarch64-gdb branch from 698b851 to 3e3983f Compare October 26, 2026 16:30

This comment has been minimized.

Sign in to view

cshung force-pushed the cshung/whp-aarch64-gdb branch from 3e3983f to bed283f Compare October 30, 2026 16:47

This comment has been minimized.

Sign in to view

cshung and others added 2 commits November 2, 2026 08:09

cshung force-pushed the cshung/whp-aarch64-gdb branch from bed283f to 905ca05 Compare November 2, 2026 16:19

This comment has been minimized.

Sign in to view

cshung mentioned this pull request Nov 3, 2026

feat(aarch64): add GDB debugging for Linux KVM #2016

Draft

Conversation

cshung commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Recovery

Validation

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

syntactically commented Oct 12, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

hyperlight-gh-bot Bot commented Nov 3, 2026

Benchmark Results

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

cshung commented Oct 8, 2026 •

edited

Loading