Repository navigation
live move: every chunk is SHA-256 hashed on both ends; measure its share of the final round on a large guest #463
Description
A live move hashes every chunk twice: the source hashes each 16 MiB chunk with SHA-256 while it scans the round (store.ScanFile, carried in the tar header by store/peer/transport.go), and the target hashes it again while writing (store.WriteChunk verifies the sum).
Measured
Test host, x86, 2 pinned cores, ABBA-interleaved, 4 rounds, spread under 2%:
| step | throughput | per GiB |
|---|---|---|
| SHA-256 of a 16 MiB chunk (source side) | 1.82 GB/s | 0.59 s |
target write with verify (WriteChunk) |
1.46-1.50 GB/s | 0.72 s |
| target write without verify (same file, page cache) | 8.4-8.7 GB/s | 0.12 s |
Why it is not changed now
- Not the bottleneck today. The moves measured on the testbed ran at about 45 MB/s (a 512 MiB baseline round took 11.5 s), so hashing is roughly 2-3% of a round.
- The final round is the open question. Its capture, transfer and receive sit inside the guest's pause. Two hash passes over a 100 MiB final delta add about 0.11 s, 5% of the default 2 s
max_downtime_ms. Idle 512 MiB guests had final deltas small enough for 180-330 ms downtime, but no large or memory-writing guest has been moved yet. - The target's verify is the transfer's integrity check and should stay.
What decides it
The boundary-scale move test that live move still needs before production use: an 8-16 GiB guest under a memory-writing load, with the final round's time split into capture, hash, wire, receive and start. Act only if hashing shows up as a real part of the downtime. Options then would be overlapping the hash with the send, or a cheaper source-side sum for the final round only.
Activity
CMGS commentedon Nov 10, 2026
Related change in #553 (v0.1.31): a move answer now reports
capture_ms,send_msandcommit_msnext todowntime_ms, and a paused final round that is estimated to overrunmax_downtime_msaborts and resumes on the source (503ErrMoveDowntime). The source-side SHA-256 is insidecapture_ms/send_msand is not reported on its own, so the boundary-scale move test still needs a hash split (or a profile) to decide this issue. That test (8–16 GiB guest under a memory-writing load) has not run yet. The production moves that motivated #553 lost their final-round time to the starved capture on a loaded source, not to hashing.