Repository navigation
Conversation
A copy-on-write restore maps the snapshot file MAP_PRIVATE over guest RAM, and marked that mapping MADV_HUGEPAGE when thp is on. A private copy of a file page is always 4 KiB, so the hint buys no huge pages, but it makes each fault on the mapping read 2 MiB around the faulting page, including the faults the restore path itself takes. Measured with 16 copy-on-write clones of a 2 GiB guest, comparing thp=on with thp=off, which differ only in this hint for such a VM: - a restore on an idle host took p50 5.8 ms / p99 6.4 ms instead of 9.9 / 11.3 ms; - a restore while other clones fault in their memory took p50 9.4 ms / p99 14.3 ms instead of 49.9 / 97.6 ms; - reads from the snapshot memory files dropped by about 26%, at twice the number of major faults. Drop the hint from the overlay. The reserve policy is still applied. Signed-off-by: CMGS <ilskdw@gmail.com>
Signed-off-by: CMGS <ilskdw@gmail.com>
memory.md says guest memory is labelled MADV_HUGEPAGE when thp is on, which no longer holds for memory restored with memory_restore_mode=copyonwrite. Signed-off-by: CMGS <ilskdw@gmail.com>
Distinguish the private anonymous copies a guest write creates from the read-only PMD-sized file folios that MADV_HUGEPAGE allows, record what dropping the hint costs and saves as measured, and qualify the base page size and the readahead amount. Signed-off-by: CMGS <ilskdw@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Drops MADV_HUGEPAGE from the copy-on-write restore overlay (
do_mmap_cow_saved_regions).A page the guest writes becomes a private anonymous copy of one host base page (4 KiB on x86_64) either way. Read faults on a large-folio filesystem can map PMD-sized file folios read-only, but a read fault that misses the page cache can then request readahead of two PMD-sized folios (about 4 MiB), including the faults taken by the restore path itself. For a CoW-restored VM,
thp=onandthp=offdiffer only in this hint: the anonymous region's MADV_HUGEPAGE is replaced along with its VMA by the MAP_FIXED overlay.Evidence (test2, 16 CoW clones of a 2 GiB guest,
thp=onvsthp=offon the current dev build):Separate from #7.
Read-mostly guard (test2, kernel 6.12; prod runs 7.0). 16 CoW clones of a 2 GiB guest under a light workload, dev 750aa2b vs a build with #7 and this PR, benchmarks in the first and the last clone:
So on a large-folio filesystem the hint does map 2 MiB file folios, and dropping it costs about 3.5% of random-read throughput in a read-mostly guest, while streaming reads and CPU are unchanged and restore gets 1.5x faster idle and about 5x faster with 16 clones restoring.
Gates (node1): fmt, typos, 4 clippy variants, fuzz check, vmm memory_manager 11 tests, virtio-devices 168 tests, release build on the code commit; typos on the two docs commits.
Squash message: