From ba497806d1c0d28a5ba112f47ec372f93c70b2cd Mon Sep 17 00:00:00 2001 From: matrixarkai Date: Fri, 13 Nov 2026 07:17:56 -0800 Subject: [PATCH] A page grows with headroom, so an append stops copying the page A page payload asked the allocator for exactly the bytes each append added. Every append could then reallocate the buffer and, once the allocator could not extend it in place, copy the whole page: 7% of a profile of 1 KiB puts and a third of one of 256 KiB puts was that copy. The payload now grows with headroom of a sixteenth of its length, via reserve_page_growth. Growth is geometric, so a page reallocates about once per 6.25% it grows instead of once per append, and the spare room is bounded at 6.25% of the page. A write that grows the page by more than that asks for exactly what it needs, so a page filled by large objects carries no spare room. Doubling was the alternative and is why the exact reserve existed: it left a page holding up to 1.9x what it stores. Measured with callgrind on the journal door at three replicas: 158,343 -> 152,246 instructions per put at 1 KiB, and 25.32M -> 22.32M total over 120 puts of 256 KiB. The cost is memory, and it is stated rather than hidden: a page built by small appends now holds up to 6.25% above the floor. The published data term for 4 KiB objects reads 3.05x instead of 3.00x and its total 3.14x instead of 3.08x; 1 KiB objects read 3.34x instead of 3.32x; the 64 KiB row is unchanged. The README and docs/footprint.md tables, the striped-versus-replicated comparison, and the three tests that pinned the floor exactly now assert the floor plus the headroom band. Co-Authored-By: Claude Sonnet 5.5 --- README.md | 22 +++--- docs/footprint.md | 13 ++-- src/object_store_regression_tests.rs | 92 ++++++++++++++++++++------ src/services/chunk/chunk_writes.inc.rs | 29 ++++++-- 4 files changed, 119 insertions(+), 37 deletions(-) diff --git a/README.md b/README.md index 8957ab15..19ec3961 100644 --- a/README.md +++ b/README.md @@ -452,7 +452,8 @@ comment claiming they do. - **Erasure coding.** A page with no peers keeps a redundancy fragment, so a partial corruption is still recoverable from the single copy it has. The fragment is local to its page and is empty whenever the page has peers, which - is why the data term measures exactly 3.00x at three replicas. + is why a page written in large pieces measures exactly 3.00x at three replicas, with no + fragment on top. - **Striped payload, if you ask for it.** `stripe_redundancy_units` spreads a page across its replicas as fixed-size units instead of copying it to each, so the store holds `replica_count / (replica_count - stripe_redundancy_units)` of @@ -483,12 +484,16 @@ At three replicas with the metadata counted, measured at 26-character keys: | object size | objects | amplification | data-only figure | |---:|---:|---:|---:| | 64 KiB | 1,000 | **3.01x** | 3.00x | -| 4 KiB | 4,000 | **3.08x** | 3.00x | -| 1 KiB | 8,000 | **3.32x** | 3.02x | +| 4 KiB | 4,000 | **3.14x** | 3.05x | +| 1 KiB | 8,000 | **3.34x** | 3.02x | -The data term is exactly **3.00x** -- the replication floor for a store that -copies every page to every replica -- so everything above it is metadata, which -is **290 bytes an object plus 164,232 fixed** whatever the object size. +The data term has a floor of **3.00x** -- the replication floor for a store that +copies every page to every replica -- and reads exactly that when a page is written in +large pieces (the 64 KiB row). A page grown by many small appends also holds up to a +sixteenth of its length in spare room, so the allocator is asked to grow it about once per +6.25% rather than once per append (`reserve_page_growth`); that is the 0.05x on the 4 KiB +row. What sits above the data term is metadata, which is **290 bytes an object plus 164,232 +fixed** whatever the object size. **What that 3.00x buys depends on where the replicas are, and the figure cannot tell you.** Three copies measure 3.00x whether they are three `BTreeMap`s on one heap or three copies in @@ -529,7 +534,8 @@ Those two numbers are a line, fitted from measurements at three object counts by if this file disagrees with the tree. A store that stripes moves the data term itself, to **1.5x** at three replicas and -one unit of redundancy -- which is the least any layout can hold and still survive a +one unit of redundancy (1.53x for a page built by 1 KiB appends, which carries its +growth headroom) -- which is the least any layout can hold and still survive a loss. Every replica holds one unit of each stripe group, so keeping `p` units of redundancy over `R` replicas costs `R / (R - p)`, which is smallest at one unit; with no redundancy nothing is recoverable, and the only way further down is more replicas, @@ -538,7 +544,7 @@ which is more hardware rather than less space. replicas to eight and fails if any of them undercuts that. Counted with the metadata, a thousand objects of 1 KiB measure -**1.97x striped against 3.46x replicated**, both read out of `footprint_audit()` on a +**2.00x striped against 3.54x replicated**, both read out of `footprint_audit()` on a store nothing has read back yet. What separates the total from the 1.5x data term is the metadata line above, which striping does not change: the same 290 bytes an object either way. Striping moves the payload and nothing else. diff --git a/docs/footprint.md b/docs/footprint.md index 48d4b5d0..de14b9f8 100644 --- a/docs/footprint.md +++ b/docs/footprint.md @@ -21,8 +21,13 @@ Measured at three replicas, 26-character keys, with the metadata counted: | object size | objects | amplification | data-only figure | |---:|---:|---:|---:| | 64 KiB | 1,000 | **3.01x** | 3.00x | -| 4 KiB | 4,000 | **3.08x** | 3.00x | -| 1 KiB | 8,000 | **3.32x** | 3.02x | +| 4 KiB | 4,000 | **3.14x** | 3.05x | +| 1 KiB | 8,000 | **3.34x** | 3.02x | + +A page written in large pieces holds exactly the floor (the 64 KiB row). A page grown by many +small appends also holds up to a sixteenth of its length in spare room, because the allocator is +asked to grow it about once per 6.25% instead of once per append -- that is the 0.05x above the +floor on the 4 KiB row. Reproduce any row with the measurement harness: @@ -33,8 +38,8 @@ MEASURE_OBJECTS=8000 MEASURE_SIZE=1024 cargo test --release --lib \ **Lowering `replica_count` does not reach 1.0x.** A page with no peers keeps a redundancy fragment so that a partial corruption is still recoverable from the -single copy it has, and a page with peers does not — which is why the data term -above is exactly 3.00 rather than something above it. Measured at one replica: +single copy it has, and a page with peers does not — which is why a page written in +large pieces is exactly 3.00 rather than something above it. Measured at one replica: | bytes in the page | fragment | split | overhead | |---:|---:|---:|---:| diff --git a/src/object_store_regression_tests.rs b/src/object_store_regression_tests.rs index 462cf280..b4d607b9 100644 --- a/src/object_store_regression_tests.rs +++ b/src/object_store_regression_tests.rs @@ -3048,6 +3048,50 @@ fn object_inventory_pages_metadata_and_placement() { assert_eq!(budgeted.next_marker.as_deref(), Some("logs/a")); } +#[test] +fn a_page_filled_by_small_appends_reallocates_rarely_and_holds_bounded_slack() { + // A page used to ask the allocator for exactly the bytes each append added, so every append + // could copy the whole page. The growth is geometric now, with headroom of a sixteenth of what + // the page holds. Two claims, because each alone is easy to satisfy: appends must not + // reallocate per write (doubling would pass that), and the spare room must stay bounded + // (reserving exactly would pass that). + const APPENDS: usize = 900; + const WIDTH: usize = 1024; + let mut chunk = MatrixReplicaService::new(); + let page_id = PageId { + chunk_id: ChunkId::new(0, 992).unwrap(), + page_index: 0, + }; + let mut expected: Vec = Vec::new(); + let mut capacities = std::collections::BTreeSet::new(); + for append in 0..APPENDS { + let bytes: Vec = (0..WIDTH).map(|i| (append * 31 + i) as u8).collect(); + chunk + .write_chunk(page_id.clone(), expected.len() as u64, &bytes) + .unwrap(); + expected.extend_from_slice(&bytes); + let page = chunk.pages_local().get(&page_id).unwrap(); + capacities.insert(page.data.capacity()); + } + let page = chunk.pages_local().get(&page_id).unwrap(); + assert_eq!(page.data.as_slice(), expected.as_slice(), "the bytes are unchanged"); + + let len = page.data.len(); + let capacity = page.data.capacity(); + assert!( + capacity <= len + len / 16 + WIDTH, + "spare room must stay near a sixteenth of the page: capacity {capacity} for {len} bytes" + ); + // One distinct capacity per reallocation. Growing to 900 KiB by a sixteenth each time is about + // eighty steps; the exact-reserve policy this replaced took one per append. + println!("PAGE_GROWTH appends={APPENDS} reallocations={} slack_bytes={}", capacities.len(), capacity - len); + assert!( + capacities.len() <= APPENDS / 8, + "{} reallocations for {APPENDS} appends: growth is not geometric", + capacities.len() + ); +} + #[test] fn a_chunk_checksum_is_the_hash_of_its_payload_however_the_write_arrived() { // `write_chunk` resumes the FNV-1a fold over the appended bytes instead of re-hashing the @@ -43966,21 +44010,27 @@ fn a_store_holds_one_and_a_half_copies_of_its_payload_when_it_stripes() { striped_logical, plain_logical, "both stores must have been given the same logical bytes for the figures to compare" ); - assert_eq!( - plain_held * 1000 / plain_logical, - 3000, - "three-way replication holds three copies, and this is the figure the payload floor \ - has always reported" + // THE FLOOR, PLUS THE HEADROOM A GROWING PAGE CARRIES. Three copies is the floor and the + // figure never goes below it; a page built by small appends also holds up to a sixteenth of + // its length in spare room (`reserve_page_growth`), so the figure sits in a band above it. + // Pinned as that band rather than as the floor, because the floor alone is no longer what a + // store built by 1 KiB appends measures. + let plain_permille = plain_held * 1000 / plain_logical; + let striped_permille = striped_held * 1000 / striped_logical; + assert!( + (3000..=3188).contains(&plain_permille), + "three-way replication holds three copies plus at most a sixteenth of headroom: \ + {plain_permille}" ); - assert_eq!( - striped_held * 1000 / striped_logical, - 1500, - "a striped store should hold one and a half" + assert!( + (1500..=1594).contains(&striped_permille), + "a striped store should hold one and a half copies plus at most a sixteenth of \ + headroom: {striped_permille}" ); - assert_eq!( - striped_held * 2, - plain_held, - "exactly half, asserted as a relation so it cannot drift into a near miss: \ + // Half, within the headroom. Asserted as a relation so it cannot drift into a near miss. + assert!( + (striped_held * 2).abs_diff(plain_held) * 1000 / plain_held <= 63, + "striping should hold half what replication does, give or take headroom: \ {striped_held} against {plain_held}" ); } @@ -44034,10 +44084,11 @@ fn a_striped_store_is_still_striping_after_a_snapshot_round_trip() { .get(key) .unwrap_or_else(|| panic!("the audit should report {key}")) }; - assert_eq!( - get("replica.data.capacity") * 1000 / get("logical.written"), - 1500, - "a restored striped store should still hold one and a half copies" + let restored_permille = get("replica.data.capacity") * 1000 / get("logical.written"); + assert!( + (1500..=1594).contains(&restored_permille), + "a restored striped store should still hold one and a half copies plus at most a \ + sixteenth of headroom: {restored_permille}" ); } @@ -47632,9 +47683,10 @@ fn the_published_striped_amplification_is_the_one_this_tree_produces() { // The data terms, so a total that happened to match cannot hide a layout that did not // take: 1.5x against 3.0x is the layout doing its job, and 1,504 rather than 1,500 is // the last page of 8,000 objects not filling a whole stripe group. - assert_eq!( - replicated_data, 3000, - "three-way replication holds three copies" + assert!( + (3000..=3188).contains(&replicated_data), + "three-way replication holds three copies plus at most a sixteenth of headroom: \ + {replicated_data}" ); assert!( (1500..=1560).contains(&striped_data), diff --git a/src/services/chunk/chunk_writes.inc.rs b/src/services/chunk/chunk_writes.inc.rs index 2b2e07e5..9011a23f 100644 --- a/src/services/chunk/chunk_writes.inc.rs +++ b/src/services/chunk/chunk_writes.inc.rs @@ -1,5 +1,27 @@ // SPDX-License-Identifier: Apache-2.0 // Copyright 2026 MatrixArkAI +/// Room for a page payload to grow to `needed_len`, with headroom proportional to what it holds. +/// +/// A page filled by many small appends used to ask for EXACTLY the bytes each append added, so +/// the allocator was asked to grow the buffer on every write and, once it could not extend it in +/// place, copied the whole page: 33% of a 256 KiB-put profile was that copy, and 7% of a 1 KiB +/// one. Doubling would amortise it but a page then holds up to ~1.9x the bytes it stores +/// (measured 163 MB of capacity for 87 MB of data at three replicas). +/// +/// A sixteenth of the current length sits between the two. Growth is geometric, so a page +/// reallocates about once per 6.25% it grows rather than once per append, and the spare room is +/// bounded at 6.25% of the page -- a figure that does not depend on how many writes built it. +/// A write that grows the page by more than that asks for exactly what it needs, so a page +/// filled by large objects carries no spare room at all. +pub(crate) fn reserve_page_growth(payload: &mut Vec, needed_len: usize) { + if needed_len <= payload.capacity() { + return; + } + let len = payload.len(); + let target = needed_len.max(len.saturating_add(len / 16)); + payload.reserve_exact(target.saturating_sub(len)); +} + impl MatrixReplicaService { pub fn write_chunk( &mut self, @@ -47,10 +69,7 @@ impl MatrixReplicaService { let checksum_before_write = chunk.checksum_fnv1a64; if end_index > chunk.data.len() { let payload = std::sync::Arc::make_mut(&mut chunk.data); - // reserve_exact, not resize's doubling: a chunk filled by many small - // writes otherwise holds ~1.9x the bytes it stores (measured 163 MB - // of capacity for 87 MB of data at three replicas). - payload.reserve_exact(end_index.saturating_sub(payload.len())); + reserve_page_growth(payload, end_index); payload.resize(end_index, 0); } std::sync::Arc::make_mut(&mut chunk.data)[start_index..end_index].copy_from_slice(data); @@ -345,7 +364,7 @@ impl MatrixReplicaService { })?; if projected_len_index > chunk.data.len() { let payload = std::sync::Arc::make_mut(&mut chunk.data); - payload.reserve_exact(projected_len_index.saturating_sub(payload.len())); + reserve_page_growth(payload, projected_len_index); payload.resize(projected_len_index, 0); } for (offset, data) in coalesced {