perf: reuse quantile summary buffers during merge

[peterxcli]

Which issue does this PR close?

Closes #4874.

Rationale for this change

The native QuantileSummaries port behind approx_percentile / percentile_approx allocates more than necessary on its hot paths:

• QuantileSummaries::merge takes &self / &other and allocates a fresh summary, so merge_batch reallocates on every incoming digest.
• with_head_buffer_inserted rebuilds the whole sampled vector on every flush.

What changes are included in this PR?

• Reuse scratch buffers in QuantileSummaries during flush, compression, and merge.
• Change summary merging to update in place while preserving the first-digest move fast path.
• Update memory accounting and add a regression test for buffer reuse.

How are these changes tested?

benchmark

• flamegraph:
• source:
  • baseline-nativ-bytes.html
  • patch-native-bytes.html

result:

• merge_batch: (15GB -> 11GB)
  • upstream:
  • pr:
• with_head_buffer_inserted: (10GB -> 1GB)
  • upstream:
  • pr:

[andygrove]

Thanks for this, the buffer reuse is a clean win and the before/after allocation flamegraphs make the impact easy to see. I traced the in-place merge against the old return-value version and the behavior looks preserved, including the non-commutative operand order and the memory accounting via heap_size. Two small suggestions, neither a blocker.

1. Defensive clear() for consistency

compress_immut defensively clears its output buffer before filling it, but with_head_buffer_inserted and merge take sampled_buffer and only reserve before pushing:

let mut new_samples = std::mem::take(&mut self.sampled_buffer);
new_samples.reserve(self.sampled.len() + sorted.len());

This is correct today because sampled_buffer is always empty on entry. It does leave a bit of a trap though. If that invariant ever slips in a future change, this path would silently append the stale elements (it pushes after reserve) and corrupt the result rather than fail loudly. A new_samples.clear() right after the take is essentially free and keeps this consistent with compress_immut. Same thought for merged_sampled in merge.

2. A multi-merge test

The new flush_and_merge_reuse_sampled_buffers test locks in the pointer swap, which is great. merge_is_within_bound only does a single merge though, and merge_batch folds many digests through the reused buffers in sequence. Would you consider adding a test that merges three or more summaries in place and checks the query result against the exact percentile? That would exercise the buffer reuse across repeated merges, which is exactly where a reuse bug would surface.

Review assisted by an LLM.

[peterxcli]

@andygrove Thanks for the review, addressed all of your suggestion! please take another look.

[andygrove]

Thanks for the revisions. Both points from last round are handled well. The defensive clear() calls are in place on all three paths now, and repeated_merges_are_within_bound folding three summaries is exactly the coverage I was hoping for.

I wanted to be confident the in-place merge did not perturb results, so I built a differential harness that fingerprints to_bytes() plus a 7-percentile query, and ran matching scenarios on main and on this branch. Sequential folds crossing the 50000 head-flush boundary, reversed merge order to hit the non-commutative interleave, and a duplicate-heavy case all produce identical bytes. That part looks solid.

One thing I would like to settle before merging. Since groups_accumulator_supported is false, grouped aggregation holds one QuantileSummaries per group through GroupsAccumulatorAdapter. Two allocations that main freed are now retained for the accumulator's lifetime. head_sampled keeps its capacity across flushes, where main dropped it at the mem::take. And sampled_buffer keeps the capacity from the last merge. The second one is amplified by res.reserve(current_samples.len()) in compress_immut, which sizes the output buffer to the uncompressed merged length even though the compressed result is much smaller. After a merge both sampled and sampled_buffer end up near the merged length rather than the compressed length.

heap_size() accounts for all of it, so the memory tracking stays truthful, which is good. My worry is peak footprint on a wide group-by, where doubling per-group bytes could push us into spilling and cost more than the allocation savings gain. Could you run CometAggregateExpressionBenchmark before and after and share wall-clock numbers, particularly approx_percentile_double_high_card alongside approx_percentile_double_median? The PR Benchmark Check job was skipped here, so we have allocation volume but no timing evidence yet.

If the high-cardinality case does regress, one option is to drop the reserve inside compress_immut and let the output grow amortized, since the compressed length is bounded by the compress threshold rather than by the merged input length. Would also be worth a test that asserts heap_size() stays bounded across many repeated merges, so the retention cannot quietly grow in a later change.

Review assisted by an LLM.

[peterxcli]

Hi @andygrove, thanks for pointing this out. After investigating, I plan to change the implementation to use a native GroupsAccumulator with a single QuantileSummaries scratch buffer shared across all groups, similar to HLLPlusPlusGroupsAccumulator.

For retained buffer memory after compression/merge, let:

• G = number of groups
• C = compressed summary capacity per group
• R = raw merge/flush workspace
• H = empty head-buffer capacity retained after flushing

The approximate memory shape is:

• main: G × C, but repeatedly allocates and frees temporary buffers
• Previous PR version: G × (C + R + H)
• Updated version: G × C + max(R)

Each group now retains only its compressed summary. The raw workspace is reused across all groups, and the head buffer is released after flushing.

For the high-cardinality benchmark—1,048,576 rows, 100,000 groups, and roughly 10 values per group—the buffer payload is approximately 56 MiB for the previous PR design versus 24 MiB for the shared-scratch design, around 2.3× lower. This is a capacity model rather than measured process RSS; the raw scratch component itself goes from roughly 100,000 buffers to one.

In my current local change, the seeded high-cardinality benchmark also improved from 558 ms on main and 523 ms on the previous PR version to 236 ms with shared scratch. The high-cardinality result was relatively stable, with a 19 ms standard deviation.
I also added coverage for grouped update/merge, nulls, filters, partial emission, array results, and scratch-buffer ownership.

Assisted by an LLM.