Stream VecReduceBackend output into open chunks per tile - #828
Merged
Conversation
The backend staged every retire's output as rows in tile_rows, building chunks only at finish — instrumented at a 336MB high-water against a 2.6MB windowed presentation at 4M keys, the backend's whole memory spike. Emits arrive per tile in disjoint ascending key ranges, each consolidated, so emit can push rows straight into an open chunk and seal at TARGET; finish just wraps the sealed chunks. Also records why merged_run's cross-batch merge must stay: it is load-bearing for ordinal id sharing (netting in the id-keyed accumulations), measured at ~25% of churn when removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
VecReduceBackendstaged every retire's output as rows (tile_rows), building chunks only atfinish. Instrumented at 4M-key churn, that staging was the backend's entire memory spike: a 336MB high-water mark, while the windowed presentation — the part the windowing design bounds — peaked at 2.6MB. Atfinishthe staged rows and the built chunks were briefly co-resident besides.emit's calls arrive per tile in disjoint, ascending key ranges (the windows partition the key space in order), each consolidated. Soemitnow resolves its records, restores row order within each hash (vid order need not be row order), and pushes straight into the tile's openVecChunk, sealing atTARGET;finishjust wraps the sealed chunks in batches. The retire's output is never resident twice, and a chunk is immutable from the moment it seals — the shape that would let chunks spill. Bench times are neutral. This is a backend-local change: the tactic and its protocol are untouched.Also records in
merged_run's doc why its cross-batch merge must stay: the merge is load-bearing for the ordinal id scheme — grouping equal payloads across batches is what lets them share one id and net in the id-keyed accumulations — not for the record order. Presenting runs unmerged measured ~25% slower on churn overall.🤖 Generated with Claude Code