perf(vindex): optimize IVF index build reads - #800

Merged

JingsongLi merged 25 commits into

apache:mainfrom

jerry-024:codex/ivf-sparse-read-add-clean

Sep 21, 2026

Commits on Sep 20, 2026

feat: optimize IVF index build reads
jerry-024
committed
f2061fd
perf(vindex): complete IVF build read optimization
jerry-024
committed
3bc5ad5
fix(vindex): stratify IVF training sampling
jerry-024
committed
fad54e3
fix(vindex): preserve IVF sampling quality
jerry-024
committed
0ffc393
fix(vindex): harden sparse build pipeline
jerry-024
committed
af2bcd3
perf(vindex): parallelize sparse parquet reads
jerry-024
committed
999a530
perf(parquet): budget sparse reads by selected pages
jerry-024
committed
17db1f8
fix(vindex): guard sparse training reads
jerry-024
committed
e7bb7b7
fix(vindex): require meaningful sparse savings and cover safety regressions
jerry-024
committed
000a1c1
refactor(vindex): keep sparse build changes focused
jerry-024
committed
8325dc9
fix(vindex): scope sparse savings to current shard
jerry-024
committed
c866b82
feat(vindex): build IVF indexes with granule pipeline
jerry-024
committed
8ec4e02
fix(vindex): simplify granule build pipeline
jerry-024
committed
43022ec
test(vindex): use search result row ids
jerry-024
committed
07758fb
fix: handle overlapping vindex granule providers
jerry-024
committed
799df50
fix: report granule spill metrics
jerry-024
committed
5fd19b7
feat(vindex): add granule build toggle
jerry-024
committed
fc46acc
fix(vindex): jitter granule training samples
jerry-024
committed
407e396
fix(vindex): adapt granule reads to read budget API
jerry-024
committed
1a12958
fix(vindex): report actual trainer sample counts
jerry-024
committed
a2ab2f8
fix(vindex): sample granule training in logical row space
jerry-024
committed
2125a30
refactor(vindex): remove redundant granule planning work
jerry-024
committed
de03ff2
Merge upstream/main and resolve PR 800 test conflicts
jerry-024
committed
ae42c75
fix(vindex): honor Parquet page index option in granule planning
jerry-024
committed
d1f86cd
fix(vindex): isolate channel workers from blocking pool
jerry-024
committed
8af080f

Navigation Menu

perf(vindex): optimize IVF index build reads - #800

perf(vindex): optimize IVF index build reads#800
JingsongLi merged 25 commits into
apache:mainfrom
jerry-024:codex/ivf-sparse-read-add-clean

Commits on Sep 20, 2026

feat: optimize IVF index build reads

perf(vindex): complete IVF build read optimization

fix(vindex): stratify IVF training sampling

fix(vindex): preserve IVF sampling quality

fix(vindex): harden sparse build pipeline

perf(vindex): parallelize sparse parquet reads

perf(parquet): budget sparse reads by selected pages

fix(vindex): guard sparse training reads

fix(vindex): require meaningful sparse savings and cover safety regressions

refactor(vindex): keep sparse build changes focused

fix(vindex): scope sparse savings to current shard

feat(vindex): build IVF indexes with granule pipeline

fix(vindex): simplify granule build pipeline

test(vindex): use search result row ids

fix: handle overlapping vindex granule providers

fix: report granule spill metrics

feat(vindex): add granule build toggle

fix(vindex): jitter granule training samples

fix(vindex): adapt granule reads to read budget API

fix(vindex): report actual trainer sample counts

fix(vindex): sample granule training in logical row space

refactor(vindex): remove redundant granule planning work

Merge upstream/main and resolve PR 800 test conflicts

fix(vindex): honor Parquet page index option in granule planning

fix(vindex): isolate channel workers from blocking pool