diff --git a/README.md b/README.md index 0cde03e..565c5b8 100644 --- a/README.md +++ b/README.md @@ -115,11 +115,15 @@ this one carries scale.) ### Filtered search: selectivity-adaptive A sparse allow-bitmap makes graph traversal the wrong algorithm (too few -eligible nodes ever fill the beam), so when `popcount(allow) ≤ 16·ef` the -search switches to a zero-allocation **exact scan of the allowed set** — -cheaper than the degraded walk *and* recall-1.0 by construction. Measured at -100k vectors, ef = 100, k = 10, against exact filtered ground truth (arm64 -CI runner, single thread): +eligible nodes ever fill the beam), so while +`popcount(allow) ≤ max(16·ef, capacity/8)` the search switches to a +zero-allocation **exact scan of the allowed set** — cheaper than the +degraded walk *and* recall-1.0 by construction. The crossover was measured, +not guessed: scanning wins up to ~15–20% selectivity (at 2%, the scan is +88 µs/1.000 recall vs 6,654 µs/0.989 for forced traversal), so the default +switches at 12.5% of capacity; `set_filter_scan_limit()` overrides it. +Measured at 100k vectors, ef = 100, k = 10, against exact filtered ground +truth (arm64 CI runner, single thread): | selectivity | allowed ids | recall@10 | mean latency | QPS (1 thread) | |---|---|---|---|---| @@ -443,10 +447,13 @@ v0.8. Known limitations, in priority order: invalidates outstanding `SearchContext`s (their searches safely return 0 until recreated). Growing the vector block itself is the caller's job — an mmap'ed block means writing and remapping a larger file -2. **Mid-selectivity filters (roughly 5–50%) still pay a traversal penalty** - — sparse filters get the exact scan and dense filters barely notice, but - the band in between runs the routed walk with a partially-filled beam - (0.998 recall at 10%, at ~10x the unfiltered latency) +2. **Filters between ~12.5% and ~50% selectivity pay a traversal penalty** — + below the measured crossover the exact scan is both faster and perfect; + above ~50% the beam barely notices; the band between runs the routed walk + with a partially-filled beam (recall ~0.94–0.98 at a few-fold latency + cost). An ACORN-style two-hop expansion was evaluated and rejected: it + targets recall (already ≥0.94 here) at the price of latency (the actual + cost), so it does not earn its complexity 3. **Parallel builds are nondeterministic** in link structure (insertion order interleaves); use serial `insert()` or `insert_batch(..., 1)` when bit-reproducible graphs matter diff --git a/include/edgevector/hnsw_graph.hpp b/include/edgevector/hnsw_graph.hpp index bd85006..8139c0d 100644 --- a/include/edgevector/hnsw_graph.hpp +++ b/include/edgevector/hnsw_graph.hpp @@ -49,8 +49,11 @@ // small relative to the beam (popcount(allow) <= 16 * ef; the popcount is a // ~microsecond pass over the bitmap), the search switches to a direct scan // of the allowed ids, which is both cheaper AND exact - recall 1.0 over the -// allowed set by construction. Larger filters keep the graph traversal, with -// filtered-out nodes still routing. Both paths are zero-allocation. +// allowed set by construction. The measured default policy scans while +// popcount(allow) <= max(16 * ef, capacity / 8) - see +// set_filter_scan_limit() to override. Larger filters keep the graph +// traversal, with filtered-out nodes still routing. Both paths are +// zero-allocation. // // DELETION AND SLOT RECLAMATION // ----------------------------- @@ -559,6 +562,20 @@ class HNSWGraph { return ((deleted_[id >> 6u] >> (id & 63u)) & 1ull) != 0ull; } + // Filtered-search strategy override: filtered queries take the exact + // allowed-set scan while popcount(allow) <= limit, the routed graph + // traversal above it. Pass kFilterScanAuto (the default) for the + // measured policy max(16*ef, capacity/8); 0 forces traversal always; + // UINT64_MAX forces the scan always. Not thread-safe against concurrent + // queries; set it at configuration time. + static constexpr std::uint64_t kFilterScanAuto = ~0ull - 1u; + void set_filter_scan_limit(std::uint64_t limit) noexcept { + filter_scan_limit_ = limit; + } + std::uint64_t filter_scan_limit() const noexcept { + return filter_scan_limit_; + } + // Slot reclamation: relinks a tombstoned id around the NEW vector bytes // the caller has already written at index `id` of the external block. // Workflow: remove(id) -> overwrite the bytes -> reinsert(id). Returns @@ -1370,13 +1387,27 @@ class HNSWGraph { // The candidate-pool collector behind all three search modes: picks the // scan for sparse filters (exact and cheaper), the beam otherwise. + // Crossover measured empirically (see the benchmark's filtered table): + // the scan's cost is ~one Hamming distance per allowed id, while a + // filter-throttled traversal costs several times the unfiltered walk, so + // scanning wins until the allowed set is a sizeable fraction of the + // whole index. Default policy: scan while allowed <= max(16*ef, + // capacity/8); override with set_filter_scan_limit(). std::uint32_t collect_pool(SearchContext& ctx, const std::uint8_t* q, std::uint32_t ef, const std::uint64_t* allow) const noexcept { - if (allow != nullptr && - popcount_allow(allow) <= - static_cast(ef) * 16u) { - return filtered_scan(ctx, q, ef, allow); + if (allow != nullptr) { + std::uint64_t limit = filter_scan_limit_; + if (limit == kFilterScanAuto) { + const std::uint64_t ef_floor = + static_cast(ef) * 16u; + const std::uint64_t cap_share = + static_cast(capacity_) / 8u; + limit = (ef_floor > cap_share) ? ef_floor : cap_share; + } + if (popcount_allow(allow) <= limit) { + return filtered_scan(ctx, q, ef, allow); + } } return beam_layer0(ctx, q, ef, allow); } @@ -1694,6 +1725,7 @@ class HNSWGraph { std::uint32_t ef_construction_; std::uint32_t ef_limit_ = 0u; double ml_ = 0.0; + std::uint64_t filter_scan_limit_ = kFilterScanAuto; std::mt19937_64 rng_; // --- graph state --------------------------------------------------------