Skip to content

Disk-resident indexes drop selected docs when the ID selector is smaller than k (no exact-match path) #49

Description

@chishui

What is the bug?

detail::should_run_exact_match is called only from nsparse/seismic_index.cpp:171 and
seismic_scalar_quantized_index.cpp:218. No disk_seismic* file calls it, so an enumerable selector
of size <= k takes the block-budget path and most selected docs are never candidates.

How can one reproduce the bug?

2000 docs, dim 1024, 30 nnz/doc, k=10, 20 queries,
8bit|vmin=0|vmax=1|lambda=300|beta=200|alpha=0.4, selector = the query's exact top-n.
Members returned, memory and mmap alike:

selector seismic_sq disk_seismic_sq
1 20/20 13/20
10 200/200 113/200

test_disk_seismic_sq_index.py builds lambda=25|beta=4, where disk returns 5/5 — hence no coverage.
disk_seismic_contract.py:150 asserts only that non-members are excluded.

What is the expected behavior?

Selector <= k: return every member, as seismic_sq does. Dispatch to detail::ExactMatcher.

Do you have any additional context?

In OpenSearch neural-search a per_block field filtered to 5 docs with k=10 returns 0 hits.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions