Skip to content

feat: Vamana two_pass_build and query prefetch (#220) - #234

Merged
s2x merged 3 commits into
mainfrom
feat/220-vamana-two-pass-prefetch
Sep 29, 2026
Merged

s2x merged 3 commits into
mainfrom
feat/220-vamana-two-pass-prefetch

Conversation

@s2x

@s2x s2x commented Sep 29, 2026

Copy link
Copy Markdown
Member

Closes #220.

Summary

Two upstream Vamana options that were not exposed. HNSW prefetch already existed from #181, so this follows the same shape.

ZVecIndexParams::forVamana(..., bool $twoPassBuild = false)
ZVecVectorQuery::setVamanaPrefetch(int $prefetchOffset, int $prefetchLines): self

two_pass_build runs a second full-graph Vamana construction pass — better graph quality, slower build. prefetch_offset / prefetch_lines tune software prefetch during the graph search; offset 0 disables it, lines 0 means "derive from vector size".

Part A — two_pass_build

Trailing and optional on forVamana(), so existing positional calls are unaffected. Upstream and Python both default to false.

It is a separate FFI function rather than an extra argument to zvec_index_params_set_vamana(). That function's signature is part of the ABI other code may have been compiled against; extending it in place would break those callers, a new function breaks nothing. The Vamana constructor is now called with its full argument list, passing an empty QuantizerParam — safe, because the rotate branch in build() still overwrites it when setQuantizerEnableRotate() was used.

Part B — Vamana prefetch

Both prefetch setters share one stored field pair. VectorQueryHolder already keeps prefetch_offset_ / prefetch_lines_, so I reused those rather than adding Vamana-specific fields, and extended merge_stored_query_settings() to apply them to VamanaQueryParams as well as HnswQueryParams. That makes call order irrelevant — setVamanaPrefetch() then setVamanaParams() works, and so does the reverse — which is the same guarantee #197/#181 gave HNSW. When prefetch was never set, the stored values are the upstream defaults, so re-applying them is a no-op.

zvec_vector_query_set_vamana_prefetch() creates VamanaQueryParams when no params exist yet, not HnswQueryParams: upstream rejects params whose type does not match the field's index type, and the prefetch-only case in the test is what catches that.

The PHP setter validates and throws ZVecException; the C setter returns void so there is no status to check, and clamps negatives to 0 as a second safety net.

Known limitation, documented not fixed

Prefetch is honoured only by queryVector(). The legacy query() path sends just queryParamType, ef/nprobe, radius, isLinear and isUsingRefiner through zvec_collection_query_ex, so prefetch is silently dropped there. HNSW prefetch has the same limitation today. Fixing it needs new arguments on that function and applies to both index types, so it belongs in its own issue. Noted in the README.

Verification

Index params cannot be read back from an existing index, so test_vamana_two_pass_build.phpt asserts on upstream's own CollectionSchema::to_string(), which embeds ,two_pass_build:true per field:

schema has two_pass_build:true: yes
query: doc1,doc2,doc3
default schema has two_pass_build:false: yes
positional call ok

test_vamana_prefetch.phpt covers both call orders, prefetch with no setVamanaParams(), 0/0, and negative rejection.

Full suite:

Number of tests : 193               193
Tests skipped   :   0 (  0.0%)
Tests failed    :   0 (  0.0%)
Expected fail   :   2 (  1.0%)
Tests passed    : 191 ( 99.0%)

test_hnsw_prefetch.phpt, test_vamana_index.phpt, test_vector_query_vamana_params.phpt and test_docs_consistency.phpt pass unchanged. test_dbs/ is left with only .gitignore.

🤖 Generated with Claude Code

s2x added 3 commits September 29, 2026 08:18
Two upstream Vamana options that zvec-php did not expose. HNSW prefetch
already existed from #181, and this follows the same shape.

Part A -- two_pass_build, an index param added in v0.7.0 that runs a
second full-graph Vamana construction pass: better graph quality, slower
build. forVamana() takes it as a trailing optional argument, so existing
positional calls are unaffected.

It is a separate FFI function rather than an extra argument to
zvec_index_params_set_vamana(). That function's signature is part of the
ABI other code may have been compiled against, and extending it in place
would break those callers; a new function breaks nothing. The Vamana
constructor is called with its full argument list, passing an empty
QuantizerParam -- safe, because the rotate branch in build() still
overwrites it when setQuantizerEnableRotate() was used.

Part B -- prefetch_offset / prefetch_lines, query params from v0.5.1 that
tune software prefetch during the Vamana graph search.

Both prefetch setters now share the single stored prefetch_offset_ /
prefetch_lines_ pair already on VectorQueryHolder, rather than adding
Vamana-specific fields, and merge_stored_query_settings() applies them to
VamanaQueryParams as well as HnswQueryParams. That makes the call order
irrelevant -- setVamanaPrefetch() then setVamanaParams() works, and so
does the reverse -- which is the same guarantee #197 and #181 gave HNSW.
When prefetch was never set, the stored values are the upstream defaults,
so re-applying them does nothing.

zvec_vector_query_set_vamana_prefetch() creates VamanaQueryParams when no
params exist yet, not HnswQueryParams: upstream rejects params whose type
does not match the field's index type, and the prefetch-only case in the
test is what catches that.

The PHP setter validates its input and throws ZVecException; the C setter
returns void, so there is no status to check, and it clamps negatives to 0
as a second safety net. queryVector() and createIndex() errors keep flowing
through self::checkStatus().

Known limitation, documented in the README rather than fixed here:
prefetch is honoured only by queryVector(). The legacy query() path sends
just queryParamType, ef/nprobe, radius, isLinear and isUsingRefiner
through zvec_collection_query_ex, so prefetch is silently dropped. HNSW
prefetch has the same limitation today; fixing it needs new arguments on
that function and belongs in its own issue.

Tests: 193/193, 0 skipped, 0 failed, 2 expected fail. Index params cannot
be read back from an existing index, so test_vamana_two_pass_build.phpt
asserts on upstream's own CollectionSchema::to_string() output, which
embeds ",two_pass_build:true" per field. test_vamana_prefetch.phpt covers
both call orders, the 0/0 disable combination, and negative rejection.
test_hnsw_prefetch.phpt, test_vamana_index.phpt,
test_vector_query_vamana_params.phpt and test_docs_consistency.phpt pass
unchanged.
…ass-prefetch

# Conflicts:
#	CHANGELOG.md
#	tests/test_ffi_load.phpt
…ass-prefetch

# Conflicts:
#	CHANGELOG.md
#	tests/test_ffi_load.phpt
@s2x
s2x merged commit 9c74c69 into main Sep 29, 2026
6 checks passed
@s2x
s2x deleted the feat/220-vamana-two-pass-prefetch branch September 29, 2026 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Vamana two_pass_build index param and Vamana query prefetch

1 participant