Skip to content

C API: expose update_entries and replace_external_id for dynamic index - #468

Draft
yuejiaointel wants to merge 2 commits into
dev/razdoburdin/concuttent_update_entriesfrom
dev/yuejiao/capi-update-replace-id
Draft

yuejiaointel wants to merge 2 commits into
dev/razdoburdin/concuttent_update_entriesfrom
dev/yuejiao/capi-update-replace-id

Conversation

@yuejiaointel

@yuejiaointel yuejiaointel commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Exposes two concurrent-index methods through the C API so integrations (e.g. valkey-search) can change a vector in place or rename its ID without delete + re-insert.

Stacked on #467 (update_entries); base is that PR's branch.

New C functions

  • svs_index_dynamic_update_points(index, points, ids, num_points, err) → update_entries
    • FINE_GRAIN: supported, one point per call (same restriction as add_points / delete_points).
    • NONE / GLOBAL: returns SVS_ERROR_NOT_IMPLEMENTED (the non-concurrent index has no update_entries).
  • svs_index_dynamic_replace_id(index, old_id, new_id, err) → replace_external_id
    • Works for all sync kinds.
    • Missing old_id or existing new_id → SVS_ERROR_INVALID_ARGUMENT.

Plumbed through DynamicVamanaInterface / DynamicVamana and the C API DynamicIndex wrappers. New virtuals are appended at the end of both interfaces.

Testing

  • Existing C API tests: ctest 26/26 pass (public build, SVS_RUNTIME_ENABLE_LVQ_LEANVEC=OFF).
  • Local functional check (not part of this PR), 200 random 16-d vectors, L2, all three sync kinds:
    • FINE_GRAIN update: after updating id 5 with a new vector v, searching v returns id 5 at distance 0 (before the update it returned id 51).
    • replace_id(5, 1000): has_id(5) false, has_id(1000) true, search returns 1000.
    • Bad ids → SVS_ERROR_INVALID_ARGUMENT; update on NONE/GLOBAL → SVS_ERROR_NOT_IMPLEMENTED.
  • Not built locally: the LVQ/LeanVec static-library variant.

🤖 Generated with Claude Code

@eshenayo

eshenayo commented Nov 10, 2026 •

Copy link
Copy Markdown

Thanks @yuejiaointel for exposing update_points and replace_id so quickly. I've started integrating against this branch.

One follow-up request:

svs_index_dynamic_update_points only works on FINE_GRAIN handles, but
svs_index_load_stream_dynamic(builder, stream, blocksize_bytes, err) takes no
sync kind. Every index restored from a stream therefore comes back as
SVS_SYNC_KIND_NONE. On such an index:

  • update_points returns SVS_ERROR_NOT_IMPLEMENTED, and
  • delete_points erases the id from the translator, so the deleted slot can no
    longer be revived with update_points or renamed with replace_id.

valkey-search loads every index from a stream (RDB), so after a restart or a
replica full sync it cannot reuse deleted slots until the index is rebuilt.

Could #468 (or a follow-up) add the stream counterpart of
svs_index_load_dynamic_ex?

SVS_API svs_index_h svs_index_load_stream_dynamic_ex(
    svs_index_builder_h builder,
    svs_stream_i stream,
    const svs_dynamic_index_params_t* params /*=NULL*/,
    svs_error_h out_err /*=NULL*/
);

Same semantics as svs_index_load_dynamic_ex: the block fields and
sync_kind from params are used, and NULL means defaults. With FINE_GRAIN
the loaded handle should be the concurrent index, so update_points and
replace_id work on it immediately.

@eshenayo

Copy link
Copy Markdown

For the C API side of the proposal in #467 (comment) (benchmark numbers there). If the approach holds up, valkey-search would need, for FINE_GRAIN dynamic indexes:

  • reuse_empty for add_points, so adds refill the slots consolidation freed ([C API] Reuse slots freed by consolidate in svs_index_dynamic_add_points #438). The orchestrator's DynamicVamana::add_points already takes it; ConcurrentIndexVamana::add_points in bindings/c/src/index_vamana.cpp doesn't pass it.
  • A step-wise consolidation entry point, for example svs_index_dynamic_consolidate_step(index, max_work, bool* out_done, err), over a resumable pass in the concurrent index.
  • A stream load that takes the sync kind, so RDB-restored indexes are FINE_GRAIN too (requested above and in [C API] Stream load that accepts svs_dynamic_index_params_t #427). It is still needed, because restored indexes can't use any of this.

The prototype (patch in the #467 comment) is local only; nothing here is a request to change this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants