feat: expose the DiskANN I/O backend (#224) - #230
Merged
Merged
Conversation
zvec picks an I/O backend for DiskANN disk reads on first use: on Linux
it tries io_uring, then libaio, then falls back to synchronous pread();
macOS arm64 always uses pread(). The choice dominates DiskANN
throughput -- pread on Linux usually means libaio simply is not installed
-- and until now there was no way to observe it from PHP.
Three static getters on ZVec, next to getVersion() and following the
same pattern the Python SDK uses for its module-level zvec.io_backend_type():
ZVec::getIoBackendType(): int
ZVec::getIoBackendTypeName(int $type): string
ZVec::getIoBackendDescription(): string
plus IO_BACKEND_PREAD / IO_BACKEND_LIBAIO / IO_BACKEND_IO_URING (0/1/2),
the same values as the upstream C ABI. The description is where upstream
explains how to install an async backend when pread is in use.
These are deliberately static rather than per-collection: the value is
process-wide, it is not part of GlobalConfig, and upstream probes it
lazily, so the getters work without ZVec::init() at all. The test asserts
that.
Adhering to the #215 singleton rule: the adapter includes only the public
zvec/ailego/io/io_backend.h and calls upstream's exported
current_io_backend_type() / current_io_backend_description(), which are
compiled inside libzvec, so the IOBackend singleton is created only there.
The internal io_backend_def.h is not part of the SDK and IOBackend::Instance()
is never referenced from this module. The comment in ffi/zvec_ffi.cc records
why, so it does not get "fixed" later.
$ nm -C ffi/build/libzvec_ffi.so | grep -c 'IOBackend::Instance'
0
$ nm -D --defined-only ffi/build/libzvec_ffi.so | grep io_backend
zvec_get_io_backend_description
zvec_get_io_backend_type
zvec_get_io_backend_type_name
No status to check: these return plain values, matching getVersion(), so
self::checkStatus() is not involved and FFI::free() is not needed -- the
C side owns the memory (a static literal for the name, a thread-local
buffer for the description).
Tests: 192/192, 0 skipped, 0 failed, 2 expected fail. The new
tests/test_io_backend.phpt covers the constants, the name mapping including
the "unknown" fallback, that the value is valid before init(), that the
description mentions the reported backend, the macOS pread rule, and that
the cached value is stable across repeated calls and across init().
tests/test_ffi_load.phpt gained the three symbols (60 -> 63).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #224.
Summary
zvec picks an I/O backend for DiskANN disk reads on first use. On Linux it tries
io_uring, thenlibaio, then falls back to synchronouspread(); macOS arm64 always usespread(). The choice dominates DiskANN throughput —preadon Linux usually just means libaio is not installed — and it was not observable from PHP at all.Design notes
Static, not per-collection. The value is process-wide, it is not part of
GlobalConfig(upstreamconfig.hhas no io-backend accessor), and upstream probes it lazily — so the getters work withoutZVec::init(), which the test asserts. This mirrors the Python SDK's module-levelzvec.io_backend_type()and the existing version API.The #215 singleton rule holds. The adapter includes only the public
zvec/ailego/io/io_backend.hand calls upstream's exportedcurrent_io_backend_type()/current_io_backend_description(), which are compiled inside libzvec, so theIOBackendsingleton is created only there. The internalio_backend_def.his not part of the SDK, andIOBackend::Instance()is never referenced from this module. There is a comment inffi/zvec_ffi.ccexplaining why, so it does not get "fixed" later.No status handling needed. These return plain values rather than
zvec_status_t, soself::checkStatus()is not involved, matchinggetVersion(). The C side owns the memory: a static literal for the name, athread_localbuffer for the description, so noFFI::free().The name mapping is written as a local
switchin the adapter rather than pulled from upstream's internal header, returning"unknown"for any unrecognised value.Verification
Full suite from a clean build:
tests/test_io_backend.phptcovers the constant values, the name mapping including theunknownfallback, that the value is valid beforeinit(), that the description mentions the reported backend, the macOS pread rule, and that the cached value is stable across repeated calls and acrossinit().tests/test_ffi_load.phptgained the three symbols (60 → 63).test_dbs/is left with only.gitignore.🤖 Generated with Claude Code