zvec-php

PHP bindings for Alibaba's zvec vector database.

Two binding modes are available:

Native PHP extension (php-ext/) — compiled C++ extension, no FFI overhead (recommended)
FFI bindings (php/ + ffi/) — pure PHP via FFI, no compilation needed beyond the shared library

Bundled zvec version: v0.7.0. zvec itself is not compiled from source — we link against the official prebuilt SDK published by alibaba/zvec and compile only our adapter. See MIGRATION.md for what changed in this version, and note the one thing that will break your workflow: the test-suite command now needs an extra flag.

Overview

zvec-php provides PHP bindings for the zvec vector database through FFI (Foreign Function Interface). It allows you to create collections, insert documents with vectors, perform similarity searches, and manage indexes - all from PHP.

Features

Collections: Create, open, close, and destroy vector collections
Schema Definition: Define fields with various data types (INT64, STRING, FLOAT, DOUBLE, BOOL, INT32, UINT32, UINT64, VECTOR_FP32, VECTOR_FP64, VECTOR_FP16, VECTOR_INT4, VECTOR_INT8, VECTOR_INT16, VECTOR_BINARY32, VECTOR_BINARY64, SPARSE_VECTOR_FP32, SPARSE_VECTOR_FP16, BINARY, array types)
Document Operations: Insert, upsert, update, fetch, and delete documents
Vector Search: Perform similarity searches with HNSW, HNSW-RaBitQ, IVF, Flat, or Vamana indexes
Filtering: Pre-filter candidates with boolean expressions
Schema Evolution: Add, drop, rename, and alter columns dynamically
Index Management: Create and drop indexes via unified createIndex() + ZVecIndexParams
Doc Introspection: Check field existence, nullability, and enumerate scalar/vector fields
Batch Operations: Per-document status on batch insert/upsert/update
Reranking: Two-stage retrieval with RRF or weighted score fusion
Multi-Vector Search: Search across multiple vector fields simultaneously with hybrid fusion
Group-By Query: Group results by field value (limited upstream support)
Version API: Check zvec library version at runtime

Requirements

PHP 8.1+ (FFI mode) or PHP 8.5+ (native extension mode)
Linux x86_64 (for pre-built FFI library) or build tools for other platforms
CMake 3.14+ (if building from source)

Installation

Quick start via Composer (FFI mode, Linux x86_64)

composer require crazy-goat/zvec
vendor/bin/zvec-install

The zvec-install tool detects your OS and architecture and places two libraries in lib/: our small FFI adapter (libzvec_ffi, from this project's GitHub Releases) and the official prebuilt zvec SDK library (libzvec, from alibaba/zvec releases, pinned SHA-256). The version is auto-detected from Composer's installed.json. If omitted, specify it explicitly: vendor/bin/zvec-install v0.7.0.

Supported platforms (all pre-built):

Linux x86_64 / aarch64 (glibc 2.28+) — libzvec_ffi-linux-{x86_64,aarch64}.tar.gz
Linux x86_64 / aarch64 (musl, e.g. Alpine) — libzvec_ffi-linux-musl-{x86_64,aarch64}.tar.gz
macOS arm64 (macOS 15+) — libzvec_ffi-darwin-aarch64.tar.gz

Manual build (all platforms)

1. Clone

git clone <repository-url>
cd zvec-php

2. Build the native library

./build_zvec.sh

This will:

Download the official prebuilt zvec SDK into sdk/ (./fetch_zvec_sdk.sh, SHA-256 verified)
Build the FFI wrapper (ffi/build/libzvec_ffi.so) and copy libzvec next to it

Requires CMake 3.14+ and a C++17 compiler. zvec itself is no longer built from source.

3. Verify installation (FFI mode)

php run-tests.php -n -d extension=ffi.so tests/

Note: The goal is FFI enabled and the legacy zvec extension disabled. -n (no php.ini) handles the second, but it also strips any conf.d ini that provides FFI — on such a machine the suite then reports every test as SKIP. -d extension=ffi.so restores FFI afterwards. Check that Tests skipped is 0; a run with skips is a broken run, not a pass (#188).

Alternative: Build the native PHP extension

The native extension provides the same API without FFI overhead. It links dynamically with the zvec SDK library, which build_ext.sh copies next to modules/zvec.so.

# 1. Fetch the zvec SDK first (if not already done)
./fetch_zvec_sdk.sh

# 2. Build the extension
cd php-ext
bash build_ext.sh

# 3. Enable the extension (add to php.ini)
echo "extension=$(pwd)/modules/zvec.so" >> $(php -r 'echo php_ini_loaded_file();')

# 4. Verify
php -r 'echo extension_loaded("zvec") ? "zvec extension loaded" : "not loaded";'

When the extension is loaded, require_once 'src/ZVec.php' is a no-op (guard clause skips the FFI implementation), so existing code works without changes.

Upgrading from v0.4.x

See MIGRATION.md for a complete migration guide covering:

Index Creation — replace createHnswIndex(), createFlatIndex(), etc. with createIndex() + ZVecIndexParams
Statistics — replace stats() JSON parsing with typed getStatsStruct()
Schema Introspection — new getFieldSchema() API
Collection Options — use ZVecCollectionOptions with createWith()/openWith()
Query Object — use ZVecVectorQuery builder with queryVector()
Reranker in Queries — use queryWithReranker() instead of $reranker param on query()
Deprecated Schema Methods — rename addField*() to add*() (e.g. addFieldBinary() → addBinary())

Quick Start

<?php
require_once 'src/ZVec.php';

// Initialize zvec
ZVec::init(logType: ZVec::LOG_CONSOLE, logLevel: ZVec::LOG_WARN);

// Define schema
$schema = new ZVecSchema('my_collection');
$schema->addInt64('id', nullable: false, withInvertIndex: true)
    ->addString('title', nullable: false)
    ->addVectorFp32('embedding', dimension: 128, metricType: ZVecSchema::METRIC_IP);

// Create collection
$collection = ZVec::create('./my_collection', $schema);

// Insert documents
$doc = new ZVecDoc('doc_1');
$doc->setInt64('id', 1)
    ->setString('title', 'Hello World')
    ->setVectorFp32('embedding', [0.1, 0.2, 0.3, /* ... 128 dims */]);
$collection->insert($doc);

// Search
$results = $collection->query(
    'embedding',
    [0.1, 0.2, 0.3, /* ... query vector */],
    topk: 10
);

foreach ($results as $doc) {
    echo $doc->getPk() . ': ' . $doc->getScore() . "\n";
}

// Cleanup
$collection->close();

API Reference

ZVec (Collection)

// Library lifecycle
ZVec::init(
    int $logType = ZVec::LOG_CONSOLE,
    int $logLevel = ZVec::LOG_WARN,
    ?string $logDir = null,
    ?string $logBasename = null,
    int $logFileSize = 0,
    int $logOverdueDays = 0,
    int $queryThreads = 0,
    int $optimizeThreads = 0,
    float $invertToForwardScanRatio = 0.0,
    float $bruteForceByKeysRatio = 0.0,
    int $memoryLimitMb = 0,
    ?string $allowedBasePath = null,
    bool $verboseErrors = false,
    ?string $jiebaDictDir = null,          // folder with jieba.dict.utf8 + hmm_model.utf8
    ?float $ftsBruteForceByKeysRatio = null  // 0.0-1.0, upstream default 0.05
): void
ZVec::isInitialized(): bool
ZVec::getFtsBruteForceByKeysRatio(): float
ZVec::getJiebaDictDir(): string
ZVec::shutdown(): void
ZVec::getLastErrorDetails(): array
ZVec::clearError(): void
ZVec::getVersion(): string
ZVec::checkVersion(int $major, int $minor, int $patch): bool
ZVec::getVersionMajor(): int
ZVec::getVersionMinor(): int
ZVec::getVersionPatch(): int
ZVec::getIoBackendType(): int                    // ZVec::IO_BACKEND_PREAD|LIBAIO|IO_URING
ZVec::getIoBackendTypeName(int $type): string    // "pread" | "libaio" | "io_uring" | "unknown"
ZVec::getIoBackendDescription(): string          // DiskANN I/O backend, with install hints on Linux

// Collection lifecycle (static factories)
$collection = ZVec::create(string $path, ZVecSchema $schema, bool $readOnly = false, bool $enableMmap = true, int $maxBufferSize = 67108864): self
$collection = ZVec::open(string $path, bool $readOnly = false, bool $enableMmap = true, int $maxBufferSize = 67108864): self
$collection = ZVec::createWith(string $path, ZVecSchema $schema, ZVecCollectionOptions $options): self
$collection = ZVec::openWith(string $path, ZVecCollectionOptions $options): self

$collection->close(): void                     // flushes + releases the lock; idempotent; throws ZVecException if upstream refuses (e.g. iterators are open)
$collection->destroy(): void
$collection->flush(): void
$collection->optimize(int $concurrency = 0): void

// Collection introspection
$collection->schema(): string              // JSON string
$collection->path(): string                // On-disk path
$collection->options(): array              // readOnly, enableMmap, maxBufferSize
$collection->getOptions(): ZVecCollectionOptions
$collection->stats(): string               // JSON string
$collection->getStatsStruct(): ZVecCollectionStats
$collection->getFieldSchema(string $fieldName): ZVecFieldSchema

// Schema operations (add columns at runtime)
$collection->addColumnInt64(string $name, bool $nullable = true, string $defaultExpr = '0', int $concurrency = 0): void
$collection->addColumnFloat(string $name, bool $nullable = true, string $defaultExpr = '0', int $concurrency = 0): void
$collection->addColumnDouble(string $name, bool $nullable = true, string $defaultExpr = '0', int $concurrency = 0): void
$collection->addColumnString(string $name, bool $nullable = true, string $defaultExpr = '', int $concurrency = 0): void
$collection->addColumnBool(string $name, bool $nullable = true, string $defaultExpr = 'false', int $concurrency = 0): void
$collection->addColumnInt32(string $name, bool $nullable = true, string $defaultExpr = '0', int $concurrency = 0): void
$collection->addColumnUint32(string $name, bool $nullable = true, string $defaultExpr = '0', int $concurrency = 0): void
$collection->addColumnUint64(string $name, bool $nullable = true, string $defaultExpr = '0', int $concurrency = 0): void
$collection->dropColumn(string $name): void
$collection->renameColumn(string $oldName, string $newName, int $concurrency = 0): void
$collection->alterColumn(string $columnName, ?string $newName = null, ?int $newDataType = null, ?bool $nullable = null, int $concurrency = 0): void

// Index management
$collection->createIndex(string $fieldName, ZVecIndexParams $params, int $concurrency = 0): void
$collection->dropIndex(string $fieldName): void

// Deprecated index methods (use createIndex() + ZVecIndexParams instead)
$collection->createHnswIndex(...): void       // @deprecated
$collection->createHnswRabitqIndex(...): void // @deprecated
$collection->createFlatIndex(...): void       // @deprecated
$collection->createIvfIndex(...): void        // @deprecated
$collection->createInvertIndex(...): void     // @deprecated

// Document operations
$collection->insert(ZVecDoc ...$docs): void
$collection->upsert(ZVecDoc ...$docs): void
$collection->update(ZVecDoc ...$docs): void
$collection->insertBatch(ZVecDoc ...$docs): array     // Returns per-doc status array
$collection->upsertBatch(ZVecDoc ...$docs): array     // Returns per-doc status array
$collection->updateBatch(ZVecDoc ...$docs): array     // Returns per-doc status array
$collection->delete(string ...$pks): void
$collection->deleteByFilter(string $filter): void
$collection->fetch(string ...$pks): ZVecDoc[]        // also fetch(array $pks, ?array $outputFields = null, includeVector: bool = true)
$collection->iterDocs(?array $outputFields = null, bool $includeVector = true): ZVecDocIterator  // full scan over a snapshot; foreach ($it as $pk => $doc)
                                                     // named `outputFields:` supported; named-arg `pks:` and mixing scalar PKs with an array are rejected with a hint

// Search
$collection->query(string|ZVecVectorQuery $fieldName, array $queryVector = [], int $topk = 10, ...): ZVecDoc[]
$collection->queryFp16(string $fieldName, array $queryVector, int $topk = 10, ...): ZVecDoc[]
$collection->queryFp64(string $fieldName, array $queryVector, int $topk = 10, ...): ZVecDoc[]
$collection->queryVector(ZVecVectorQuery $query): ZVecDoc[]
$collection->queryMulti(array $vectorQueries, ZVecReRanker $reranker, int $topk = 10, ?string $filter = null, ?array $outputFields = null): ZVecRerankedDoc[]  // dense-only, fused in PHP
$collection->multiQuery(array $subQueries, ZVecRrfReRanker|ZVecWeightedReRanker $reranker, int $topk = 10, ?int $numCandidates = null, ?string $filter = null, ?array $outputFields = null, bool $includeVector = false, bool $includeDocId = false): ZVecDoc[]  // hybrid FTS + dense, fused in C++
$collection->queryByFilter(string $filter, int $topk = 100, ?array $outputFields = null): ZVecDoc[]
$collection->queryById(string $fieldName, string $docId, ...): ZVecDoc[]
$collection->queryWithReranker(string|ZVecVectorQuery $fieldName, ..., ?ZVecReRanker $reranker = null): ZVecRerankedDoc[]
$collection->groupByQuery(string|ZVecVectorQuery $fieldName, ...): array
$collection->groupByVectorQuery(ZVecGroupByVectorQuery $query): array

ZVecSchema

$schema = new ZVecSchema('collection_name');

// Segment configuration
$schema->setMaxDocCountPerSegment(int $count): self

// Scalar fields
$schema->addInt64(string $name, bool $nullable = false, bool $withInvertIndex = false): self
$schema->addString(string $name, bool $nullable = false, bool $withInvertIndex = false, ?ZVecIndexParams $indexParams = null): self
$schema->addFloat(string $name, bool $nullable = true): self
$schema->addDouble(string $name, bool $nullable = true): self
$schema->addBool(string $name, bool $nullable = false, bool $withInvertIndex = false): self
$schema->addInt32(string $name, bool $nullable = false, bool $withInvertIndex = false): self
$schema->addUint32(string $name, bool $nullable = false, bool $withInvertIndex = false): self
$schema->addUint64(string $name, bool $nullable = false, bool $withInvertIndex = false): self

// Dense vector fields
$schema->addVectorFp32(string $name, int $dimension, int $metricType = METRIC_IP): self
$schema->addVectorFp64(string $name, int $dimension, int $metricType = METRIC_IP): self
$schema->addVectorFp16(string $name, int $dimension, int $metricType = METRIC_IP): self
$schema->addVectorInt8(string $name, int $dimension, int $metricType = METRIC_IP): self
$schema->addVectorInt4(string $name, int $dimension, int $metricType = METRIC_IP): self
$schema->addVectorInt16(string $name, int $dimension, int $metricType = METRIC_IP): self
$schema->addVectorBinary32(string $name, int $dimension, int $metricType = METRIC_IP): self
$schema->addVectorBinary64(string $name, int $dimension, int $metricType = METRIC_IP): self

// Sparse vector fields
$schema->addSparseVectorFp32(string $name, int $metricType = METRIC_IP): self
$schema->addSparseVectorFp16(string $name, int $metricType = METRIC_IP): self

// Binary field
$schema->addBinary(string $name, bool $nullable = true): self

// Array fields
$schema->addArrayString(string $name, bool $nullable = true): self
$schema->addArrayBool(string $name, bool $nullable = true): self
$schema->addArrayInt32(string $name, bool $nullable = true): self
$schema->addArrayInt64(string $name, bool $nullable = true): self
$schema->addArrayUint32(string $name, bool $nullable = true): self
$schema->addArrayUint64(string $name, bool $nullable = true): self
$schema->addArrayFloat(string $name, bool $nullable = true): self
$schema->addArrayDouble(string $name, bool $nullable = true): self

// Metric type constants
$schema::METRIC_L2     = 1   // Euclidean distance
$schema::METRIC_IP     = 2   // Inner Product (default)
$schema::METRIC_COSINE = 3   // Cosine similarity
$schema::METRIC_MIPSL2 = 4   // Modified Inner Product with L2

// Note: addFieldBinary(), addFieldArrayString(), etc. are deprecated aliases
// that emit E_USER_DEPRECATED warnings.

ZVecDocIterator

Returned by iterDocs(). Scans every document with constant memory, over a snapshot taken when the iterator was created — so documents written afterwards are not visible to it. This is the full-scan path: no primary keys, no filter, no topk. Useful for export, backup and migration, which fetch() (needs PKs) and queryByFilter() (needs a filter and a topk) cannot serve.

foreach ($collection->iterDocs() as $pk => $doc) {
    fwrite($fh, json_encode(['pk' => $pk, 'id' => $doc->getInt64('id')]) . "\n");
}

Forward-only. rewind() after the first advance throws. Call iterDocs() again to scan afresh.
Auto-closes on exhaustion. A completed foreach needs no cleanup. Call close() explicitly when breaking out early; it is idempotent, and isClosed() reports the state.
Guards the collection. While one is open, close(), destroy(), addColumn*() / alterColumn() / dropColumn() and optimize() throw ZVecException with code 5 (FAILED_PRECONDITION). Writes, flush() and queries keep working. Close iterators first.
Seals a segment. On a writable collection each call seals the current writing segment, which may create a small one. A read-only collection is scanned without writing.
outputFields accepts scalar fields only — null means all of them, [] means primary key only. Vector or unknown names are rejected by upstream. Nullable fields that are unset come back as null, matching fetch().

ZVecDoc

$doc = new ZVecDoc('primary_key');

// Scalar setters (each returns self)
$doc->setInt64(string $field, int $value): self
$doc->setString(string $field, string $value): self
$doc->setFloat(string $field, float $value): self
$doc->setDouble(string $field, float $value): self
$doc->setBool(string $field, bool $value): self
$doc->setInt32(string $field, int $value): self
$doc->setUint32(string $field, int $value): self
$doc->setUint64(string $field, int $value): self

// Vector setters
$doc->setVectorFp32(string $field, array $vector): self
$doc->setVectorFp64(string $field, array $vector): self
$doc->setVectorFp16(string $field, array $vector): self   // uint16[] raw values
$doc->setVectorInt8(string $field, array $vector): self
$doc->setVectorInt4(string $field, array $vector): self
$doc->setVectorInt16(string $field, array $vector): self
$doc->setVectorBinary32(string $field, array $data): self
$doc->setVectorBinary64(string $field, array $data): self

// Sparse vector setters
$doc->setSparseVectorFp32(string $field, array $indices, array $values): self
$doc->setSparseVectorFp16(string $field, array $indices, array $values): self

// Binary setter
$doc->setBinary(string $field, string $data): self

// Array setters
$doc->setArrayInt32(string $field, array $data): self
$doc->setArrayInt64(string $field, array $data): self
$doc->setArrayUint32(string $field, array $data): self
$doc->setArrayUint64(string $field, array $data): self
$doc->setArrayFloat(string $field, array $data): self
$doc->setArrayDouble(string $field, array $data): self
$doc->setArrayString(string $field, array $strings): self
$doc->setArrayBool(string $field, array $data): self

// Document control
$doc->setFieldNull(string $field): self
$doc->removeField(string $field): self
$doc->clear(): self

// Scalar getters (each returns ?type, null if field missing)
$doc->getPk(): string
$doc->getDocId(): int        // Internal numeric id; 0 unless the query enabled it
$doc->getScore(): float
$doc->getInt64(string $field): ?int
$doc->getString(string $field): ?string
$doc->getFloat(string $field): ?float
$doc->getDouble(string $field): ?float
$doc->getBool(string $field): ?bool
$doc->getInt32(string $field): ?int
$doc->getUint32(string $field): ?int
$doc->getUint64(string $field): ?int

// Vector getters (each returns ?array)
$doc->getVectorFp32(string $field): ?array     // float[]
$doc->getVectorFp64(string $field): ?array     // float[]
$doc->getVectorFp16(string $field): ?array     // uint16[]
$doc->getVectorInt8(string $field): ?array     // int[]
$doc->getVectorInt4(string $field): ?array     // int[]
$doc->getVectorInt16(string $field): ?array    // int[]
$doc->getVectorBinary32(string $field): ?array // uint32[]
$doc->getVectorBinary64(string $field): ?array // uint64[]

// Sparse vector getters
$doc->getSparseVectorFp32(string $field): ?array  // {indices: int[], values: float[]}
$doc->getSparseVectorFp16(string $field): ?array  // {indices: int[], values: uint16[]}

// Binary getter
$doc->getBinary(string $field): ?string

// Array getters
$doc->getArrayInt32(string $field): ?array     // int[]
$doc->getArrayInt64(string $field): ?array     // int[]
$doc->getArrayUint32(string $field): ?array    // int[]
$doc->getArrayUint64(string $field): ?array    // int[]
$doc->getArrayFloat(string $field): ?array     // float[]
$doc->getArrayDouble(string $field): ?array    // float[]
$doc->getArrayString(string $field): ?array    // string[]
$doc->getArrayBool(string $field): ?array      // bool[]

// Introspection
$doc->hasField(string $name): bool
$doc->hasVector(string $name): bool
$doc->fieldNames(): string[]
$doc->vectorNames(): string[]
$doc->isFieldNull(string $field): bool
$doc->isEmpty(): bool

// Serialization
$doc->serialize(): string
$doc = ZVecDoc::deserialize(string $data): self
$doc->merge(ZVecDoc $other): self

// Operator control
$doc->setOperator(int $op): self
$doc->getOperator(): int
$doc::OP_INSERT = 0
$doc::OP_UPDATE = 1
$doc::OP_UPSERT = 2
$doc::OP_DELETE = 3

// Utility
$doc->getMemoryUsage(): int

ZVecIndexParams

// Factory methods for each index type:

// HNSW — graph-based index for high-dimensional vectors
$params = ZVecIndexParams::forHnsw(
    int $metricType = METRIC_IP,
    int $m = 50,
    int $efConstruction = 500,
    int $quantizeType = QUANTIZE_UNDEFINED,
    bool $useContiguousMemory = false
): self

// HNSW-RaBitQ — HNSW with Randomized Bit Quantization
$params = ZVecIndexParams::forHnswRabitq(
    int $metricType = METRIC_IP,
    int $totalBits = 7,
    int $numClusters = 16,
    int $m = 50,
    int $efConstruction = 500,
    int $sampleCount = 0
): self

// Flat — brute force exact search
$params = ZVecIndexParams::forFlat(
    int $metricType = METRIC_IP,
    int $quantizeType = QUANTIZE_UNDEFINED
): self

// IVF — inverted file index for large-scale search
$params = ZVecIndexParams::forIvf(
    int $metricType = METRIC_IP,
    int $nList = 1024,
    int $nIters = 10,
    bool $useSoar = false,
    int $quantizeType = QUANTIZE_UNDEFINED
): self

// IVF-RaBitQ — IVF partitions storing RaBitQ-quantized vectors
// Linux x86_64 with AVX2/AVX-512 only; FP32 vectors, dimension 64..4095,
// metric L2/IP/COSINE. Elsewhere createIndex() fails with NOT_SUPPORTED.
$params = ZVecIndexParams::forIvfRabitq(
    int $metricType = METRIC_IP,
    int $nList = 1024,
    int $totalBits = 7,
    int $sampleCount = 0
): self

// Vamana (DiskANN) — disk-based graph for 10K+ documents
$params = ZVecIndexParams::forVamana(
    int $metricType = METRIC_IP,
    int $maxDegree = 64,
    int $searchListSize = 100,
    float $alpha = 1.2,
    bool $saturateGraph = false,
    bool $useContiguousMemory = false,
    bool $useIdMap = false,
    int $quantizeType = QUANTIZE_UNDEFINED,
    bool $twoPassBuild = false   // v0.7.0: second full-graph build pass (better graph, slower build)
): self

// DiskANN — disk-based graph for billion-scale corpora
$params = ZVecIndexParams::forDiskAnn(
    int $metricType = METRIC_IP,
    int $maxDegree = 100,
    int $listSize = 50,
    int $pqChunkNum = 0,
    int $quantizeType = QUANTIZE_UNDEFINED
): self
// On Linux, check ZVec::getIoBackendType(). IO_BACKEND_PREAD means io_uring and
// libaio are both unavailable, so DiskANN reads are synchronous; install libaio
// (libaio1t64 on Ubuntu 24.04+) for async I/O.

// Full-Text Search — inverted index over a STRING column
$params = ZVecIndexParams::forFts(
    string $tokenizer = 'standard',
    string[] $filters = ['lowercase'],
    string $extraParams = ''
): self
// The same index can be declared up front, so it exists from the first insert
// instead of via a later createIndex() call:
//   $schema->addString('body', indexParams: ZVecIndexParams::forFts());
//   $schema->addString('tag',  indexParams: ZVecIndexParams::forInvert());
//
// Tokenizers: "standard", "ngram", "jieba", "whitespace"
// Filters:   "lowercase", "ascii_folding", "stemmer"
// extraParams is a JSON object, e.g. '{"stemmer_lang":"english"}',
//             '{"ngram_min":2,"ngram_max":3}' or '{"cut_mode":"mix"}'
// The jieba dictionary ships in zvec_data/jieba_dict next to the library and is
// used automatically. Lookup order: per-field extraParams jieba_dict_dir, then
// ZVEC_JIEBA_DICT_DIR, then ZVec::init(jiebaDictDir:), then the bundled copy.
//
// A bad jieba dictionary path is not a catchable error: cppjieba calls abort(),
// which kills the process with exit code 134. The bundled copy avoids this, and
// ZVec::init(jiebaDictDir:) validates that both dictionary files exist.

// Invert — keyword-based inverted index
$params = ZVecIndexParams::forInvert(
    bool $enableRange = true,
    bool $enableWildcard = false
): self

// Random rotation before INT8/INT4 quantization (reduces quantization error,
// improves recall; only effective with QUANTIZE_INT8 / QUANTIZE_INT4).
$params = ZVecIndexParams::forHnsw(ZVecSchema::METRIC_IP, quantizeType: ZVec::QUANTIZE_INT8)
    ->setQuantizerEnableRotate(true);

// Usage:
$collection->createIndex('embedding', ZVecIndexParams::forHnsw(ZVecSchema::METRIC_IP, quantizeType: ZVec::QUANTIZE_FP16));

ZVecCollectionOptions

$options = new ZVecCollectionOptions();
$options->readOnly = false;        // bool
$options->enableMmap = true;       // bool
$options->maxBufferSize = 67108864; // int (64 MB default)

// Factory methods:
$options = ZVecCollectionOptions::readOnly();
$options = ZVecCollectionOptions::readWrite();
$options = ZVecCollectionOptions::defaults();

// Fluent setters:
$options->setReadOnly(bool $readOnly): self
$options->getReadOnly(): bool
$options->setEnableMmap(bool $enableMmap): self
$options->getEnableMmap(): bool
$options->setMaxBufferSize(int $maxBufferSize): self
$options->getMaxBufferSize(): int

// Use with createWith/openWith:
$collection = ZVec::createWith('/path/to/collection', $schema, $options);
$collection = ZVec::openWith('/path/to/collection', $options);

ZVecFieldSchema

$schema = $collection->getFieldSchema('embedding');

$schema->getName(): string
$schema->getDataType(): int               // e.g. ZVec::TYPE_VECTOR_FP32
$schema->getElementDataType(): int         // element type for arrays/vectors
$schema->getElementDataSize(): int         // bytes per element
$schema->getDimension(): int               // vector dimension (0 for scalars)
$schema->isVectorField(): bool
$schema->isDenseVector(): bool
$schema->isSparseVector(): bool
$schema->isArrayType(): bool
$schema->isNullable(): bool
$schema->hasInvertIndex(): bool
$schema->hasIndex(): bool
$schema->getIndexType(): int              // ZVec::INDEX_TYPE_HNSW, etc.

ZVecCollectionStats

$stats = $collection->getStatsStruct();

$stats->getDocCount(): int
$stats->getIndexCount(): int
$stats->getIndexName(int $index): string
$stats->getIndexCompleteness(int $index): float
$stats->getAllIndexCompleteness(): array  // ['field' => completeness, ...]
$stats->toArray(): array                  // ['doc_count' => N, 'index_completeness' => [...]]

ZVecVectorQuery

$query = new ZVecVectorQuery('embedding', [0.1, 0.2, 0.3, ...]);

// Also create from document ID -- the vector is fetched when the query runs,
// so this works with query(), queryVector(), queryWithReranker() and
// groupByQuery() alike:
$results = $collection->query(ZVecVectorQuery::fromId('embedding', 'doc_1')->setTopk(5));

// Query parameters
$query->setFp64(bool $fp64 = true): self            // Use FP64 precision
$query->setHnswParams(int $ef): self                 // HNSW ef_search
$query->setHnswPrefetch(int $prefetchOffset, int $prefetchLines): self  // HNSW prefetch, 0 disables
$query->setVamanaPrefetch(int $prefetchOffset, int $prefetchLines): self  // Vamana prefetch, 0 disables
$query->setHnswRabitqParams(int $ef): self           // HNSW-RaBitQ ef_search
$query->setIvfParams(int $nprobe): self              // IVF nprobe
$query->setIvfRabitqParams(int $nprobe = 10): self  // IVF-RaBitQ nprobe
$query->setFlatParams(): self                        // Brute force mode
$query->setVamanaParams(int $efSearch): self         // Vamana ef_search
$query->setDiskAnnParams(int $listSize): self        // DiskANN search list size
$query->setFts(string $fieldName, string $queryString = '', string $matchString = '', string $defaultOperator = FTS_OPERATOR_OR): self
$query->setRadius(float $radius): self               // Range search radius
$query->setSparseVector(array $indices, array $values): self  // Sparse query vector; needs no dense vector
$query->setLinear(bool $linear): self                // Linear scan
$query->setUsingRefiner(bool $refiner): self         // Two-stage refine
$query->setTopk(int $topk): self
$query->setIncludeVector(bool $include): self
$query->setIncludeDocId(bool $include): self  // Also return internal numeric doc id
$query->setFilter(string $filter): self
$query->setOutputFields(array $fields): self

// Note: setHnswPrefetch() and setVamanaPrefetch() are honoured by queryVector().
// The legacy query() path passes only queryParamType, ef/nprobe, radius,
// isLinear and isUsingRefiner upstream, so prefetch is silently dropped there.

// Use with:
$results = $collection->queryVector($query);
$groups = $collection->groupByVectorQuery($query);

ZVecGroupByVectorQuery

$query = new ZVecGroupByVectorQuery(
    string $fieldName,       // Vector field to search
    array $queryVector,      // Query vector
    string $groupByField,    // Field to group results by
    int $groupCount = 2,     // Number of groups
    int $groupTopk = 3       // Results per group
);

// Setters
$query->setGroupByField(string $field): self
$query->setGroupCount(int $count): self
$query->setGroupTopk(int $topk): self
$query->setRadius(float $radius): self
$query->setLinear(bool $linear): self
$query->setUsingRefiner(bool $refiner): self
$query->setIncludeVector(bool $include): self
$query->setFilter(string $filter): self
$query->setOutputFields(array $fields): self

// Note: setTopk(), setHnswParams(), setHnswRabitqParams(), setIvfParams(),
// setFlatParams(), and setVamanaParams() throw ZVecException for group-by queries.

// Usage:
$groups = $collection->groupByVectorQuery($query);
// Returns: [['group_value' => '...', 'docs' => ZVecDoc[]], ...]

ZVecReRanker (Interface)

interface ZVecReRanker {
    public function rerank(array $queryResults): array;
    // $queryResults format: ['fieldName' => ZVecDoc[], ...]
    // Returns: ZVecRerankedDoc[]
}

ZVecRrfReRanker

Reciprocal Rank Fusion (RRF) reranker. Combines results from multiple vector fields using the RRF scoring formula: score = sum(1 / (rankConstant + rank)).

$reranker = new ZVecRrfReRanker(int $topn = 10, int $rankConstant = 60);

$reranker->getTopn(): int
$reranker->setTopn(int $topn): self
$reranker->getRankConstant(): int
$reranker->setRankConstant(int $rankConstant): self

// Usage:
$results = $collection->queryWithReranker('embedding', [...], reranker: $reranker);

ZVecWeightedReRanker

Weighted score fusion reranker. Combines results using normalized weighted scores (min-max normalisation per field).

$reranker = new ZVecWeightedReRanker(
    array $weights,                       // ['field_name' => weight, ...]
    int $topn = 10,
    int $metricType = ZVecSchema::METRIC_IP
);

$reranker->getTopn(): int
$reranker->setTopn(int $topn): self
$reranker->getMetricType(): int
$reranker->setMetricType(int $metricType): self
$reranker->getWeights(): array
$reranker->setWeights(array $weights): self

// Usage:
$reranker = new ZVecWeightedReRanker(['embedding' => 0.7, 'keywords' => 0.3]);
$results = $collection->queryMulti(
    [$query1, $query2],
    $reranker,
    topk: 10
);

Hybrid search (FTS + dense)

multiQuery() is the native path: it builds one upstream MultiQuery and upstream runs the sub-queries in parallel and fuses them in C++. queryMulti() cannot do this — it runs each sub-query separately through the legacy scalar path, where an FTS sub-query has no vector at all.

An FTS sub-query is an ordinary ZVecVectorQuery configured for text search ($fts), paired with a dense one:

$docs = $collection->multiQuery(
    [$fts, new ZVecVectorQuery('embedding', $queryVector)],
    new ZVecRrfReRanker(),
    topk: 10,
    numCandidates: 100,   // hits per sub-query before fusion; default max(topk*2, 100)
);

At least two sub-queries, and each field must exist and be indexed for the clause type used.
$numCandidates is the per-sub-query candidate count; larger means better recall and more work. topk is how many fused results come back.
The fused score is in $doc->getScore() and is not comparable with queryMulti(). Upstream normalises weighted fusion per field with atan, not min-max, so the numbers differ from ZVecWeightedReRanker even for the same inputs. RRF is the same 1/(k + rank + 1) in both. This is why queryMulti() keeps its PHP fusion for backward compatibility.
Weights must be a positional list, not a field-keyed map: fusion is positional upstream, and duplicate field names are legal. A map is rejected with an explicit error rather than silently mapped in the wrong order.
Returns plain ZVecDoc[], not ZVecRerankedDoc[] — the fusion happened upstream, so there are no per-field source ranks to report.
fromId() sub-queries are not supported yet; fetch the vector first.
Two sub-queries on the same field work here (upstream allows it) and are rejected by queryMulti(), which keyed its results by field name and would have dropped one.

ZVecRerankedDoc

Returned by reranker operations (queryWithReranker(), queryMulti()).

$rerankedDoc->getPk(): string
$rerankedDoc->getOriginalScore(): float      // Original score from first-pass query
$rerankedDoc->getCombinedScore(): float      // Reranked combined score
$rerankedDoc->getDoc(): ZVecDoc              // Access the underlying document
$rerankedDoc->getSourceRanks(): array        // ['fieldName' => rank, ...]
$rerankedDoc->getSourceScores(): array       // ['fieldName' => score, ...]

Data Types

Type Constant	Value	PHP Type	Description
`TYPE_INT32`	4	`int`	32-bit signed integer
`TYPE_INT64`	5	`int`	64-bit signed integer
`TYPE_UINT32`	6	`int`	32-bit unsigned integer
`TYPE_UINT64`	7	`int`	64-bit unsigned integer
`TYPE_FLOAT`	8	`float`	32-bit float (IEEE 754)
`TYPE_DOUBLE`	9	`float`	64-bit double (IEEE 754)
`TYPE_BOOL`	3	`bool`	Boolean
`TYPE_STRING`	10	`string`	UTF-8 string
`TYPE_BINARY`	1	`string`	Binary blob
`TYPE_VECTOR_FP32`	23	`float[]`	32-bit float vector
`TYPE_VECTOR_FP64`	24	`float[]`	64-bit float vector
`TYPE_VECTOR_FP16`	22	`int[]`	16-bit float vector (stored as uint16)
`TYPE_VECTOR_INT4`	25	`int[]`	4-bit integer vector
`TYPE_VECTOR_INT8`	26	`int[]`	8-bit integer vector
`TYPE_VECTOR_INT16`	27	`int[]`	16-bit integer vector
`TYPE_VECTOR_BINARY32`	20	`array`	32-dim binary vector
`TYPE_VECTOR_BINARY64`	21	`array`	64-dim binary vector
`TYPE_SPARSE_VECTOR_FP32`	31	`array`	Sparse float vector
`TYPE_SPARSE_VECTOR_FP16`	30	`array`	Sparse half-precision vector
`TYPE_ARRAY_STRING`	41	`string[]`	String array
`TYPE_ARRAY_BOOL`	42	`bool[]`	Boolean array
`TYPE_ARRAY_INT32`	43	`int[]`	Int32 array
`TYPE_ARRAY_INT64`	44	`int[]`	Int64 array
`TYPE_ARRAY_UINT32`	45	`int[]`	UInt32 array
`TYPE_ARRAY_UINT64`	46	`int[]`	UInt64 array
`TYPE_ARRAY_FLOAT`	47	`float[]`	Float array
`TYPE_ARRAY_DOUBLE`	48	`float[]`	Double array

Index Types

Constant	Value	Description
`INDEX_TYPE_HNSW`	1	Hierarchical Navigable Small World — graph-based, best for high-dim
`INDEX_TYPE_IVF`	2	Inverted File — partition-based, good for large-scale
`INDEX_TYPE_FLAT`	3	Brute force exact search — no index structure
`INDEX_TYPE_HNSW_RABITQ`	4	HNSW + RaBitQ quantization — memory-efficient
`INDEX_TYPE_IVF_RABITQ`	7	IVF + RaBitQ quantization (Linux x86_64, AVX2/AVX-512)
`INDEX_TYPE_VAMANA`	5	Vamana — in-memory graph, the DiskANN family for 10K+ documents
`INDEX_TYPE_DISKANN`	6	DiskANN — disk-based graph for billion-scale corpora
`INDEX_TYPE_INVERT`	10	Keyword-based inverted index
`INDEX_TYPE_FTS`	11	Full-Text Search over a STRING column

Metric Types

Constant	Value	Description
`METRIC_L2`	1	Euclidean distance (L2 norm)
`METRIC_IP`	2	Inner Product (dot product)
`METRIC_COSINE`	3	Cosine similarity
`METRIC_MIPSL2`	4	Modified Inner Product with L2

Quantize Types

Constant	Value	Description
`QUANTIZE_UNDEFINED`	0	No quantization (full precision)
`QUANTIZE_FP16`	1	16-bit float (2x memory reduction)
`QUANTIZE_INT8`	2	8-bit integer (4x memory reduction)
`QUANTIZE_INT4`	3	4-bit integer (8x memory reduction)
`QUANTIZE_RABITQ`	4	RaBitQ, implied by the HNSW-RaBitQ / IVF-RaBitQ index types; not valid with plain `forIvf()`

Query Param Types

Returned by the set*Params() / full-text-search methods on ZVecVectorQuery via its queryParamType property.

Constant	Value	Set by
`QUERY_PARAM_NONE`	0	default, no index-specific tuning
`QUERY_PARAM_HNSW`	1	`setHnswParams()`, `setHnswPrefetch()`
`QUERY_PARAM_IVF`	2	`setIvfParams()`
`QUERY_PARAM_FLAT`	3	`setFlatParams()`
`QUERY_PARAM_HNSW_RABITQ`	4	`setHnswRabitqParams()`
`QUERY_PARAM_VAMANA`	5	`setVamanaParams()`
`QUERY_PARAM_IVF_RABITQ`	7	`setIvfRabitqParams()`
`QUERY_PARAM_DISKANN`	6	`setDiskAnnParams()`
`QUERY_PARAM_FTS`	11	`setFts()`

FTS Constants

Constant	Value	Description
`FTS_OPERATOR_OR`	`'OR'`	A document may contain any of the terms (default)
`FTS_OPERATOR_AND`	`'AND'`	A document must contain every term

Running Tests

.phpt test suite (FFI mode)

php run-tests.php -n -d extension=ffi.so tests/

Note: -n disables php.ini, which avoids a legacy pre-installed zvec extension shadowing the FFI classes (#188) but can also take FFI with it. If your PHP has FFI compiled in, plain -n is enough; otherwise add -d extension=ffi.so. Either way Tests skipped must be 0.

Integration tests

php run-tests.php -n -d extension=ffi.so tests/

Project Structure

zvec-php/
├── src/                          # FFI-based PHP implementation
│   ├── ZVec.php                  # Main library class (ZVec)
│   ├── ZVecException.php         # Custom exception
│   ├── ZVecCollectionOptions.php # Collection open/create options
│   ├── ZVecCollectionStats.php   # Collection statistics object
│   ├── ZVecFieldSchema.php       # Field schema introspection
│   ├── ZVecIndexParams.php       # Index creation parameters
│   ├── ZVecQueryInterface.php    # Query interface
│   ├── ZVecVectorQuery.php       # Vector query builder
│   ├── ZVecGroupByVectorQuery.php # Group-by vector query builder
│   ├── ZVecSchema.php            # Schema definition
│   ├── ZVecDoc.php               # Document handle
│   ├── ZVecDocIterator.php       # Full-scan document iterator
│   ├── ZVecReRanker.php          # Base re-ranker interface
│   ├── ZVecRerankedDoc.php       # Reranked document class
│   ├── ZVecRrfReRanker.php       # RRF re-ranker
│   ├── ZVecWeightedReRanker.php  # Weighted re-ranker
│   ├── embeddings/               # Embedding function interfaces
│   └── Installer.php             # Composer CLI installer
├── examples/                     # Usage examples
├── php-ext/                      # Native PHP extension (C++)
│   ├── zvec_collection.cc/h      # ZVec class
│   ├── zvec_schema.cc/h          # ZVecSchema class
│   ├── zvec_doc.cc/h             # ZVecDoc class
│   ├── zvec_vector_query.cc/h    # ZVecVectorQuery class
│   ├── zvec_reranker.cc/h        # ZVecReRanker interface
│   ├── zvec_rrf_reranker.cc/h    # ZVecRrfReRanker
│   ├── zvec_weighted_reranker.cc/h # ZVecWeightedReRanker
│   ├── config.m4                 # phpize build configuration
│   └── build_ext.sh              # Build script
├── ffi/                          # C FFI bridge (used by php/ implementation)
│   ├── zvec_ffi.h                # C header with FFI declarations
│   ├── zvec_ffi_php.h            # PHP-specific FFI header
│   ├── zvec_ffi.cc               # C++ implementation
│   └── CMakeLists.txt            # Build configuration
├── tests/                        # .phpt test suite
├── tasks/                        # Feature planning documents
├── fetch_zvec_sdk.sh             # Downloads the official prebuilt zvec SDK into sdk/
├── build_ffi.sh                  # Builds FFI shared library
├── build_zvec.sh                 # Orchestrator: fetch SDK + build FFI
└── sdk/                          # Official zvec SDK (downloaded, not committed)

Implemented Features

See tasks/done/ for detailed planning documents.

Security

See SECURITY.md for the full security policy, including how to report vulnerabilities and which versions are supported.

Trust Model

The FFI shared library (libzvec_ffi.so) is loaded into the PHP process address space and runs with the same system privileges as the PHP process itself. Only load .so files from trusted sources. Pre-built libraries are downloaded from GitHub Releases with SHA-256 checksum verification.

Input Validation

Collection paths ($path in create()/open()): Should NOT come from untrusted user input. An attacker could create or access arbitrary directories. Use ZVec::init(allowedBasePath: '/data/zvec') to restrict paths.
Filter expressions ($filter in query()/deleteByFilter()): Passed directly to the zvec C++ engine. Sanitize any user-controlled filter strings. The filter language does not support SQL injection, but malformed expressions may cause errors or unexpected behavior.
Document primary keys: Passed directly to the C++ layer. Ensure PK values are validated if sourced from user input.

Memory Limits

Use ZVec::init(memoryLimitMb: 1024) to cap collection cache memory. Without a memory limit, collections may consume all available system memory under heavy write or search load.

File Permissions

Collection directories inherit the OS default umask. On shared hosting, ensure collection paths are not world-readable. Set restrictive permissions explicitly after creation if needed:

chmod 700 /path/to/collection

Supply Chain

Pre-built shared libraries are downloaded from GitHub Releases via vendor/bin/zvec-install: the FFI adapter from this project's release (checked against its checksums.sha256) and the zvec SDK from alibaba/zvec (checked against a SHA-256 pinned in src/Installer.php). Downloads use HTTPS with TLS verification (CA bundle). SHA-256 checksums are verified before extraction using hash_equals() (timing-safe comparison).

For production deployments, build the FFI adapter yourself with ./build_zvec.sh (it compiles ffi/zvec_ffi.cc against the same pinned upstream SDK).

Known Limitations

GroupByQuery: The C++ API has this method but it returns all documents in a single group with empty group value. This is a known issue in upstream zvec (marked as "Coming Soon" in zvec docs).
Platform: Pre-built for Linux x86_64/aarch64 (glibc and musl) and macOS arm64 (macOS 15 or newer — the upstream SDK is built for 15.0). No macOS x86_64 build (upstream ships no SDK for it).
alterColumn(): Requires nullable to be explicitly specified when changing data type. Cannot rename AND change type in one call. Only scalar numeric types (INT32, INT64, UINT32, UINT64, FLOAT, DOUBLE) are supported for type changes.
GroupByVectorQuery: Does not support HNSW, IVF, Flat, or Vamana query parameters (setTopk(), setHnswParams(), etc. throw ZVecException).
queryById() and ZVecVectorQuery::fromId(): both fetch the source document's vector client-side and then run a normal query, because upstream has no native "query by document id" — the Python SDK does the same. So neither is a single round-trip to the C++ layer. The source document is normally the first result but is not removed from it. fromId() works with query(), queryVector(), queryWithReranker(), queryMulti() and groupByQuery(); FP16 fields need queryById(), which has a dedicated half-precision path.

Known Security Limitations

ID	Description	Issue	Status
SEC-001	SHA-256 checksum verification added to downloaded `.so` files	#70	✅ Fixed v0.4.12
SEC-002	`tempnam()` symlink race replaced with cryptographically random temp directory	#71	✅ Fixed v0.4.12
SEC-004	Null pointer safety in C++ FFI bridge — 50+ handle-accepting functions	#73	🔄 In Progress
SEC-008	API key masking in `var_dump()` + `sodium_memzero()` in destructor	#76	✅ Fixed v0.4.12
SEC-012	Explicit SSL certificate verification in embedding API requests	#80	✅ Fixed v0.4.12

License

Same as upstream zvec project (Apache License 2.0)

Name		Name	Last commit message	Last commit date
.github		.github
bin		bin
docs		docs
examples		examples
ffi		ffi
lib		lib
php-ext		php-ext
src		src
test_dbs		test_dbs
tests		tests
.gitignore		.gitignore
AGENTS.md		AGENTS.md
CHANGELOG.md		CHANGELOG.md
MIGRATION.md		MIGRATION.md
README.md		README.md
SECURITY.md		SECURITY.md
build_ffi.sh		build_ffi.sh
build_zvec.sh		build_zvec.sh
composer.json		composer.json
fetch_zvec_sdk.sh		fetch_zvec_sdk.sh
repomix-output.xml		repomix-output.xml
run-tests.php		run-tests.php

Latest commit

History

Folders and files

Repository files navigation