feat(init): jiebaDictDir and ftsBruteForceByKeysRatio options (#221)

[s2x]

Closes #221.

Summary

Two upstream GlobalConfig::ConfigData fields now have PHP equivalents:

ZVec::init(
    // ...existing parameters...
    ?string $jiebaDictDir = null,            // folder with jieba.dict.utf8 + hmm_model.utf8
    ?float $ftsBruteForceByKeysRatio = null, // 0.0-1.0, upstream default 0.05
);

ZVec::getJiebaDictDir(): string
ZVec::getFtsBruteForceByKeysRatio(): float

jiebaDictDir is the folder the jieba FTS tokenizer loads its dictionary from. ftsBruteForceByKeysRatio is the point at which an FTS query stops walking posting lists and starts scoring candidates one by one — that pays off when the scalar filter is very selective, and it is separate from the existing bruteForceByKeysRatio because scoring one FTS candidate costs more.

With neither option given, the dictionary shipped in zvec_data/jieba_dict is still found automatically.

Why the validation has to be in PHP

This is the substantive part of the change, so to be explicit about it:

Upstream validates too late to be safe. GlobalConfig::initialize() sets its "initialized" flag before it validates. If it then rejects the value, the library is still marked initialized, with default settings, and every later init() returns OK without applying anything. A bad value would therefore be a silent misconfiguration rather than an error. Rejecting in PHP, before any FFI call, is the only place this can be caught.

NAN slips through upstream. NAN < 0 and NAN > 1 are both false, so the upstream range test accepts it.

A wrong jieba folder is fatal, not an exception. cppjieba calls abort() rather than returning a Status, so a bad folder kills the process with exit code 134 later — when a jieba FTS index is created — with nothing catchable in PHP:

FATAL exp: [ifs.is_open()] false. open /nonexistent/dir/jieba.dict.utf8 failed.

The check therefore requires both jieba.dict.utf8 and hmm_model.utf8, not just the directory. Six rejection cases are asserted, each followed by isInitialized() still being false:

input	message
ftsBruteForceByKeysRatio: 1.5	must be between 0 and 1, got: 1.5
ftsBruteForceByKeysRatio: -0.1	must be between 0 and 1, got: -0.1
ftsBruteForceByKeysRatio: NAN	must be between 0 and 1, got: NAN
jiebaDictDir: ''	does not exist:
jiebaDictDir: '/nonexistent/zvec/jieba'	does not exist: /nonexistent/zvec/jieba
an existing but empty dir	is missing jieba.dict.utf8: ...

null rather than 0.0 as the ratio default, because 0.0 is a real value upstream accepts, not a "use the default" marker like the other ratios.

The getters go through global_config_ptr() rather than GlobalConfig::Instance(), per the #215 singleton rule.

One small detail: the ratio message uses var_export(), because interpolating NAN emits a PHP warning (unexpected NAN value was coerced to string).

Verification

The two new tests run in separate processes, because upstream config is applied only once per process.

tests/test_init_fts_options.phpt initializes against a copy of the bundled dictionary and then runs an end-to-end jieba FTS query with no per-field jieba_dict_dir, which is what proves the custom folder is the one actually used:

ratio: 0.20
dir matches: yes
fts hit: j1

Full suite from a clean build:

Number of tests : 194               194
Tests skipped   :   0 (  0.0%)
Tests failed    :   0 (  0.0%)
Expected fail   :   2 (  1.0%)
Tests passed    : 192 ( 99.0%)

tests/test_ffi_load.phpt gained the four FFI symbols (63 → 67). test_dbs/ is left with only .gitignore.

Follow-up worth filing

The same cppjieba abort() is reachable through per-field extraParams jieba_dict_dir / user_dict_path and through the ZVEC_JIEBA_DICT_DIR environment variable, none of which are validated. That is out of scope here (it would mean validating inside ZVecIndexParams::forFts()) but it is the same failure mode.

🤖 Generated with Claude Code