[IlyaKhalafi]

This PR adds the larger Chinese BGE model requested in #289. With this, BAAI/bge-large-zh-v1.5 produces 1024-dimensional embeddings using Xenova's ONNX export and the original model's 512-token context limit.

The explicit tokenizer cap handles the export's unknown-length sentinel while preserving any stricter configured limit.

Fixes #289.

57 focused tests passed, and 6 real-model checks passed in manual mode against values from the original PyTorch model. Normal PR CI skips the large-model download. Pre-commit, mypy and the type stub check passed. GPU was not tested.

Checklist

• Explained why the model is included
• Added canonical values from the original model and generation code
• Tested locally
• Ran pre-commit hooks

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 9fb87132-eef7-4f39-bc3c-7a3ef3b42456

📥 Commits

Reviewing files that changed from the base of the PR and between f48af32 and 56b343b.

📒 Files selected for processing: 5

• fastembed/common/preprocessor_utils.py
• fastembed/text/onnx_embedding.py
• tests/test_bge_large_zh.py
• tests/test_preprocessor_utils.py
• tests/test_text_onnx_embeddings.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

📝 Walkthrough

The tokenizer-loading functions now accept an optional context limit and use the smallest valid limit from the supplied value and tokenizer configuration. The supported ONNX model list adds BAAI/bge-large-zh-v1.5, and its tokenizer loads with a 512-token limit. New tests cover context validation, token counts, embedding reference values, normalization, and sequential versus parallel results.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Feature

Merge Risk: ⚪ Minimal · up to 56b34

No actionable issue is established for the new model support; it is mergeable after normal checks.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 35.71% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 5 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description check		The description clearly explains the addition of BAAI/bge-large-zh-v1.5, its tokenizer limit, testing, and issue coverage.
Title check		The title clearly and concisely summarizes the main change: support for the larger Chinese BGE model.
Linked Issues check		Issue #289 requests support for BAAI/bge-large-zh-v1.5. The PR adds this model to supported_onnx_models with 1024 dimensions, the Xenova/bge-large-zh-v1.5 source, and onnx/model.onnx. `OnnxTex…
Out of Scope Changes check		The changes remain within issue #289. The tokenizer limit changes provide the required 512-token behavior and preserve stricter configured limits. The added tests verify the new model and the tokenize…

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}