feat: add 6 vector-search backend plugins#320
Merged
Conversation
…figuration and sync functionality
There was a problem hiding this comment.
Pull request overview
This PR adds six new vector-search backend plugins to the Apache Answer plugins repo to address Issue #319, enabling multiple external vector stores to be used for semantic search.
Changes:
- Introduced 6 new vector-search backend plugins: pgvector, qdrant, milvus, weaviate, elasticsearch, chromadb (each as its own Go module under
vector-search-*). - Implemented common capabilities across backends: config-driven initialization, embedding generation via Answer’s embedding API integration, search-by-vector, upsert/delete, and background full-sync via
VectorSearchSyncer. - Added per-plugin documentation and i18n strings (en_US / zh_CN) for Admin UI configuration.
Reviewed changes
Copilot reviewed 48 out of 53 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| vector-search-weaviate/weaviate.go | Weaviate backend implementation: config, client init, schema/class creation, search/upsert/delete. |
| vector-search-weaviate/sync.go | Background full-sync and bulk indexing for Weaviate backend. |
| vector-search-weaviate/README.md | Usage and configuration guide for Weaviate plugin. |
| vector-search-weaviate/info.yaml | Plugin metadata (slug/type/version/author/link) for Weaviate plugin. |
| vector-search-weaviate/i18n/zh_CN.yaml | Chinese translations for Weaviate plugin name/description/config fields. |
| vector-search-weaviate/i18n/en_US.yaml | English translations for Weaviate plugin name/description/config fields. |
| vector-search-weaviate/i18n/translation.go | Translation keys/constants for Weaviate plugin. |
| vector-search-weaviate/go.mod | Go module definition and dependencies for Weaviate plugin. |
| vector-search-qdrant/qdrant.go | Qdrant backend implementation: config, gRPC client, collection management, search/upsert/delete. |
| vector-search-qdrant/sync.go | Background full-sync and bulk indexing for Qdrant backend. |
| vector-search-qdrant/README.md | Usage and configuration guide for Qdrant plugin. |
| vector-search-qdrant/info.yaml | Plugin metadata for Qdrant plugin. |
| vector-search-qdrant/i18n/zh_CN.yaml | Chinese translations for Qdrant plugin. |
| vector-search-qdrant/i18n/en_US.yaml | English translations for Qdrant plugin. |
| vector-search-qdrant/i18n/translation.go | Translation keys/constants for Qdrant plugin. |
| vector-search-qdrant/go.mod | Go module definition and dependencies for Qdrant plugin. |
| vector-search-pgvector/pgvector.go | PostgreSQL+pgvector backend implementation: config, schema/table/index management, search/upsert/delete. |
| vector-search-pgvector/sync.go | Background full-sync and bulk indexing for pgvector backend. |
| vector-search-pgvector/README.md | Usage and configuration guide for pgvector plugin. |
| vector-search-pgvector/info.yaml | Plugin metadata for pgvector plugin. |
| vector-search-pgvector/i18n/zh_CN.yaml | Chinese translations for pgvector plugin. |
| vector-search-pgvector/i18n/en_US.yaml | English translations for pgvector plugin. |
| vector-search-pgvector/i18n/translation.go | Translation keys/constants for pgvector plugin. |
| vector-search-pgvector/go.mod | Go module definition and dependencies for pgvector plugin. |
| vector-search-milvus/milvus.go | Milvus backend implementation: config, collection/index management, search/upsert/delete. |
| vector-search-milvus/sync.go | Background full-sync and bulk indexing for Milvus backend. |
| vector-search-milvus/README.md | Usage and configuration guide for Milvus plugin. |
| vector-search-milvus/info.yaml | Plugin metadata for Milvus plugin. |
| vector-search-milvus/i18n/zh_CN.yaml | Chinese translations for Milvus plugin. |
| vector-search-milvus/i18n/en_US.yaml | English translations for Milvus plugin. |
| vector-search-milvus/i18n/translation.go | Translation keys/constants for Milvus plugin. |
| vector-search-milvus/go.mod | Go module definition and dependencies for Milvus plugin. |
| vector-search-elasticsearch/es.go | Elasticsearch backend implementation: config, index creation, kNN search, upsert/delete. |
| vector-search-elasticsearch/sync.go | Background full-sync and bulk indexing for Elasticsearch backend. |
| vector-search-elasticsearch/README.md | Usage and configuration guide for Elasticsearch plugin. |
| vector-search-elasticsearch/info.yaml | Plugin metadata for Elasticsearch plugin. |
| vector-search-elasticsearch/i18n/zh_CN.yaml | Chinese translations for Elasticsearch plugin. |
| vector-search-elasticsearch/i18n/en_US.yaml | English translations for Elasticsearch plugin. |
| vector-search-elasticsearch/i18n/translation.go | Translation keys/constants for Elasticsearch plugin. |
| vector-search-elasticsearch/go.mod | Go module definition and dependencies for Elasticsearch plugin. |
| vector-search-chromadb/chromadb.go | ChromaDB backend implementation via REST: collection ensure, search, upsert/delete. |
| vector-search-chromadb/sync.go | Background full-sync and bulk indexing for ChromaDB backend. |
| vector-search-chromadb/README.md | Usage and configuration guide for ChromaDB plugin. |
| vector-search-chromadb/info.yaml | Plugin metadata for ChromaDB plugin. |
| vector-search-chromadb/i18n/zh_CN.yaml | Chinese translations for ChromaDB plugin. |
| vector-search-chromadb/i18n/en_US.yaml | English translations for ChromaDB plugin. |
| vector-search-chromadb/i18n/translation.go | Translation keys/constants for ChromaDB plugin. |
| vector-search-chromadb/go.mod | Go module definition and dependencies for ChromaDB plugin. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The syncing flag was read outside the mutex before launching the sync goroutine, and re-checked after the goroutine had already reset it. This was a data race and could allow a redundant full sync to start. Claim the flag under the lock synchronously before spawning the goroutine, and reset it under the lock on exit.
The debug logs printed the entire raw config payload (which can include the embedding API key) and the full PostgreSQL DSN (which can include username/password). Log the config size and whether a DSN is set instead.
ensureClass failures triggered an anonymous-access retry for any error when an API key was set, which could silently downgrade to anonymous access on transient/network/schema errors. Gate the retry on an actual authentication error (WeaviateClientError with a 401 or 403 status code).
ensureIndex only created the index when absent and never validated the existing mapping, so a change in embedding dimensions left a stale index with mismatched dims (contradicting the README). Read the existing mapping and delete/recreate the index when the embedding dimensions differ, matching the pgvector, milvus, and qdrant plugins.
The pgvector plugin was the only vector-search plugin without a checked-in go.sum. Generate it (and the corresponding go.mod indirect requirements) so the module is reproducible, matching the sibling plugins.
indexEmbeddingDims used the client's GetMapping helper, which targets the
typed /_mapping/{type} endpoint removed in Elasticsearch 8, returning HTTP
400 and breaking the plugin configuration update. Query the untyped
GET /{index}/_mapping endpoint directly instead. Also make the dimension
check best-effort so a mapping-read failure logs a warning and keeps the
existing index rather than failing the whole config update.
Member
Author
|
resolved all comments and validated locally. |
LinkinStars
approved these changes
Jul 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Close #319