fix(db): index the keyword projection by document, and drop the unread binary ANN indexes

[waleedlatif1]

Summary

• Missing index (the fix): the document ACL trigger from the projection source/ACL rollout fans a document's source and ACL out to embedding_keyword_tin by document_id, and that projection had no index on the column. Every document ACL or source change scanned the whole projection; on a large deployment that is seconds per document update, thousands of times a day, and it slows every connector sync and the projection backfill with it. Adds embedding_keyword_tin_document_idx concurrently, mirroring the index the GIN keyword projection already has
• Unread indexes (the cleanup): drops the five binary-quantized HNSW indexes on embedding_search. Ranking walks the half-precision graphs and no code path orders by a hamming distance on this projection, so they were maintained on every chunk write for nothing; the populated one doubles the per-row cost of the projection backfill. Drops run CONCURRENTLY with the COMMIT breakpoint and lock-timeout handling the earlier index-drop migration uses
• The binary columns stay: the projection sync trigger still writes them. Their contract-pending marker in the schema now says so

Type of Change

• Bug fix

Testing

• bun run check:migrations origin/staging passes (backward-compatible)
• bun run lint and bun run check:audits pass
• Grepped the search layer and schema for readers of the binary columns or indexes: none outside an integration test that asserts they are not used

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 21, 2026 4:30pm UTC

[cubic-dev-ai]

No issues found across 4 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

<sub>Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic</sub>

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; the active half-precision search path remains indexed and the migration metadata is consistent.

Summary

This PR removes five unused binary-quantized HNSW indexes from the embedding_search projection while retaining the trigger-maintained binary columns.

• Drops each index concurrently and replay-safely after leaving the migration transaction.
• Removes the corresponding Drizzle schema declarations.
• Adds matching migration journal and snapshot metadata.
• Keeps the active half-precision cosine indexes and search path unchanged.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Source[Source embedding writes] --> Trigger[Projection sync trigger]
    Trigger --> Projection[embedding_search]
    Projection --> Binary[Binary columns retained]
    Projection --> Half[Half-precision vector columns]
    Binary -. five unused HNSW indexes dropped .-> Removed[Reduced index maintenance]
    Half --> Cosine[Cosine HNSW indexes retained]
    Cosine --> Search[Active ranking path]

Loading

_{Reviews (1) · Last reviewed commit: "chore(db): drop the unread binary ANN in..."}