fix(web): deduplicate repository lookups in search results

[dipeshbabu]

Fixes #1681

Batch uncached repository IDs into one scoped Prisma query per chunk and cache missing IDs/names for the request. Preserve legacy lookups and file order; tests cover mixed identifiers and streamed metadata mappings.

Validation: application tests, lint, Docker builds, Prisma migration gate, and regression checks passed.

---

Note

Low Risk
Search-path performance fix with behavior preserved (file order, legacy name lookups, omitting unknown repos); risk is limited to repository metadata resolution logic, now covered by expanded tests.

Overview
Fixes redundant Prisma calls when enriching Zoekt file matches with repository metadata.

createReposMapForChunk now collects unique repository identifiers per chunk, batches numeric IDs into a single findMany, and records missing IDs/names in the per-request cache (Repo | null) so later files and streamed chunks do not re-query. Legacy shards without repository_id still resolve via one findFirst per distinct name; numeric IDs and name "1" stay separate.

Adds changelog entry and regression tests for unary/streaming search (batching, missing repos, cache scope per stream, mixed ID/name shards).

^{Reviewed by Cursor Bugbot for commit 0f2600d. Bugbot is set up for automated code reviews on this repo. Configure here.}

---

Summary by cubic

Fixes #1681 so search results with many files from the same repository no longer trigger a database lookup per file.

Deduplicates repository identifiers before resolving repository metadata in zoektSearcher.ts, so 100 files from one repository make one lookup and 100 files across two repositories make two. Legacy name-based lookups, missing-repository handling, result order, and the existing metadata cache across streamed chunks are preserved. Adds a changelog entry.

Tests

• Adds regression coverage for repository ID deduplication, name-based lookups, missing repositories, and streaming chunks reusing cached metadata.

^{Written for commit cde93da. Summary will update on new commits.}

Summary by CodeRabbit

• Bug Fixes
  • Reduced repeated repository metadata lookups when processing search results, including across streamed result batches.
  • Improved search handling for missing repository information: files without a matching repository are omitted, and repeated checks for missing repositories are avoided.
  • Search now handles numeric repository IDs and legacy repository names separately, while grouping numeric ID lookups to reduce redundant queries.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 84f48708-9b4c-4fec-9ec2-d2db2e6d9f7b

---

📥 Commits

Reviewing files that changed from the base of the PR and between 5fd8298 and 0f2600d.

---

📒 Files selected for processing (1)

• CHANGELOG.md

---

🚧 Files skipped from review as they are similar to previous changes (1)

• CHANGELOG.md

---

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

---

---

Walkthrough

The searcher deduplicates repository lookups within result chunks. It batches uncached numeric IDs, resolves legacy names, and caches missing repositories during unary and streaming searches.

Changes

Repository lookup deduplication

Layer / File(s)	Summary
Deduplicate repository lookups per chunk packages/web/src/features/search/zoektSearcher.ts, packages/web/src/features/search/zoektSearcher.test.ts, CHANGELOG.md	createReposMapForChunk batches lookups for unique numeric IDs and resolves legacy names. It caches missing results. Tests cover lookup batching, missing repositories, legacy names, and streaming cache behavior. The changelog records the fix.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 0f260

No actionable merge-blocking issue is identified in the supplied change; it is mergeable after normal checks.

Pre-merge checks | 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: deduplicating repository lookups during web search result processing.
Linked Issues check		The implementation satisfies issue #1681. createReposMapForChunk deduplicates identifiers per chunk, batches uncached numeric IDs with one scoped findMany, and resolves each uncached legacy name o…
Out of Scope Changes check		The changes stay within issue #1681. Production changes remove duplicate repository lookups and preserve search hydration behavior. Regression tests verify the required lookup and cache behavior. The …
Docstring Coverage		No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 2…

---

---

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

---

---

---

_{Comment @coderabbitai help to get the list of available commands.}

[cubic-dev-ai]

No issues found across 3 files

_{Re-trigger cubic}

[cubic-dev-ai]

2 issues found and verified against the latest diff

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/web/src/features/search/zoektSearcher.test.ts">

<violation number="1" location="packages/web/src/features/search/zoektSearcher.test.ts:188">
P3: The streaming test verifies only file and repositoryInfo counts, never that each `response.files[i].repositoryId` corresponds to the emitted `repository_id`. The unary tests assert this mapping; the streaming chunk test should too, so a cache mis-association or reordering that keeps counts unchanged cannot slip through — it also directly covers the PR's "result order preserved" claim for the cross-chunk cache path.</violation>
</file>

<file name="packages/web/src/features/search/zoektSearcher.ts">

<violation number="1" location="packages/web/src/features/search/zoektSearcher.ts:344">
P3: Within a chunk the dedup is correct, but repositories that are not in the database never get cached (`if (repo) { reposMapCache.set(id, repo); }` only stores hits), so a missing repo referenced by a shard is still looked up once per streaming chunk instead of once per stream. Since this PR exists to collapse repeated lookups, track ids already queried in this stream (e.g., a `Set` of looked-up ids next to `_reposMapCache`) and skip re-querying them.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[riteshvish02]

Great catch @dipeshbabu! Deduplicating repository IDs before resolving metadata eliminates redundant lookups when multiple file matches belong to the same repository.

One additional optimization to consider: Currently, the PR deduplicates IDs but still uses Promise.all(repoIds.map(...)) with individual findUnique calls:

const repoIds = [...new Set(chunk.files.map(getRepoIdForFile))];
await Promise.all(repoIds.map(async (id) => {
    ...
    await prisma.repo.findUnique({ where: { id } })
}));

While this collapses 100 files in 1 repo to 1 query, in broad or multi-repository searches where a chunk spans matches across 15–20 repositories, repoIds.map(findUnique) still fires 15–20 individual concurrent database queries.

We could batch all uncached lookups into a single query using Prisma's findMany:

const repos = await prisma.repo.findMany({
    where: {
        id: { in: uncachedNumericIds }
    }
});

This guarantees that regardless of how many repositories a chunk touches (1 or 20), exactly one single batch query is dispatched to PostgreSQL, completely preventing connection pool contention.

[dipeshbabu]

Batched uncached IDs with findMany, keeping the scoped Prisma client. Missing rows are cached per stream too. Tests, lint and Docker builds passed.

[brendan-kellam]

thanks for the PR - unfortunately considering that this is on a critical path and do not have the bandwidth atm to review this thoroughly, I'm going to close this for the time being.