Skip to content

improvement(search): reduce redundant metadata reads - #7728

Merged
waleedlatif1 merged 1 commit into
stagingfrom
codex/search-retrieval-reuse
Sep 10, 2026
Merged

waleedlatif1 merged 1 commit into
stagingfrom
codex/search-retrieval-reuse

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Batch knowledge-base references and tag definitions during search, reducing those reads from 40 statements to 2 for 20 bases while preserving ordering, missing entries, and fresh authorization.
  • Skip GitHub connector discovery for explicitly filtered non-GitHub searches. Keep default/GitHub discovery, ranking, and live access checks unchanged.

Type of Change

  • Performance improvement

Testing

  • 820 unit tests and 62 PostgreSQL integration tests passed, covering ACLs, source/date filters, GitHub access, organization MCP search, and batching parity.
  • Local synthetic benchmark: 480 paired scenario groups; 20-base reference/tag p50 improved from 7.67 ms to 1.91 ms. This measures preparation only; single-base SQL is unchanged.
  • Lint, repository audits, block-registry validation, and docs manifest validation passed. The batching regression is included in the existing PostgreSQL CI job.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Sep 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Sep 10, 2026 10:26am UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with batching and discovery optimizations preserving the existing authorization and result semantics.

Summary

  • Preserves requested reference ordering, duplicates, missing entries, and soft-deletion behavior.
  • Reuses batched tag definitions for filter resolution and result metadata naming.
  • Retains live GitHub authorization discovery for unfiltered and explicitly GitHub searches.
  • Adds unit and PostgreSQL integration coverage for batching parity, freshness, authorization, and source filtering.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Knowledge search request] --> B[Validate up to 20 knowledge bases]
    B --> C[Batch-load active references]
    C --> D[Authorize canonical workspace and bases]
    D --> E[Batch-load tag definitions]
    E --> F[Resolve tag filters]
    F --> G[Select ranked candidates]
    G --> H{Explicit non-GitHub source?}
    H -- Yes --> I[Skip GitHub connector discovery]
    H -- No --> J[Discover live GitHub grants]
    I --> K[Hydrate with current ACL checks]
    J --> K
    K --> L[Name metadata using batched definitions]
Loading

Reviews (1) · Last reviewed commit: "improvement(search): reduce redundant me..."

@waleedlatif1
waleedlatif1 merged commit 6dba116 into staging Sep 10, 2026
33 checks passed
@waleedlatif1
waleedlatif1 deleted the codex/search-retrieval-reuse branch September 10, 2026 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant