improvement(knowledge): page search results, take a custom date window, and rerank a person's search by waleedlatif1 · Pull Request #8060 · simstudioai/sim

waleedlatif1 · 2026-09-20T17:54:37Z

Summary

Closes the remaining experience gaps between the Search page and Glean/Onyx, on top of the filters fix (#8059):

Paging. The search asks for 20 chunks at first paint, as today; Show more runs a second, cached 50-chunk search under its own query key instead of widening the first request. Measured on a production-shaped copy: asking for 50 up front raises the keyword leg of a common term from 46–99 ms to 174–270 ms server time for a broad reader, and from 48–213 ms to 222–466 ms for a reader who reaches 3% of the index — so it is paid only when asked for
Custom date window. updated gains a custom range (from/to days in the URL), sent as an inclusive modifiedAfter/modifiedBefore window. modifiedBefore joins the filter contract (with a cross-field check), the filter intersection, the document conditions, the on-row date test, the filtered-set estimate and the bounded probe
Reranking. The dashboard search opts into the platform's cross-encoder reranker whenever a Cohere key is configured, with the default model; reranking stays best-effort, so a provider outage leaves the fused order in place
A filter-first probe (index-driven, bounded by the document limit) gets its own 1.5 s budget: a window at the limit enumerates in ~220 ms and ranks exactly in ~600 ms on the copy, a predictable path where a walk through a mostly-excluded neighbourhood is not
A walk that still has to ask the document per tuple (tag or date filter) keeps the default 20k scan instead of the on-row 100k cap, so a mostly-excluded neighbourhood ends in a short answer rather than a missed deadline

Not in this PR: an author filter (authors live in per-connector document tag slots with no index; it needs an index and the same estimate-then-probe treatment), and mirroring the document date onto the ranking rows so a date filter never asks the document at all — the cold-cache cost of wide date windows is the remaining gap and that mirror is its fix.

Type of Change

New feature

Testing

Results component: Show more appears only after a full first page and asks for the wider search; the custom window is sent as an inclusive range; the hook keys the wider search apart from the first paint
Route: the dashboard opts into the reranker when a key is configured. Contract and filter intersection tests for modifiedBefore. Retrieval tests: the filter-first probe budget, and the default scan for a walk carrying a document lookup. Each verified to fail with its fix reverted
Lint, block registry, check:audits, docs manifest pass; type-check clean; 2,175 UI/search tests pass

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-09-20T17:54:42Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Sep 20, 2026 6:53pm UTC

cubic-dev-ai

All reported issues were addressed across 16 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

greptile-apps · 2026-09-20T17:57:54Z

Confidence Score: 5/5

The PR appears safe to merge; the latest change resolves the incomplete custom-range usability gap without introducing a new actionable defect.

Summary

Adds wider on-demand knowledge-search paging, inclusive custom date windows, credential-aware reranking, and retrieval-planner adjustments for filtered searches.

Keeps the initial search at 20 results and performs expansion as a separately cached 50-result query.
Propagates the custom upper date bound through validation, filter intersection, estimation, and keyword/vector retrieval.
Enables best-effort dashboard reranking when either workspace or platform Cohere credentials are available.
Gives index-driven filtered probes a dedicated budget and limits graph traversal when document lookups remain necessary.
The latest change ensures an incomplete custom range displays the date picker instead of leaving the user unable to finish the range.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Reader submits knowledge search] --> B{Custom range complete?}
    B -- No --> C[Show filters and wait for both dates]
    B -- Yes --> D[Run initial 20-result search]
    D --> E[Apply source and inclusive date filters]
    E --> F{Filtered set can be probed exactly?}
    F -- Yes --> G[Run bounded filter-first probe]
    F -- No --> H[Run constrained keyword/vector retrieval]
    G --> I[Fuse candidates]
    H --> I
    I --> J{Reranker credential available?}
    J -- Yes --> K[Best-effort cross-encoder reranking]
    J -- No --> L[Keep fused ordering]
    K --> M[Render results]
    L --> M
    M --> N{Reader selects Show more?}
    N -- Yes --> O[Run separately cached 50-result search]

_{Reviews (8) · Last reviewed commit: "fix(knowledge): show the filters while a..."}

waleedlatif1 · 2026-09-20T18:11:14Z

@greptile

waleedlatif1 · 2026-09-20T18:11:16Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-20T18:11:21Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

Review completed against the latest diff

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

waleedlatif1 · 2026-09-20T18:26:28Z

@greptile

waleedlatif1 · 2026-09-20T18:26:29Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-20T18:26:38Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

waleedlatif1 · 2026-09-20T18:27:07Z

@greptile

waleedlatif1 · 2026-09-20T18:27:08Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-20T18:27:14Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 21 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

waleedlatif1 · 2026-09-20T18:32:07Z

@greptile

waleedlatif1 · 2026-09-20T18:32:08Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-20T18:32:15Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

…w, and rerank a person's search - the Search page asks for 50 chunks, collapses them to one card per document, shows ten at a time and reveals more on request - `updated` gains a custom range: `from`/`to` days in the URL, sent as an inclusive `modifiedAfter`/`modifiedBefore` window; `modifiedBefore` joins the filter contract, the filter intersection, the document conditions, the on-row date test, the filtered-set estimate and the bounded probe - the dashboard search opts into the platform's cross-encoder reranker whenever a Cohere key is configured; reranking stays best-effort

…der asks for more

…x-driven read can use

…e document per tuple

… and start a refined search at its first page - the filters schema stays a plain object so the Assistant's search input can still extend it; the window's order is checked on the request body - the dashboard asks for reranking outright, and the use case reranks only when the workspace or the platform holds a key - a refinement of the filters starts over at the first page after the reader asked for more - a custom window's days are the reader's local days

…y, and let a custom window be cleared - the credential check tries the platform key before a workspace lookup, and counts a caller's own key only off hosted Sim, as the resolver does - a custom window with no days yet searches nothing and says so; another window drops the days; the picker's Clear reaches the URL through a forwarded `onClear`

…next flush

waleedlatif1 · 2026-09-20T18:34:16Z

@greptile

waleedlatif1 · 2026-09-20T18:34:17Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-20T18:34:22Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 21 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

…n range, and keep the default scan while rows are unfilled - a custom window searches only once both days are chosen; the picker shows the chosen range instead of a fixed label - while the projection still holds rows the backfill has not filled, an on-row walk keeps the default scan cap, since an unfilled row is decided through its document; the answer is read off the unfilled-rows index and remembered for a minute

waleedlatif1 · 2026-09-20T18:43:39Z

@greptile

waleedlatif1 · 2026-09-20T18:43:40Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-20T18:43:46Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 21 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

…days

waleedlatif1 · 2026-09-20T18:53:02Z

@greptile

waleedlatif1 · 2026-09-20T18:53:03Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-20T18:53:09Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 21 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

vercel Bot temporarily deployed to Preview September 20, 2026 17:54 Inactive

cubic-dev-ai Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread apps/sim/lib/api/contracts/knowledge/search.ts Outdated

Comment thread ...orkspace/[workspaceId]/home/components/knowledge-search-results/knowledge-search-results.tsx Outdated

greptile-apps Bot reviewed Sep 20, 2026

View reviewed changes

vercel Bot temporarily deployed to Preview September 20, 2026 18:11 Inactive

greptile-apps Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread apps/sim/lib/knowledge/reranker.ts Outdated

waleedlatif1 force-pushed the improvement/search-filters-on-row branch from 318f2eb to 1b59f04 Compare September 20, 2026 18:13

waleedlatif1 force-pushed the improvement/search-experience-parity branch from c56e22f to 4d3e75d Compare September 20, 2026 18:15

vercel Bot temporarily deployed to Preview September 20, 2026 18:15 Inactive

cubic-dev-ai Bot reviewed Sep 20, 2026

View reviewed changes

vercel Bot temporarily deployed to Preview September 20, 2026 18:27 Inactive

greptile-apps Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread ...orkspace/[workspaceId]/home/components/knowledge-search-results/knowledge-search-results.tsx Outdated

cubic-dev-ai Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread ...orkspace/[workspaceId]/home/components/knowledge-search-results/knowledge-search-results.tsx Outdated

waleedlatif1 changed the base branch from improvement/search-filters-on-row to staging September 20, 2026 18:31

waleedlatif1 added 3 commits September 20, 2026 11:32

improvement(knowledge): widen a search to 50 chunks only when the rea…

414fdc2

…der asks for more

improvement(knowledge): give a filter-first probe the budget its inde…

d30b790

…x-driven read can use

waleedlatif1 added 5 commits September 20, 2026 11:32

improvement(knowledge): keep the default scan for a walk that asks th…

86ed6e1

…e document per tuple

test(knowledge): assert only the dropped days, not the URL adapter's …

6e4a1fb

…next flush

chore(knowledge): narrow the filters once before estimating them

3d5cb80

waleedlatif1 force-pushed the improvement/search-experience-parity branch from 368f1d3 to 3d5cb80 Compare September 20, 2026 18:34

vercel Bot deployed to Preview September 20, 2026 18:36 View deployment

cubic-dev-ai Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread apps/sim/lib/knowledge/search/queries.ts Outdated

vercel Bot temporarily deployed to Preview September 20, 2026 18:43 Inactive

cubic-dev-ai Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread apps/sim/lib/knowledge/search/queries.ts

Comment thread ...orkspace/[workspaceId]/home/components/knowledge-search-results/knowledge-search-results.tsx

fix(knowledge): show the filters while a custom window waits for its …

e2e9093

…days

vercel Bot temporarily deployed to Preview September 20, 2026 18:53 Inactive

cubic-dev-ai Bot reviewed Sep 20, 2026

View reviewed changes

waleedlatif1 merged commit ea31ca7 into staging Sep 20, 2026
37 checks passed

waleedlatif1 mentioned this pull request Sep 21, 2026

v0.8.48: prewarm, tin pg extensions, knowledge search speedups, ui consolidation, keyword search speedups #8096

Merged

waleedlatif1 deleted the improvement/search-experience-parity branch September 21, 2026 06:47

Conversation

waleedlatif1 commented Sep 20, 2026

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Sep 20, 2026

Uh oh!

waleedlatif1 commented Sep 20, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 20, 2026

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Sep 20, 2026

Uh oh!

waleedlatif1 commented Sep 20, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 20, 2026

Uh oh!

waleedlatif1 commented Sep 20, 2026

Uh oh!

waleedlatif1 commented Sep 20, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 20, 2026

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

waleedlatif1 commented Sep 20, 2026

Uh oh!

waleedlatif1 commented Sep 20, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 20, 2026

Uh oh!

waleedlatif1 commented Sep 20, 2026

Uh oh!

waleedlatif1 commented Sep 20, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 20, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

waleedlatif1 commented Sep 20, 2026

Uh oh!

waleedlatif1 commented Sep 20, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 20, 2026

vercel Bot commented Sep 20, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

greptile-apps Bot commented Sep 20, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading