Skip to content

Do not use the final result count as the retrieval candidate limit #88

Description

@tulayha

When a caller asks VidXP for three final results, the same limit is currently used while reading candidates from each underlying search index.

That can hide a strong combined result before ranking happens. For example, a moment ranked fourth in dialogue and fourth in visual search may become the best result when both signals are combined, but VidXP never sees it if each search returns only three candidates.

What should change:

  • Use a separate internal candidate limit for each search channel.
  • Keep that internal pool bounded so searches cannot grow without limit.
  • Continue using top_k only for the number of final results returned to the caller.
  • Preserve the existing CLI, API, and MCP meaning of top_k.
  • Add a regression test where the best combined result falls outside the initial per-channel top_k.
  • Use Add end-to-end retrieval evaluation #76 to measure the recall improvement and added search cost before choosing the default candidate depth.

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is needed

    Type

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions