Dialogue indexing currently groups the transcript into fixed sets of five words and searches those groups using semantic embeddings. This creates unnatural fragments and can split a quote, name, or complete thought across multiple entries.
Whisper already returns the recognized words with their start and end times. VidXP should store that timed transcript once, then build useful searchable segments from it. Every search match must still map back to the exact transcript text and video time range.
What should change:
- Keep the recognized word text and start/end time as the shared transcript source.
- Retain the current five-word grouping as a baseline for comparison.
- Compare overlapping windows and natural sentence or punctuation boundaries.
- Add exact/keyword matching alongside semantic dialogue search.
- Return the matched transcript text and correct video start/end time in evidence.
- Rebuilding the same transcript with the same settings should produce the same segment IDs.
- Treat a change in segmentation settings as a new index version rather than silently changing existing results.
Speaker labels can be added to this transcript later through #86, but diarization is not part of this issue. Use #76 to compare retrieval quality.
Dialogue indexing currently groups the transcript into fixed sets of five words and searches those groups using semantic embeddings. This creates unnatural fragments and can split a quote, name, or complete thought across multiple entries.
Whisper already returns the recognized words with their start and end times. VidXP should store that timed transcript once, then build useful searchable segments from it. Every search match must still map back to the exact transcript text and video time range.
What should change:
Speaker labels can be added to this transcript later through #86, but diarization is not part of this issue. Use #76 to compare retrieval quality.