Skip to content

Fix Atlas scans in text identity and cancellation recovery (JVNAUTOSCI-2727) - #576

Merged
witbrock merged 1 commit into
mainfrom
fix/JVNAUTOSCI-2727-atlas-scans
Sep 7, 2026
Merged

Fix Atlas scans in text identity and cancellation recovery (JVNAUTOSCI-2727)#576
witbrock merged 1 commit into
mainfrom
fix/JVNAUTOSCI-2727-atlas-scans

Conversation

@witbrock

@witbrock witbrock commented Sep 7, 2026

Copy link
Copy Markdown
Member

Merge decision: ready — bounded live execution plans eliminate full-language text identity scans and empty cancellation-queue collection scans, with existing behaviour preserved by 106 targeted tests.

User outcome

Reduce recurring Atlas database work in text creation/reuse and interrupted-cancellation recovery. An English-NZ fingerprint miss previously examined 183,644 documents; an empty cancellation recovery poll examined all 1,074 queue documents.

Material changes

  • Route fingerprint lookups, duplicate-insert reconciliation and text-relation removal through a repository lookup that explicitly implies the existing partial compound index's string predicate.
  • Add a non-unique partial index on status, cancellation request time and ID, retaining the existing recovery filter and ordering.
  • Preserve normalised/exact text identity, language matching and all cancellation decisions.

JVNAUTOSCI-2727 owns delivery status, including the separately attributed obsolete Windows embedding client. The corrected materialised embedding queue query already exists on main.

Evidence

  • 106 targeted tests passed across text fingerprint identity, newline preservation, language-specific relation removal, duplicate recovery, queue behaviour, collection index setup and existing embedding queue semantics.
  • Live read-only executionStats: English-NZ fingerprint miss 183,644 documents / 342 ms → zero documents / 0 ms; English miss 2,251 → zero; both select fingerprint_lang_unique.
  • Canonical additive index setup materialised only status_cancellation_requested_id, preserving every existing index. The unchanged recovery query moved from COLLSCAN plus SORT, 1,074 examined documents, to the new index with zero examined documents and no blocking sort.
  • Current embedding query uses embedding_status_updated_at_desc, examining one key/document for one result.
  • Python 3.11 grammar check (1,440 files), PDM lock check and diff whitespace check passed. Black reports pre-existing formatting outside the changed lines in three files; no unrelated reformatting included.

Ship boundary

  • Minimum ship criteria: preserve text identity and cancellation semantics while proving index-backed execution for the two changed paths; required CI passes.
  • Stop-ship conditions: changed language/identity results, incorrect cancellation, destructive index replacement or no demonstrated scan reduction.
  • Non-blocking observations: historical Atlas Query Shape Insights returns 403, so these cumulative counters do not establish the exact alert's cause. The old embedding query is from Windows and requires that client's update/restart; it is tracked in Jira and is not claimed fixed by this PR.
  • Non-goals: alert suppression, query framework changes, semantic policy changes, unrelated search optimisation or broad remote deployment.

@witbrock
witbrock merged commit cbd05cc into main Sep 7, 2026
2 checks passed
@witbrock
witbrock deleted the fix/JVNAUTOSCI-2727-atlas-scans branch September 7, 2026 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant