fix(knowledge): retry a projection fill page after an index lock or statement timeout instead of ending the chain

[waleedlatif1]

Summary

• A projection fill page that fails with a lock timeout (the keyword index's background maintenance holds its metapage while the page's update needs it) or a statement timeout is now retried in place after a bounded pause (10 s doubling to 60 s with jitter), up to 12 consecutive failures, instead of propagating; before, the Trigger task's run-level retry restarted from the original cursor, re-walked already-filled rows into a statement timeout, and ended the chain after three attempts, which left the keyword projection unfillable while maintenance ran
• The progress cursor only advances on a successful page, so a continuation never rescans filled rows; a page still failing when the run's budget is spent is handed to the continuation from the last committed page rather than retried past the budget
• Any other database error still propagates immediately

Type of Change

• Bug fix

Testing

• packages/db: vitest run script-migrations/0021_embedding_search_connector.test.ts — 13 passed. New cases: a 55P03 page and a 57014 page are each retried and the cursor is bound to the last committed page on every retry; the run fails once a page times out more than the limit in a row; a non-timeout error propagates without retry; a page still failing at the budget returns done: false with the last committed cursor. Verified the four new cases go red with the retry removed
• apps/sim: vitest run lib/knowledge/search/projection-source-acl-backfill.test.ts — 21 passed
• bun run type-check in packages/db and apps/sim, bun run lint at the root, bun run check:migrations origin/staging (no new migrations)

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 22, 2026 1:54pm UTC

[waleedlatif1]

@greptile

[waleedlatif1]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; the previously reported cancellation and run-budget issues are fixed and their threads are resolved.

Summary

This PR makes projection source/ACL backfill pages retry transient lock and statement timeouts without advancing the committed cursor.

• Retries transient page failures with bounded exponential backoff and jitter.
• Distinguishes statement timeouts from explicit PostgreSQL cancellations.
• Rechecks the run deadline after retry pauses before starting another page.
• Adds coverage for cursor preservation, retry limits, cancellation handling, and budget exhaustion.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Run projection page] --> B{Page result}
    B -->|Committed| C[Advance cursor and reset timeout count]
    C --> D{More rows and budget remains?}
    D -->|Yes| A
    D -->|No| E[Return progress]
    B -->|Non-timeout error or explicit cancellation| F[Propagate error]
    B -->|Lock or statement timeout| G[Increment consecutive timeout count]
    G --> H{Retry limit or budget exhausted?}
    H -->|Yes| I[Fail or return continuation cursor]
    H -->|No| J[Sleep with bounded backoff and jitter]
    J --> K{Budget remains after sleep?}
    K -->|Yes| A
    K -->|No| I

Loading

_{Reviews (3) · Last reviewed commit: "fix(knowledge): stop the fill page retry..."}

[cubic-dev-ai]

All reported issues were addressed across 2 files

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[waleedlatif1]

@greptile

[waleedlatif1]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

No issues found across 2 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

_{Re-trigger cubic}