You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Workers that die, are wrongly judged dead, or lose their database or OpenSearch.
Dead workers: a worker whose heartbeat is older than 5 minutes is dead. Any worker or draining CLI run checks for dead workers when it looks for work. A dead worker's job returns to waiting with its attempt count raised, and is marked failed on its third return.
Clean shutdown: a worker shutting down on SIGTERM or SIGINT removes its worker row and releases its job at once.
Claim ownership: a job checks it still owns its claim before writing its result, so a worker wrongly judged dead drops its work.
Outages: during a database or OpenSearch outage, the worker pauses and reconnects rather than failing every waiting job.
Many workers: any number of workers and CLI runs can drain one queue, and each job is claimed once.
Acceptance criteria
A test stages a worker row with an old heartbeat and a running job naming it; working the queue returns the job to waiting and runs it. With the attempt count preset, the third return marks it failed. No fake timers.
A test at the indexer's internal job seam takes a claim away mid-job, and the job's result write is refused.
Two "work the queue" runs at once over several crates index each crate once.
A worker stopped with SIGTERM leaves no worker row and no running job.
The outage behaviour is checked by hand once, with the steps recorded in a comment here. There's no automated test.
Indexer coverage stays at 100%. pnpm test and pnpm lint pass.
Activity