Skip to content

Check worker health during output waits - #824

Open
IlyaKhalafi wants to merge 1 commit into
qdrant:mainfrom
IlyaKhalafi:fix/worker-crash-wait
Open

IlyaKhalafi wants to merge 1 commit into
qdrant:mainfrom
IlyaKhalafi:fix/worker-crash-wait

Conversation

@IlyaKhalafi

Copy link
Copy Markdown

This PR fixes delayed error reporting when an embedding worker crashes before returning a result. With this, waits retry in short intervals and report the failed worker without resetting the overall timeout.

Fixes #823

Four regression cases fail on the original code. The focused tests pass on Python 3.10, 3.12 and 3.14. The full suite passes with 203 passed and 13 skipped. The configured type checks and pre-commit checks pass.

@IlyaKhalafi
IlyaKhalafi requested a review from joein as a code owner October 15, 2026 15:54
@coderabbitai

coderabbitai Bot commented Oct 15, 2026

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 29faecd5-19cf-44ab-b70d-8d04d14e1f79
📥 Commits

Reviewing files that changed from the base of the PR and between de502ca and 0730679.

📒 Files selected for processing: 2
  • fastembed/parallel_processor.py
  • tests/test_worker_crash_wait.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

📝 Walkthrough

semi_ordered_map now uses _get_output for both blocking output reads. _get_output polls the output queue in intervals of up to one second, checks worker health after an empty read, and raises Empty when the configured processing timeout expires. New tests cover worker crashes during waits, delayed results, timeout behavior, and process cleanup.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 07306

Worker crashes during output waits can now be reported promptly without extending the configured timeout. No merge-blocking issue was identified.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: checking worker health during output waits.
Description check Passed The description is directly related to the changeset and explains the worker-crash handling, timeout behavior, tests, and validation results.
Linked Issues check Passed Issue #823 is directly linked and remains open. The PR replaces both blocking output reads in semi_ordered_map with _get_output. _get_output polls for at most one second, calls `check_worker_hea…
Out of Scope Changes check Passed The changes stay within issue #823. The production change only adds health checks to output waits and the tests provide regression coverage for the requested crash and timeout behavior. No unrelated p…
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Worker crashes go unnoticed while waiting for a result

1 participant