Skip to content

Retry timeout before model cascade fallback #60

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
morgaesis merged 3 commits into from
Jul 12, 2026
Merged

Retry timeout before model cascade fallback #60

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
defdf85
Add timeout retries before model cascade fallback
morgaesis Jul 12, 2026
56bab76
Fix clippy too-many-arguments lint
morgaesis Jul 12, 2026
25b4c82
Clarify README timeout retry feature and rationale
morgaesis Jul 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,11 +190,7 @@ REVIEW_MODEL=<served-model-name> \
postil review --staged --output json
```

Hosted remote reviews use a 540-second total LLM budget so model cascades and
scoring finish inside the worker watchdog. Local reviews do not use a total
budget unless `POSTIL_LLM_TOTAL_TIMEOUT_SECS` is set. Each model request still
uses `POSTIL_LLM_REQUEST_TIMEOUT_SECS`, which defaults to 480 seconds locally
and 420 seconds for hosted remote reviews.
Hosted remote reviews use a 240-second request timeout with a single timeout retry capped at 90 seconds, reducing unnecessary fallback to weaker models when the primary model is slow but working. The entire review model phase is capped at 420 seconds, with the remaining 120 seconds of the 540-second total LLM budget reserved for scoring inside the worker watchdog. A timeout triggers one automatic retry at the same model level before cascading to the next model. Local reviews default to a 480-second request timeout and do not use a total deadline unless `POSTIL_LLM_TOTAL_TIMEOUT_SECS` is set. Exhausting a review or total deadline is terminal.

Use the live benchmark harness before standardizing on a model:

Expand Down
Loading
Loading