Skip to content

Keep run_until polling through matching backend-lock refusals - #219

Merged
rmcdaniel merged 3 commits into
mainfrom
fix/218-run-until-poll-refusals
Oct 24, 2026
Merged

rmcdaniel merged 3 commits into
mainfrom
fix/218-run-until-poll-refusals

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

Worker.run_until() now recovers from a complete, matching typed transient poll refusal within its original caller deadline. Previously that helper exited even when its deadline had time remaining. Workflow capacity is released before backoff, stop interrupts the wait, and malformed/foreign refusals, authorization failures and unsafe admitted operations still raise. Business operations and durable attempts are not retried by this change.

Only HTTP 503 with a null task, matching namespace/queue/task kind and a bounded integer retry delay qualifies. Registration remains inside the original helper budget and shutdown keeps its separate drain policy.

Validation: 163 focused helper/polling/replay controls, Ruff, strict mypy and corpus validation pass. Normally installed 2.11.0 fails both helper recovery controls and the separate cold-history control. The broad local original remains 3513 passed, twelve failures, three errors and two skips; four prepared-group failures also reproduce on unchanged 2.11.0 and remain unexplained. No full local unit pass is claimed. All applicable candidate and independent main hosted checks pass.

Delivered in normally published Python SDK 2.11.1, exact source 3511924, whole 4466d411b8f9b20eebd1d1c4c082b46f546d2b49.

All applicable candidate and independent fresh-main checks pass, including Python 3.10/3.11/3.12, integration, quality, corpus, CodeQL and boundaries. The protected PyPI OIDC publisher 40061568529 completed successfully. The normal PyPI 2.11.1 package is available; wheel SHA-256 is 1e7f0210ef30ae33e2224f6857386a43e7793b725a298a0974b2bd6e53c0318f. All 39 installed production files match the tagged Git source and all 51 RECORD hashes are verified, including five declared console launchers.

Actual installed acceptance: five new originals on unchanged published PHP Server 2.11.1, immutable index bf8bed13f5f809ee3425233180a581353eac4683adbee5c4ecbf63bb18bbd102, pass: healthy default plus workflow-poll and activity-poll pressure separately in default and named namespaces. Each pressure case receives three actual typed HTTP 503 backend_lock_pressure responses from a real SQLite writer lock through the normal Client, with null task and exact namespace/queue/kind/retry delay, then completes the same original. Responses are neither mocked nor rewritten.

Every original retains its 25-second helper budget, one-second poll and 30-second run/execution deadline, with separate ten-second shutdown policy. Completion plus shutdown takes 1.68 seconds healthy and 3.24–3.40 seconds under pressure. Stopped SQL and cold reads prove 5 completed runs, 5 real callbacks, 5 first activity attempts, 15 first-attempt completed tasks, 30 events, zero repairs and zero failed jobs. Exact int64, Unicode, false/null values and history identities survive cold delivery. All five APIs exit 0 without OOM.

The earlier normal-PyPI 2.11.0 counterfactual failures, broad local original and unchanged-baseline diagnostic failures remain recorded. The initial installed-file verifier refused pip --target console paths; the corrected verifier resolves only the five declared console entries and validates every hash, without skipping files or changing the package. No historical native/shared cohort failure is rescued or retupled by this new acceptance.

All owned applications, databases, networks, installed SDK/dependencies, tooling and raw database state are removed; empty task inventories and borrowed-image preservation were verified at 04:39:14 UTC, before the declared 04:50 bound. General's SDK acceptance is complete. Server #721 retains independent cohort scheduling/adoption and every original observation.

Closes #218 after normal publication and installed acceptance.

@sonarqubecloud

Copy link
Copy Markdown

@rmcdaniel
rmcdaniel marked this pull request as ready for review October 24, 2026 01:53
@rmcdaniel
rmcdaniel merged commit 3511924 into main Oct 24, 2026
18 checks passed
@rmcdaniel
rmcdaniel deleted the fix/218-run-until-poll-refusals branch October 24, 2026 02:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Keep run_until alive through typed retryable poll refusals

1 participant