fix(testing): deliver due waits on checkpoint#588
Open
yaythomas wants to merge 1 commit into
Open
Conversation
yaythomas
marked this pull request as draft
July 27, 2026 08:27
Complete due waits and step retries at checkpoint time so the completion is returned in the same checkpoint response delta and a suspended parallel branch resumes within the running invocation. The real service resumes such branches inline in a single invocation; the local runner previously waited for the invocation to return and took an extra invocation. - Add Execution.complete_due_operations holding the shared due predicate and transitions for waits and step retries - Pick up due operations in CheckpointCore.apply before the response delta is computed, covering both checkpoint entry points - Delegate Executor._fire_due_operations to the shared method - Add e2e tests: a wait in a parallel branch completes inline in one invocation (skip and wall clock), and a wait completed while suspended is delivered as new on the next invocation - Add unit tests for the due-operation walk and for checkpoint-time delivery of a due wait Closes #587
yaythomas
force-pushed
the
fix/testing-arm-timer-inprocess-checkpoint
branch
from
July 27, 2026 09:02
4395fe0 to
2d930bb
Compare
yaythomas
marked this pull request as ready for review
July 27, 2026 09:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In
DurableFunctionTestRunner, a map/parallel branch that waits cannot resume while a sibling branch keeps working. The runner takes an extra invocation that the real service does not take: verified against Lambda in us-west-2, the service resumes the waiting branch inline in a single invocation while the sibling is still running.Closes #587
Fix
Complete due waits and step retries at checkpoint time, inside
CheckpointCore.apply, immediately before the response delta is computed. Each completion touches its operation, so the transition lands in the same checkpoint response and the suspended branch resumes within the running invocation. Between invocations, the existing earliest-pending timer remains the trigger, unchanged.Execution.complete_due_operationsholds the shared due predicate and transitions (waitSTARTEDwithscheduled_end_timestamp <= now, stepPENDINGwithnext_attempt_timestamp <= now)CheckpointCore.applypicks up due operations before pinning the paginator, covering both checkpoint entry points identicallyExecutor._fire_due_operationsdelegates to the shared method instead of duplicating the walkTests
wait_inline_completion_test.py)on_operation_endfiring once withis_replayed=False(wait_suspend_replay_test.py, guards the contract previously only covered by the examples plugin/otel tests)execution_wait_retry_test.py)processor_test.py)Verification
hatch run dev-testing:test1362 passedhatch run dev-testing:typecheckcleanhatch run dev-core:test1457 passedhatch run dev-examples:pytest --runner-mode=local85 passed, 2 skipped (pre-existing cloud/X-Ray skips), includingtest_pluginandtest_plugin_on_operation_end_called_for_wait_completed_during_suspendhatch fmt --checkclean