Skip to content

fix: hold an actor's turn until a cancelled call finishes - #1877

Merged
siri-varma merged 1 commit into
dapr:masterfrom
javier-aliaga:fix/actors-a-call-cancelled-while-it-runs-frees-the
Oct 27, 2026
Merged

siri-varma merged 1 commit into
dapr:masterfrom
javier-aliaga:fix/actors-a-call-cancelled-while-it-runs-frees-the

Conversation

@javier-aliaga

@javier-aliaga javier-aliaga commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Description

When a caller cancels an actor call while the method runs, daprd releases the actor and sends the next call for the same actor ID. The SDK kept running the cancelled method, so two calls ran on the same actor at once. Over HTTP the cancelled call also skipped onPostActorMethod, so its state changes were not saved.

ActorManager now runs activation, method calls (timers and reminders included) and deactivation through a new per-actor ActorTurnLock. Calls for one actor run one at a time, in arrival order. Calls for different actors still run in parallel.

  • A call cancelled while it runs keeps the actor until the method and onPostActorMethod finish. Its caller stops waiting right away.
  • A call cancelled while it waits is removed from the queue and never runs.
  • A queued call starts on Schedulers.boundedElastic(), so a long queue does not grow the stack.
  • Each call sees the Reactor context of the subscriber that queued it.

Behavior changes:

  • Concurrent first calls to the same actor now create one instance. Before, each call could create its own.
  • The lock has no timeout and no reentrancy, and the SDK does not configure reentrancy today. A method that never returns blocks every later call to that actor ID. An actor that calls itself waits until the caller's timeout, which is what daprd already does without reentrancy.

Tests:

  • ActorTurnLockTest covers ordering, cancellation, errors, empty results, a 10,000-call queue and context propagation.
  • ActorManagerTest adds three cases. A cancelled call still runs onPostActorMethod before the next call starts. Ten concurrent activations create one actor. Deactivation waits for the running call.
  • ActorCancelledCallIT (HTTP app) and ActorCancelledCallGrpcHostIT (actor event stream) run against daprd. Each cancels five 2-second calls after 500 ms and checks that only one ran at a time. Both fail on master with 4 calls running at once, and pass with this change.

Issue reference

Fixes #1871

Checklist

  • Code compiles correctly
  • Created/updated tests
  • Extended the documentation

When a caller cancels an actor call while it runs, daprd releases the
actor and sends the next call, but the SDK kept running the cancelled
method, so two turns ran at once on the same actor.

ActorManager now runs activation, method calls and deactivation through
a per-actor turn lock. A turn keeps the actor until its code and the
state save finish, even if its request is cancelled. Turns that are
still waiting are dropped when cancelled.

Fixes dapr#1871

Signed-off-by: Javier Aliaga <javier@diagrid.io>
@javier-aliaga
javier-aliaga requested review from a team as code owners October 27, 2026 09:59
Copilot AI lite review requested due to automatic review settings October 27, 2026 09:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@codecov

codecov Bot commented Oct 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.63%. Comparing base (00f769f) to head (a4e1158).

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #1877      +/-   ##
============================================
+ Coverage     80.52%   80.63%   +0.10%     
- Complexity     2835     2850      +15     
============================================
  Files           274      275       +1     
  Lines          8412     8459      +47     
  Branches        905      911       +6     
============================================
+ Hits           6774     6821      +47     
  Misses         1272     1272              
  Partials        366      366              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@siri-varma
siri-varma merged commit 47b9bb5 into dapr:master Oct 27, 2026
23 of 24 checks passed
javier-aliaga added a commit that referenced this pull request Oct 27, 2026
)

When a caller cancels an actor call while it runs, daprd releases the
actor and sends the next call, but the SDK kept running the cancelled
method, so two turns ran at once on the same actor.

ActorManager now runs activation, method calls and deactivation through
a per-actor turn lock. A turn keeps the actor until its code and the
state save finish, even if its request is cancelled. Turns that are
still waiting are dropped when cancelled.

Fixes #1871


(cherry picked from commit 47b9bb5)

Signed-off-by: Javier Aliaga <javier@diagrid.io>
Co-authored-by: Javier Aliaga <javier@diagrid.io>
@javier-aliaga
javier-aliaga deleted the fix/actors-a-call-cancelled-while-it-runs-frees-the branch October 28, 2026 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

3 participants