Skip to content

Prevent post-return activity cancel from killing pool thread#1654

Open
tconley1428 wants to merge 2 commits into
mainfrom
fix-thread-cancel-after-sync-activity-return
Open

Prevent post-return activity cancel from killing pool thread#1654
tconley1428 wants to merge 2 commits into
mainfrom
fix-thread-cancel-after-sync-activity-return

Conversation

@tconley1428

@tconley1428 tconley1428 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

What Changed

  • Scope threaded activity cancellation injection to the period where the sync activity callable is actively running on the executor thread.
  • Clear the recorded thread id after the callable returns so later cancellation during interceptor/result-completion work cannot target an idle or reused ThreadPoolExecutor worker.
  • Add a regression test that holds an inbound activity interceptor open after the sync callable returns, waits for start-to-close cancellation, then verifies the one-thread executor can still run a probe activity.

Fixes #1648

Validation

  • uv run pytest tests/worker/test_activity.py::test_sync_activity_cancel_after_return_does_not_kill_thread_pool_worker -q
  • uv run pytest tests/worker/test_activity.py::test_sync_activity_thread tests/worker/test_activity.py::test_sync_activity_contextvars tests/worker/test_activity.py::test_sync_activity_cancel_after_return_does_not_kill_thread_pool_worker -q
  • uv run pyright temporalio/worker/_activity.py tests/worker/test_activity.py
  • uv run ruff check tests/worker/test_activity.py
  • uv run ruff check --ignore E722 temporalio/worker/_activity.py tests/worker/test_activity.py

I also temporarily reverted the implementation change while keeping the regression test. The test failed and logged the expected ThreadPoolExecutor Exception in worker traceback from an injected temporalio.exceptions.CancelledError, then passed again after restoring the fix.

@tconley1428 tconley1428 requested a review from a team as a code owner July 16, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Threaded activity cancellation can kill threads and stall worker

1 participant