feat(AzureStorage): support orchestrator-only and activity-only worke…#1375
Draft
stevenhvtran wants to merge 3 commits into
Draft
Conversation
…rs via WorkerDispatchMode Add a WorkerDispatchMode setting (Both/Orchestrator/Activity) so a worker can run only orchestrations+entities or only activities. Activity mode skips control-queue partition leasing; Orchestrator mode skips the work-item queue. TaskHubWorker starts only the enabled dispatchers via the existing dispatcher-count capabilities. Default Both preserves current behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6d5f4a2b-07b2-4c3e-a250-378aa68aa065
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a decoupled “worker dispatch mode” capability for the Azure Storage backend so a worker can be configured to dispatch only orchestrations/entities, only activities, or both—enabling independent scaling of orchestration vs. activity workers on the same task hub.
Changes:
- Introduces
WorkerDispatchModeand a correspondingAzureStorageOrchestrationServiceSettings.WorkerDispatchModesetting (defaultBoth). - Makes
AzureStorageOrchestrationServicedisable dispatcher counts and skip control-queue partition leasing for activity-only workers, with additional “safety net” backoff in fetch loops. - Updates
TaskHubWorkerto start only the dispatchers enabled by the backend’s dispatcher counts and adds a new Azurite-based decoupled worker test.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| Test/DurableTask.AzureStorage.Tests/DecoupledWorkerDispatchTests.cs | Adds unit + Azurite integration tests for orchestrator-only + activity-only cooperation on the same task hub. |
| src/DurableTask.Core/TaskHubWorker.cs | Gates dispatcher creation/start/stop based on dispatcher counts to support “disabled” dispatch types. |
| src/DurableTask.AzureStorage/WorkerDispatchMode.cs | Defines the new dispatch mode enum (Both/Orchestrator/Activity). |
| src/DurableTask.AzureStorage/AzureStorageOrchestrationServiceSettings.cs | Exposes WorkerDispatchMode setting with default Both. |
| src/DurableTask.AzureStorage/AzureStorageOrchestrationService.cs | Implements mode behavior via dispatcher counts, skips app lease manager in activity-only mode, and adds backoff safety nets. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (2)
Test/DurableTask.AzureStorage.Tests/DecoupledWorkerDispatchTests.cs:24
- This test file is under
Test/(capital T), but the repository’s solution references the Azure Storage test project undertest/DurableTask.AzureStorage.Tests(lowercase). There is noTest/DurableTask.AzureStorage.Tests/*.csproj, so this test will not be compiled or executed in CI, meaning it won’t validate the new dispatch-mode behavior.
namespace DurableTask.AzureStorage.Tests
{
using DurableTask.AzureStorage;
using DurableTask.Core;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
using System.Linq;
using System.Threading.Tasks;
[TestClass]
public class DecoupledWorkerDispatchTests
src/DurableTask.Core/TaskHubWorker.cs:333
TaskHubWorkernow conditionally creates the orchestration/activity dispatchers based on dispatcher counts. This meansTaskOrchestrationDispatcher/TaskActivityDispatchercan remainnulleven afterStartAsync(), which is a behavioral change in the public API surface and can causeNullReferenceExceptions for consumers that configure dispatcher counts to 0 but still access these properties (e.g., to setIncludeDetails). SinceWorkItemDispatcheralready supportsDispatcherCount = 0(it simply starts zero loops), consider always constructing these dispatcher objects and only gating theirStartAsync()calls.
if (this.dispatchOrchestrations)
{
this.orchestrationDispatcher = new TaskOrchestrationDispatcher(
this.orchestrationService,
this.orchestrationManager,
this.orchestrationDispatchPipeline,
this.logHelper,
this.ErrorPropagationMode,
this.versioningSettings,
this.ExceptionPropertiesProvider);
}
if (this.dispatchActivities)
{
this.activityDispatcher = new TaskActivityDispatcher(
this.orchestrationService,
this.activityManager,
this.activityDispatchPipeline,
this.logHelper,
this.ErrorPropagationMode,
this.ExceptionPropertiesProvider);
}
- Move DecoupledWorkerDispatchTests.cs from Test/ to test/ so the solution's test project compiles and runs it in CI. - Use TestHelpers.GetTestStorageAccountConnectionString() and GetTestTaskHubName() instead of a hard-coded UseDevelopmentStorage=true connection string and a fixed task hub name, honoring existing DurableTaskTest* configuration. - TaskHubWorker: always construct the orchestration/activity dispatchers and only gate their StartAsync() calls, so the public dispatcher properties stay non-null for callers that configure them post-start (e.g. IncludeDetails); a disabled dispatcher (count 0) still starts zero fetch loops. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6d5f4a2b-07b2-4c3e-a250-378aa68aa065
stevenhvtran
force-pushed
the
transteven-microsoft-decoupled-orchestrator-activity-workers
branch
from
July 20, 2026 01:55
bb02231 to
8e6740d
Compare
…aching in ctor Addresses review feedback: TaskHubWorker previously cached TaskOrchestrationDispatcherCount/TaskActivityDispatcherCount in the constructor, which could go stale if a backend derives those counts from mutable settings changed between construction and StartAsync. The counts are now read inside StartAsync just before starting the dispatchers, so a non-zero count is always honored. Restores the constructor to its original form. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6d5f4a2b-07b2-4c3e-a250-378aa68aa065
Comment on lines
+497
to
+501
| // Symmetric with StartAsync: activity-only workers never started the partition manager. | ||
| if (this.settings.WorkerDispatchMode != WorkerDispatchMode.Activity) | ||
| { | ||
| await this.appLeaseManager.StopAsync(); | ||
| } |
25 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a WorkerDispatchMode setting (Both/Orchestrator/Activity) so a worker can run only orchestrations+entities or only activities. Activity mode skips control-queue partition leasing; Orchestrator mode skips the work-item queue. TaskHubWorker starts only the enabled dispatchers via the existing dispatcher-count capabilities. Default Both preserves current behavior.