Skip to content

feat(AzureStorage): support orchestrator-only and activity-only worke…#1375

Draft
stevenhvtran wants to merge 3 commits into
Azure:mainfrom
stevenhvtran:transteven-microsoft-decoupled-orchestrator-activity-workers
Draft

feat(AzureStorage): support orchestrator-only and activity-only worke…#1375
stevenhvtran wants to merge 3 commits into
Azure:mainfrom
stevenhvtran:transteven-microsoft-decoupled-orchestrator-activity-workers

Conversation

@stevenhvtran

Copy link
Copy Markdown

Add a WorkerDispatchMode setting (Both/Orchestrator/Activity) so a worker can run only orchestrations+entities or only activities. Activity mode skips control-queue partition leasing; Orchestrator mode skips the work-item queue. TaskHubWorker starts only the enabled dispatchers via the existing dispatcher-count capabilities. Default Both preserves current behavior.

…rs via WorkerDispatchMode

Add a WorkerDispatchMode setting (Both/Orchestrator/Activity) so a worker can run only orchestrations+entities or only activities. Activity mode skips control-queue partition leasing; Orchestrator mode skips the work-item queue. TaskHubWorker starts only the enabled dispatchers via the existing dispatcher-count capabilities. Default Both preserves current behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 6d5f4a2b-07b2-4c3e-a250-378aa68aa065
Copilot AI review requested due to automatic review settings July 20, 2026 01:30
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a decoupled “worker dispatch mode” capability for the Azure Storage backend so a worker can be configured to dispatch only orchestrations/entities, only activities, or both—enabling independent scaling of orchestration vs. activity workers on the same task hub.

Changes:

  • Introduces WorkerDispatchMode and a corresponding AzureStorageOrchestrationServiceSettings.WorkerDispatchMode setting (default Both).
  • Makes AzureStorageOrchestrationService disable dispatcher counts and skip control-queue partition leasing for activity-only workers, with additional “safety net” backoff in fetch loops.
  • Updates TaskHubWorker to start only the dispatchers enabled by the backend’s dispatcher counts and adds a new Azurite-based decoupled worker test.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
Test/DurableTask.AzureStorage.Tests/DecoupledWorkerDispatchTests.cs Adds unit + Azurite integration tests for orchestrator-only + activity-only cooperation on the same task hub.
src/DurableTask.Core/TaskHubWorker.cs Gates dispatcher creation/start/stop based on dispatcher counts to support “disabled” dispatch types.
src/DurableTask.AzureStorage/WorkerDispatchMode.cs Defines the new dispatch mode enum (Both/Orchestrator/Activity).
src/DurableTask.AzureStorage/AzureStorageOrchestrationServiceSettings.cs Exposes WorkerDispatchMode setting with default Both.
src/DurableTask.AzureStorage/AzureStorageOrchestrationService.cs Implements mode behavior via dispatcher counts, skips app lease manager in activity-only mode, and adds backoff safety nets.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread test/DurableTask.AzureStorage.Tests/DecoupledWorkerDispatchTests.cs
Comment thread Test/DurableTask.AzureStorage.Tests/DecoupledWorkerDispatchTests.cs Outdated
Comment thread Test/DurableTask.AzureStorage.Tests/DecoupledWorkerDispatchTests.cs Outdated
Comment thread src/DurableTask.Core/TaskHubWorker.cs Outdated
Copilot AI review requested due to automatic review settings July 20, 2026 01:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

Test/DurableTask.AzureStorage.Tests/DecoupledWorkerDispatchTests.cs:24

  • This test file is under Test/ (capital T), but the repository’s solution references the Azure Storage test project under test/DurableTask.AzureStorage.Tests (lowercase). There is no Test/DurableTask.AzureStorage.Tests/*.csproj, so this test will not be compiled or executed in CI, meaning it won’t validate the new dispatch-mode behavior.
namespace DurableTask.AzureStorage.Tests
{
    using DurableTask.AzureStorage;
    using DurableTask.Core;
    using Microsoft.VisualStudio.TestTools.UnitTesting;
    using System;
    using System.Linq;
    using System.Threading.Tasks;

    [TestClass]
    public class DecoupledWorkerDispatchTests

src/DurableTask.Core/TaskHubWorker.cs:333

  • TaskHubWorker now conditionally creates the orchestration/activity dispatchers based on dispatcher counts. This means TaskOrchestrationDispatcher / TaskActivityDispatcher can remain null even after StartAsync(), which is a behavioral change in the public API surface and can cause NullReferenceExceptions for consumers that configure dispatcher counts to 0 but still access these properties (e.g., to set IncludeDetails). Since WorkItemDispatcher already supports DispatcherCount = 0 (it simply starts zero loops), consider always constructing these dispatcher objects and only gating their StartAsync() calls.
                if (this.dispatchOrchestrations)
                {
                    this.orchestrationDispatcher = new TaskOrchestrationDispatcher(
                        this.orchestrationService,
                        this.orchestrationManager,
                        this.orchestrationDispatchPipeline,
                        this.logHelper,
                        this.ErrorPropagationMode,
                        this.versioningSettings,
                        this.ExceptionPropertiesProvider);
                }

                if (this.dispatchActivities)
                {
                    this.activityDispatcher = new TaskActivityDispatcher(
                        this.orchestrationService,
                        this.activityManager,
                        this.activityDispatchPipeline,
                        this.logHelper,
                        this.ErrorPropagationMode,
                        this.ExceptionPropertiesProvider);
                }

Comment thread test/DurableTask.AzureStorage.Tests/DecoupledWorkerDispatchTests.cs
- Move DecoupledWorkerDispatchTests.cs from Test/ to test/ so the solution's test project compiles and runs it in CI.
- Use TestHelpers.GetTestStorageAccountConnectionString() and GetTestTaskHubName() instead of a hard-coded UseDevelopmentStorage=true connection string and a fixed task hub name, honoring existing DurableTaskTest* configuration.
- TaskHubWorker: always construct the orchestration/activity dispatchers and only gate their StartAsync() calls, so the public dispatcher properties stay non-null for callers that configure them post-start (e.g. IncludeDetails); a disabled dispatcher (count 0) still starts zero fetch loops.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 6d5f4a2b-07b2-4c3e-a250-378aa68aa065
Copilot AI review requested due to automatic review settings July 20, 2026 01:55
@stevenhvtran
stevenhvtran force-pushed the transteven-microsoft-decoupled-orchestrator-activity-workers branch from bb02231 to 8e6740d Compare July 20, 2026 01:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comment thread src/DurableTask.Core/TaskHubWorker.cs Outdated
…aching in ctor

Addresses review feedback: TaskHubWorker previously cached TaskOrchestrationDispatcherCount/TaskActivityDispatcherCount in the constructor, which could go stale if a backend derives those counts from mutable settings changed between construction and StartAsync. The counts are now read inside StartAsync just before starting the dispatchers, so a non-zero count is always honored. Restores the constructor to its original form.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 6d5f4a2b-07b2-4c3e-a250-378aa68aa065
Copilot AI review requested due to automatic review settings July 20, 2026 03:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comment on lines +497 to +501
// Symmetric with StartAsync: activity-only workers never started the partition manager.
if (this.settings.WorkerDispatchMode != WorkerDispatchMode.Activity)
{
await this.appLeaseManager.StopAsync();
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants