[Feature Request]: Add Scatter-Gather Design Pattern

[priyanshuvishwakarma273403]

Name of the Design Pattern

Scatter-Gather Pattern

Category

Concurrency / Integration Patterns

Intent

The Scatter-Gather design pattern is a messaging and integration pattern used to parallelize task execution. It broadcasts (scatters) a single request to multiple recipients or worker services concurrently, and then aggregates (gathers) their responses back into a single response. This significantly reduces overall response time when fetching data from multiple independent sources.

Detailed Explanation

In enterprise architectures, a client request often requires gathering information from multiple downstream services or running multiple independent calculations. Running these calls sequentially can lead to high latency.

The Scatter-Gather pattern addresses this by:

1. Scatter: Spawning multiple parallel threads or asynchronous tasks (workers) to call each downstream service or execute task partitions.
2. Gather: Waiting for all tasks to finish (or timing out if some are slow), collecting the results, and passing them to an aggregator.
3. Aggregate: Merging individual responses into a unified result format returned to the client.

Real-World Example

Consider a travel booking aggregator (like Kayak or Expedia). When a user searches for a flight, the portal scatters the query to multiple airlines' API servers simultaneously. It gathers the prices, sorts them, and displays the complete list to the user.

Conceptual Architecture

Here is a conceptual design for the Scatter-Gather pattern:

classDiagram
    direction TB
    class TaskSupplier {
        <<interface>>
        +execute() String
    }
    class AirlineService {
        -String airlineName
        -double basePrice
        +execute() String
    }
    class ScatterGatherExecutor {
        -ExecutorService executor
        +scatterGather(List~TaskSupplier~ tasks, long timeoutMs) List~String~
    }
    class Aggregator {
        +aggregate(List~String~ results) String
    }
    
    TaskSupplier <|.. AirlineService
    ScatterGatherExecutor --> TaskSupplier : scatters tasks
    ScatterGatherExecutor .. Aggregator : passes results to

Loading

[bonigo1998]

Hi, I'd like to work on this issue.
I'm planning to implement the Scatter-Gather pattern using a TaskSupplier interface, example AirlineService implementations, a ScatterGatherExecutor backed by ExecutorService, and an Aggregator to combine the results.
I’ll also add tests and documentation following the repository’s existing pattern structure and contribution guidelines. Please let me know if you recommend any changes to this approach before I begin.

[bonigo1998]

I noticed the existing fanout-fanin pattern lists Scatter-Gather as an alternate name. For #3577, should the new implementation specifically demonstrate the integration-style variant described in the issue—broadcasting one request to multiple independent services, supporting timeout/partial responses, and aggregating their responses—or would you prefer changes to the existing Fan-Out/Fan-In module instead?

[sohamcodes-ctrl]

Hi @iluwatar, I’d like to work on this issue as my first open-source contribution.

I’ve reviewed the proposed Scatter-Gather pattern and understand the implementation requirements, including parallel task execution, gathering results, and aggregating them into a unified response.

I’m interested in implementing this following the project’s existing conventions and adding appropriate tests and documentation.

Could you please assign this issue to me? I’d be happy to take ownership of the implementation and submit a PR.

Thank you!

[ylcn91]

Hi @iluwatar, I opened an independent implementation in #3601 (different domain and design from #3590) with a populated, logging App entry point, timeout handling with partial results, and an explicit note on how it differs from fanout-fanin. Happy to align with whichever direction you prefer.