Skip to content

feat: add Microservices Bulkhead pattern (#3228) - #3597

Open
ylcn91 wants to merge 1 commit into
iluwatar:masterfrom
ylcn91:feat/microservices-bulkhead
Open

feat: add Microservices Bulkhead pattern (#3228)#3597
ylcn91 wants to merge 1 commit into
iluwatar:masterfrom
ylcn91:feat/microservices-bulkhead

Conversation

@ylcn91

@ylcn91 ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds the Microservices Bulkhead pattern as a new microservices-bulkhead module.

  • Problem: when every downstream call shares one thread pool, a single slow or hanging dependency (here a payment provider) consumes all threads and healthy dependencies (inventory) start failing although nothing is wrong with them.
  • Solution: each downstream dependency gets its own Bulkhead, a dedicated fixed-size thread pool with a bounded queue. When the compartment is full the call fails fast with BulkheadFullException instead of blocking or borrowing threads from other compartments.
  • Key components:
    • Bulkhead: named, bounded ThreadPoolExecutor (threads + queue), fail-fast rejection, metrics (active, queued, rejected), AutoCloseable.
    • BulkheadFullException: unchecked exception carrying the compartment name.
    • RemoteService, PaymentService (slow), InventoryService (healthy): simulated downstream dependencies.
    • App: runs the same load first through one shared pool (inventory call gets rejected) and then through dedicated bulkheads (payment overflow rejected fast, inventory keeps answering), with log output tracing every step.
    • README.md: intent, real-world example, sequence diagram, code walkthrough, applicability, trade-offs, related patterns. PlantUML class diagram under etc/.
  • Tests: 12 JUnit 5 tests (latch-based, no sleeps) covering execution, rejection when threads and queue are full, capacity release, failure propagation, thread naming, shutdown, configuration validation, plus AppTest.
  • Module registered in the parent pom.xml. ./mvnw clean verify -pl microservices-bulkhead passes locally on JDK 21 and inside an eclipse-temurin:21 container.

Fixes #3228

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

PR Summary

Introduces a new microservices-bulkhead module implementing the Bulkhead pattern with per-downstream thread pools, fail-fast behavior, and testing. Includes App demo, Bulkhead core, services, tests, UML, and updated parent POM.

Changes

File Summary
microservices-bulkhead/README.md Documents the Bulkhead pattern implementation in a dedicated 'microservices-bulkhead' module. Explains intent, architecture, components (Bulkhead, BulkheadFullException), scenario demonstrations with dedicated compartments, and trade-offs. Includes a code walkthrough, UML diagram reference, PlantUML diagram, and guidance on when to apply this pattern.
microservices-bulkhead/etc/microservices-bulkhead.urm.puml Provides the PlantUML class diagram for the microservices bulkhead example, including Bulkhead, BulkheadFullException, RemoteService, InventoryService, PaymentService, and App.
microservices-bulkhead/pom.xml Defines the Maven module microservices-bulkhead with dependencies (slf4j-api, logback-classic, junit-jupiter-engine), the main class, and build plugin config for assembly. Ensures the module is included in the root pom. Provides test dependency to run unit tests.
microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java Demonstrates two scenarios: (1) a shared pool for all downstream calls that leads to rejection when saturated; (2) dedicated bulkheads per downstream dependency where one can still serve requests while another is saturated. Uses Bulkhead, RemoteService, and concrete services to log outcomes.
microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java Implements a per-downstream Bulkhead with a fixed-size thread pool and bounded queue. Submits tasks, auto-names threads, and uses BulkheadFullException for fast rejection when full. Tracks active, queued, and rejected counts; supports shutdown via close.
microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BulkheadFullException.java Runtime exception carrying the bulkhead name to signal a full capacity event when a call cannot be accepted.
microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java Implements RemoteService to simulate an inventory system that processes requests instantly and returns an inventory reservation message.
microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java Implements RemoteService to simulate a slow payment provider, sleeping for a configured latency; logs the received request and latency; returns a payment approval string.
microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/RemoteService.java Functional interface for downstream services with a call(String) method; used to model remote service calls routed through Bulkheads.
microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/AppTest.java JUnit tests exercising App: launching app, instantiation, inventory call failure, bulkhead rejection when full, interrupt handling, and awaited results; verifies bulkhead behavior across scenarios.
microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/BulkheadTest.java JUnit tests covering core bulkhead behavior: capacity handling, rejection counts, thread naming, shutdown behavior, invalid configs, and propagation of failures through Future results.
microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/InventoryServiceTest.java Tests that InventoryService immediately reserves inventory when called.
microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/PaymentServiceTest.java Tests that PaymentService approves a payment after latency and preserves interrupt status when interrupted.
microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/RemoteService.java Functional interface for downstream services with a call(String) method; used to model remote service calls routed through Bulkheads.

autogenerated by presubmit.ai

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 Pull request needs attention.

Review Summary

Commits Considered (1)
Files Processed (14)
  • microservices-bulkhead/README.md (1 hunk)
  • microservices-bulkhead/etc/microservices-bulkhead.urm.puml (1 hunk)
  • microservices-bulkhead/pom.xml (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BulkheadFullException.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/RemoteService.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/AppTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/BulkheadTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/InventoryServiceTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/PaymentServiceTest.java (1 hunk)
  • pom.xml (1 hunk)
Actionable Comments (2)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java [64-69]

    readability: "Logger naming consistency with Lombok"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java [105-110]

    readability: "Logger naming inconsistency in Bulkhead"

Skipped Comments (0)

Comment on lines +64 to +69
LOGGER.info("--- Scenario 1: one shared thread pool for every downstream call ---");
try (var sharedPool = new Bulkhead("shared-pool", 2, 2)) {
var paymentFutures = flood(sharedPool, payment, "order", 4);
callInventory(sharedPool, inventory, "order-5");
awaitAll(paymentFutures);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code uses LOGGER with Lombok @slf4j. Lombok typically exposes 'log' as the logger field. If LOGGER is not provided by Lombok in this project, update to the standard 'log' name or align with the project's logging convention to avoid potential compilation issues.

Comment on lines +105 to +110
LOGGER.debug(
"Bulkhead '{}' accepted call ({} active, {} queued)",
name,
executor.getActiveCount(),
executor.getQueue().size());
return future;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logger usage mirrors App.java: verify that the Lombok-provided logger is named LOGGER in this codebase. If Lombok exposes the field as 'log' by default, switch to log or configure Lombok to generate LOGGER to avoid compilation errors.

@ylcn91

ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Note on the automated review comments: LOGGER is the Lombok logger field name configured for this repository in lombok.config (lombok.log.fieldName = LOGGER), the same name every other module uses, so the code compiles as is. Local ./mvnw clean verify -pl microservices-bulkhead passes on JDK 21, also inside an eclipse-temurin:21 container.

@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.95%. Comparing base (41625d8) to head (4cd5d56).

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3597      +/-   ##
============================================
+ Coverage     83.79%   83.95%   +0.16%     
- Complexity     4277     4311      +34     
============================================
  Files          1121     1126       +5     
  Lines         15144    15253     +109     
  Branches        723      728       +5     
============================================
+ Hits          12690    12806     +116     
+ Misses         2159     2153       -6     
+ Partials        295      294       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ylcn91
ylcn91 force-pushed the feat/microservices-bulkhead branch from 94d0736 to 3ada5e2 Compare September 3, 2026 09:52
@ylcn91

ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up on the Codecov note: added five AppTest cases that exercise the failure, rejection and interruption branches of the demo helpers (callInventory, awaitAll), which are now package-private so they can be called from the test. 17 tests; the only uncovered line left in the module is the implicit App constructor. ./mvnw clean verify -pl microservices-bulkhead passes locally on JDK 21 and in an eclipse-temurin:21 container.

@ylcn91
ylcn91 force-pushed the feat/microservices-bulkhead branch from 3ada5e2 to 9532148 Compare September 3, 2026 10:11
@ylcn91

ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

The last CI run failed in BulkheadTest.shouldAcceptCallsAgainAfterCapacityIsReleased: with a queue-less bulkhead (SynchronousQueue) the completed Future does not guarantee that the single worker thread is already back polling the queue, so a submit issued in that window is rejected. The test now uses a bulkhead with a queue of one, drains it, and only then submits again, which is deterministic. Production code unchanged. Verified with 12 consecutive local runs, ./mvnw clean verify -pl microservices-bulkhead on JDK 21, and the same build inside an eclipse-temurin:21 container.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 Pull request needs attention.

Review Summary

Commits Considered (1)
Files Processed (14)
  • microservices-bulkhead/README.md (1 hunk)
  • microservices-bulkhead/etc/microservices-bulkhead.urm.puml (1 hunk)
  • microservices-bulkhead/pom.xml (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BulkheadFullException.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/RemoteService.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/AppTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/BulkheadTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/InventoryServiceTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/PaymentServiceTest.java (1 hunk)
  • pom.xml (1 hunk)
Actionable Comments (4)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java [64-64]

    bug: "Logger name mismatch with Lombok"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java [38-38]

    bug: "Logger name mismatch in InventoryService"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java [51-51]

    bug: "Logger name mismatch in PaymentService"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java [105-111]

    bug: "Logger name consistency in Bulkhead"

Skipped Comments (0)

var payment = new PaymentService(PAYMENT_LATENCY);
var inventory = new InventoryService();

LOGGER.info("--- Scenario 1: one shared thread pool for every downstream call ---");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logger field name mismatch with Lombok @slf4j. Lombok generates a logger named 'log' by default, but the code references LOGGER. This will fail to compile. Align to the project's logging convention (usually 'log') and update all references accordingly.


@Override
public String call(String request) {
LOGGER.info("Inventory system received '{}'", request);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logger field name mismatch with Lombok @slf4j. Same issue as App.java; update to use 'log' or ensure Lombok generates the expected field name.


@Override
public String call(String request) {
LOGGER.info("Payment provider received '{}', it will take {} ms", request, latency.toMillis());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logger field name mismatch with Lombok @slf4j. Update to 'log' or standardize on the project's logger field name.

Comment on lines +105 to +111
LOGGER.debug(
"Bulkhead '{}' accepted call ({} active, {} queued)",
name,
executor.getActiveCount(),
executor.getQueue().size());
return future;
} catch (RejectedExecutionException e) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logger usage in Bulkhead relies on a Lombok-generated 'log' field, but the code references LOGGER in a multi-line log call. Align to the Lombok-provided field name across the class to avoid compilation/runtime issues.

@ylcn91
ylcn91 force-pushed the feat/microservices-bulkhead branch from 9532148 to 4cd5d56 Compare September 3, 2026 11:32
@ylcn91

ylcn91 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Coverage follow-up: added AppTest.shouldBeInstantiable for the implicit App constructor, the last uncovered line. The module now reports 100% instruction, branch and line coverage in JaCoCo. Verified with ./mvnw clean verify -pl microservices-bulkhead on JDK 21 locally and in an eclipse-temurin:21 container, plus five consecutive test runs.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 Pull request needs attention.

Review Summary

Commits Considered (1)
Files Processed (14)
  • microservices-bulkhead/README.md (1 hunk)
  • microservices-bulkhead/etc/microservices-bulkhead.urm.puml (1 hunk)
  • microservices-bulkhead/pom.xml (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Bulkhead.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BulkheadFullException.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java (1 hunk)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/RemoteService.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/AppTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/BulkheadTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/InventoryServiceTest.java (1 hunk)
  • microservices-bulkhead/src/test/java/com/iluwatar/bulkhead/PaymentServiceTest.java (1 hunk)
  • pom.xml (1 hunk)
Actionable Comments (1)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java [60-69]

    readability: "Logger field name mismtach with Lombok SLF4J"

Skipped Comments (2)
  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/InventoryService.java [36-39]

    readability: "Logger field naming mismatch in InventoryService"

  • microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/PaymentService.java [50-59]

    readability: "Logger field naming mismatch in PaymentService"

Comment on lines +60 to +69
public static void main(String[] args) {
var payment = new PaymentService(PAYMENT_LATENCY);
var inventory = new InventoryService();

LOGGER.info("--- Scenario 1: one shared thread pool for every downstream call ---");
try (var sharedPool = new Bulkhead("shared-pool", 2, 2)) {
var paymentFutures = flood(sharedPool, payment, "order", 4);
callInventory(sharedPool, inventory, "order-5");
awaitAll(paymentFutures);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logger naming: Lombok @slf4j generates a logger field (default name: log). The code uses LOGGER, which will not compile unless the logger field is explicitly named LOGGER. Either switch the usage to log or annotate with @slf4j(topic = "LOGGER") to rename the field. This will apply to all references in the App class where LOGGER.info is used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Microservices Bulkhead pattern

1 participant