, timeout : Duration) : Response {static}
+ ~ shutdown(executor : ExecutorService, timeout : Duration) : void {static}
+ ~ report(response : Response) : void {static}
+ }
+}
+Response +-- Status
+Request --> Priority
+Response --> Status
+Response ..> Request
+LoadShedException --> Priority
+LoadShedException ..> Request
+LoadShedder ..> Priority
+LoadShedder ..> Request
+LoadShedder ..> LoadShedException
+RequestHandler ..> Request
+ShedGuardedService --> LoadShedder
+ShedGuardedService --> RequestHandler
+ShedGuardedService ..> Request
+ShedGuardedService ..> Response
+ShedGuardedService ..> LoadShedException
+App ..> LoadShedder
+App ..> ShedGuardedService
+App ..> RequestHandler
+@enduml
diff --git a/microservices-load-shedding/pom.xml b/microservices-load-shedding/pom.xml
new file mode 100644
index 000000000000..33582143a0e9
--- /dev/null
+++ b/microservices-load-shedding/pom.xml
@@ -0,0 +1,70 @@
+
+
+
+ 4.0.0
+
+ com.iluwatar
+ java-design-patterns
+ 1.26.0-SNAPSHOT
+
+ microservices-load-shedding
+
+
+ org.slf4j
+ slf4j-api
+
+
+ ch.qos.logback
+ logback-classic
+
+
+ org.junit.jupiter
+ junit-jupiter-engine
+ test
+
+
+
+
+
+ org.apache.maven.plugins
+ maven-assembly-plugin
+
+
+
+
+
+ com.iluwatar.loadshedding.App
+
+
+
+
+
+
+
+
+
diff --git a/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java
new file mode 100644
index 000000000000..3efce7d7c343
--- /dev/null
+++ b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/App.java
@@ -0,0 +1,193 @@
+/*
+ * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
+ *
+ * The MIT License
+ * Copyright © 2014-2022 Ilkka Seppälä
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+package com.iluwatar.loadshedding;
+
+import java.time.Duration;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+import java.util.concurrent.Semaphore;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicBoolean;
+import lombok.extern.slf4j.Slf4j;
+
+/**
+ * Load Shedding is a resilience pattern for services that receive more work than they can handle.
+ * Instead of accepting every request and slowly drowning, the service measures its own load and
+ * proactively rejects excess requests at the door, so that the requests it does accept are served
+ * with normal latency and the process never runs out of threads, memory or connections.
+ *
+ * The key ingredients demonstrated here are:
+ *
+ *
+ * - a capacity limit expressed as the number of requests in flight ({@link LoadShedder}),
+ *
- fail-fast rejection: shed requests receive an immediate {@link Response.Status#REJECTED}
+ * answer instead of waiting in a queue ({@link ShedGuardedService}),
+ *
- priority-aware shedding: low priority work is dropped first and a small reserve is kept for
+ * critical requests ({@link Priority}),
+ *
- metrics that make the shedding decisions observable.
+ *
+ *
+ * The demo runs an order service with capacity for five concurrent requests. Phase one shows
+ * normal operation. In phase two the payment provider becomes slow, four orders get stuck inside
+ * the service and probes of every priority show which of them are shed. In phase three the provider
+ * recovers, the stuck orders complete and new requests are admitted again.
+ */
+@Slf4j
+public class App {
+
+ private static final int CAPACITY = 5;
+ private static final int LOW_PRIORITY_LIMIT = 3;
+ private static final int CRITICAL_RESERVE = 1;
+ private static final int STUCK_ORDERS = 4;
+ private static final Duration WAIT = Duration.ofSeconds(5);
+
+ /**
+ * Program entry point.
+ *
+ * @param args command line arguments, not used
+ * @throws InterruptedException if the demo is interrupted while waiting for the workers
+ */
+ public static void main(String[] args) throws InterruptedException {
+ var shedder = new LoadShedder(CAPACITY, LOW_PRIORITY_LIMIT, CRITICAL_RESERVE);
+ var paymentSlow = new AtomicBoolean(false);
+ var paymentRecovered = new CountDownLatch(1);
+ var entered = new Semaphore(0);
+ var orderService =
+ new ShedGuardedService(
+ "order-service",
+ shedder,
+ simulatedPaymentProvider(paymentSlow, paymentRecovered, entered, WAIT));
+ var executor = Executors.newCachedThreadPool();
+ try {
+ LOGGER.info(
+ "Order service capacity: {} in flight, low priority shed at {}, {} slot reserved for"
+ + " critical requests",
+ CAPACITY,
+ LOW_PRIORITY_LIMIT,
+ CRITICAL_RESERVE);
+
+ LOGGER.info("--- Phase 1: light load, every request is admitted ---");
+ report(orderService.handle(new Request("r1", Priority.LOW, "prefetch recommendations")));
+ report(orderService.handle(new Request("r2", Priority.NORMAL, "view cart")));
+
+ LOGGER.info("--- Phase 2: payment provider slows down, orders pile up ---");
+ paymentSlow.set(true);
+ List> stuckOrders = new ArrayList<>();
+ for (var i = 1; i <= STUCK_ORDERS; i++) {
+ var order = new Request("order-" + i, Priority.NORMAL, "place order");
+ stuckOrders.add(executor.submit(() -> orderService.handle(order)));
+ }
+ awaitEntered(entered, STUCK_ORDERS, WAIT);
+ LOGGER.info(
+ "{} of {} slots busy, probing with every priority", shedder.getInFlight(), CAPACITY);
+ report(orderService.handle(new Request("p1", Priority.LOW, "prefetch recommendations")));
+ report(orderService.handle(new Request("p2", Priority.NORMAL, "view cart")));
+ var checkout =
+ executor.submit(
+ () -> orderService.handle(new Request("p3", Priority.CRITICAL, "checkout payment")));
+ awaitEntered(entered, 1, WAIT);
+
+ LOGGER.info("--- Phase 3: payment provider recovers, load drops ---");
+ paymentSlow.set(false);
+ paymentRecovered.countDown();
+ for (var order : stuckOrders) {
+ report(result(order, WAIT));
+ }
+ report(result(checkout, WAIT));
+ report(orderService.handle(new Request("r3", Priority.LOW, "prefetch recommendations")));
+
+ LOGGER.info(
+ "Summary: accepted={}, shed low={}, shed normal={}, shed critical={}",
+ shedder.getAccepted(),
+ shedder.getShed(Priority.LOW),
+ shedder.getShed(Priority.NORMAL),
+ shedder.getShed(Priority.CRITICAL));
+ } finally {
+ shutdown(executor, WAIT);
+ }
+ }
+
+ /**
+ * Business logic of the order service. While the payment provider is slow every admitted request
+ * blocks until the provider recovers, which is exactly the situation in which requests pile up
+ * and load shedding becomes necessary. The semaphore tells the demo how many requests are stuck.
+ */
+ static RequestHandler simulatedPaymentProvider(
+ AtomicBoolean paymentSlow,
+ CountDownLatch paymentRecovered,
+ Semaphore entered,
+ Duration recoveryTimeout) {
+ return request -> {
+ if (paymentSlow.get()) {
+ // Signal the demo that one more request is now stuck behind the slow provider.
+ entered.release();
+ try {
+ if (!paymentRecovered.await(recoveryTimeout.toMillis(), TimeUnit.MILLISECONDS)) {
+ throw new IllegalStateException("payment provider never recovered");
+ }
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ throw new IllegalStateException("interrupted while processing " + request.id(), e);
+ }
+ }
+ return "processed " + request.description();
+ };
+ }
+
+ /** Waits until the given number of requests are stuck inside the service. */
+ static void awaitEntered(Semaphore entered, int count, Duration timeout)
+ throws InterruptedException {
+ if (!entered.tryAcquire(count, timeout.toMillis(), TimeUnit.MILLISECONDS)) {
+ throw new IllegalStateException("requests did not enter the service in time");
+ }
+ }
+
+ /** Collects the response of a request that was handled on a worker thread. */
+ static Response result(Future future, Duration timeout) throws InterruptedException {
+ try {
+ return future.get(timeout.toMillis(), TimeUnit.MILLISECONDS);
+ } catch (ExecutionException | TimeoutException e) {
+ throw new IllegalStateException("worker failed", e);
+ }
+ }
+
+ /** Stops the worker pool, forcing the shutdown if workers do not finish within the timeout. */
+ static void shutdown(ExecutorService executor, Duration timeout) throws InterruptedException {
+ executor.shutdown();
+ if (!executor.awaitTermination(timeout.toMillis(), TimeUnit.MILLISECONDS)) {
+ executor.shutdownNow();
+ }
+ }
+
+ static void report(Response response) {
+ LOGGER.info("{} -> {}: {}", response.requestId(), response.status(), response.message());
+ }
+}
diff --git a/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedException.java b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedException.java
new file mode 100644
index 000000000000..a6d8eb17d92b
--- /dev/null
+++ b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedException.java
@@ -0,0 +1,55 @@
+/*
+ * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
+ *
+ * The MIT License
+ * Copyright © 2014-2022 Ilkka Seppälä
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+package com.iluwatar.loadshedding;
+
+import lombok.Getter;
+
+/**
+ * Thrown by {@link LoadShedder#acquire(Request)} when a request must be shed. It is the in-process
+ * equivalent of an HTTP 503 "Service Unavailable" response: the service is healthy but has no spare
+ * capacity for this request right now, and the caller should back off and retry later.
+ */
+@Getter
+public class LoadShedException extends RuntimeException {
+
+ private final String requestId;
+ private final Priority priority;
+
+ /**
+ * Creates the exception for a shed request.
+ *
+ * @param request the request that was shed
+ * @param inFlight number of requests being processed at the moment of the decision
+ * @param limit admission limit that applies to the request's priority
+ */
+ public LoadShedException(Request request, int inFlight, int limit) {
+ super(
+ String.format(
+ "Request %s shed: %d requests in flight, limit for %s priority is %d",
+ request.id(), inFlight, request.priority(), limit));
+ this.requestId = request.id();
+ this.priority = request.priority();
+ }
+}
diff --git a/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedder.java b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedder.java
new file mode 100644
index 000000000000..e9977c3f338d
--- /dev/null
+++ b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/LoadShedder.java
@@ -0,0 +1,140 @@
+/*
+ * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
+ *
+ * The MIT License
+ * Copyright © 2014-2022 Ilkka Seppälä
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+package com.iluwatar.loadshedding;
+
+import java.util.EnumMap;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.LongAdder;
+
+/**
+ * Admission controller that decides whether a request may enter the service. The decision is based
+ * on how many requests are already in flight and on the priority of the new request:
+ *
+ *
+ * - {@link Priority#LOW} requests are admitted only while the in-flight count is below the low
+ * priority limit, so they are the first to be shed when load builds up.
+ *
- {@link Priority#NORMAL} requests are admitted while the in-flight count is below the hard
+ * capacity minus the reserve kept for critical work.
+ *
- {@link Priority#CRITICAL} requests may use the full capacity, including the reserve.
+ *
+ *
+ * Requests that cannot be admitted are rejected immediately with a {@link LoadShedException}
+ * rather than queued. Rejecting quickly costs almost nothing, whereas letting an unbounded queue
+ * grow would increase latency for every request and eventually exhaust memory or threads.
+ *
+ *
The class is thread-safe. Admission is lock-free: the in-flight count is updated with an
+ * atomic accumulator whose function refuses to increment past the limit, so concurrent callers can
+ * never push the in-flight count above the configured capacity.
+ */
+public class LoadShedder {
+
+ private final int maxInFlight;
+ private final Map limits = new EnumMap<>(Priority.class);
+ private final AtomicInteger inFlight = new AtomicInteger();
+ private final LongAdder accepted = new LongAdder();
+ private final Map shed = new EnumMap<>(Priority.class);
+
+ /**
+ * Creates a load shedder.
+ *
+ * @param maxInFlight hard capacity: the maximum number of requests processed concurrently
+ * @param lowPriorityLimit in-flight count at which low priority requests start being shed
+ * @param criticalReserve part of the capacity that only critical requests may use
+ */
+ public LoadShedder(int maxInFlight, int lowPriorityLimit, int criticalReserve) {
+ if (maxInFlight <= 0) {
+ throw new IllegalArgumentException("maxInFlight must be positive");
+ }
+ if (criticalReserve < 0 || criticalReserve >= maxInFlight) {
+ throw new IllegalArgumentException("criticalReserve must be between 0 and maxInFlight - 1");
+ }
+ var normalLimit = maxInFlight - criticalReserve;
+ if (lowPriorityLimit <= 0 || lowPriorityLimit > normalLimit) {
+ throw new IllegalArgumentException(
+ "lowPriorityLimit must be between 1 and maxInFlight - criticalReserve");
+ }
+ this.maxInFlight = maxInFlight;
+ limits.put(Priority.CRITICAL, maxInFlight);
+ limits.put(Priority.NORMAL, normalLimit);
+ limits.put(Priority.LOW, lowPriorityLimit);
+ for (var priority : Priority.values()) {
+ shed.put(priority, new LongAdder());
+ }
+ }
+
+ /**
+ * Tries to admit the request. On success the in-flight count is incremented and the caller must
+ * invoke {@link #release()} once the work is done.
+ *
+ * @param request the request asking for admission
+ * @throws LoadShedException if the service has no spare capacity for this priority
+ */
+ public void acquire(Request request) {
+ var priority = request.priority();
+ var limit = limits.get(priority);
+ // The accumulator is a pure function, so it is safe for the atomic to re-apply it under
+ // contention: the count is only incremented while it is below the limit for this priority.
+ var previous =
+ inFlight.getAndAccumulate(
+ 1, (current, increment) -> current >= limit ? current : current + increment);
+ if (previous >= limit) {
+ // Fail fast: the caller gets an immediate rejection instead of waiting in a queue.
+ shed.get(priority).increment();
+ throw new LoadShedException(request, previous, limit);
+ }
+ accepted.increment();
+ }
+
+ /** Signals that a previously admitted request has finished, freeing one slot of capacity. */
+ public void release() {
+ inFlight.decrementAndGet();
+ }
+
+ /** Hard capacity of the service. */
+ public int getMaxInFlight() {
+ return maxInFlight;
+ }
+
+ /** Number of requests currently being processed. */
+ public int getInFlight() {
+ return inFlight.get();
+ }
+
+ /** Total number of requests admitted since creation. */
+ public long getAccepted() {
+ return accepted.sum();
+ }
+
+ /** Number of requests of the given priority that were shed since creation. */
+ public long getShed(Priority priority) {
+ return shed.get(priority).sum();
+ }
+
+ /** Total number of requests shed since creation, across all priorities. */
+ public long getTotalShed() {
+ return shed.values().stream().mapToLong(LongAdder::sum).sum();
+ }
+}
diff --git a/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Priority.java b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Priority.java
new file mode 100644
index 000000000000..a2baca4d5a53
--- /dev/null
+++ b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Priority.java
@@ -0,0 +1,39 @@
+/*
+ * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
+ *
+ * The MIT License
+ * Copyright © 2014-2022 Ilkka Seppälä
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+package com.iluwatar.loadshedding;
+
+/**
+ * Importance of a request from the point of view of the service that receives it. When the service
+ * approaches its capacity the {@link LoadShedder} sheds the least important requests first, so that
+ * the work that matters most keeps flowing while excess load is rejected.
+ */
+public enum Priority {
+ /** Must be served whenever physically possible, for example checkout or health probes. */
+ CRITICAL,
+ /** Regular user traffic. */
+ NORMAL,
+ /** Best-effort work such as prefetching, analytics or background refreshes. */
+ LOW
+}
diff --git a/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Request.java b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Request.java
new file mode 100644
index 000000000000..7ff2681ce21f
--- /dev/null
+++ b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Request.java
@@ -0,0 +1,36 @@
+/*
+ * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
+ *
+ * The MIT License
+ * Copyright © 2014-2022 Ilkka Seppälä
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+package com.iluwatar.loadshedding;
+
+/**
+ * An incoming unit of work. Besides its identity it carries the {@link Priority} the caller
+ * assigned to it, which is the only thing the {@link LoadShedder} needs to decide whether the
+ * request may enter the service.
+ *
+ * @param id unique identifier used in logs and responses
+ * @param priority importance of the request
+ * @param description human readable summary of the work
+ */
+public record Request(String id, Priority priority, String description) {}
diff --git a/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/RequestHandler.java b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/RequestHandler.java
new file mode 100644
index 000000000000..588475ae09b3
--- /dev/null
+++ b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/RequestHandler.java
@@ -0,0 +1,41 @@
+/*
+ * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
+ *
+ * The MIT License
+ * Copyright © 2014-2022 Ilkka Seppälä
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+package com.iluwatar.loadshedding;
+
+/**
+ * The actual work a service performs for an admitted request. Kept as a functional interface so
+ * that the admission control in {@link ShedGuardedService} stays independent of the business logic.
+ */
+@FunctionalInterface
+public interface RequestHandler {
+
+ /**
+ * Processes the request.
+ *
+ * @param request an admitted request
+ * @return result to return to the caller
+ */
+ String handle(Request request);
+}
diff --git a/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Response.java b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Response.java
new file mode 100644
index 000000000000..494defd14a30
--- /dev/null
+++ b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/Response.java
@@ -0,0 +1,55 @@
+/*
+ * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
+ *
+ * The MIT License
+ * Copyright © 2014-2022 Ilkka Seppälä
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+package com.iluwatar.loadshedding;
+
+/**
+ * Outcome returned to the caller. A shed request gets a {@link Status#REJECTED} response
+ * immediately instead of waiting in a queue, which is the fail-fast behaviour that keeps the
+ * service responsive under overload.
+ *
+ * @param requestId identifier of the request this response belongs to
+ * @param status whether the request was processed or shed
+ * @param message result of the work, or the reason the request was shed
+ */
+public record Response(String requestId, Status status, String message) {
+
+ /** Result of admission control. */
+ public enum Status {
+ /** The request was admitted and processed. */
+ ACCEPTED,
+ /** The request was shed because the service is at capacity. Callers may retry later. */
+ REJECTED
+ }
+
+ /** Creates the response for a request that was processed. */
+ public static Response accepted(Request request, String result) {
+ return new Response(request.id(), Status.ACCEPTED, result);
+ }
+
+ /** Creates the fast-failure response for a request that was shed. */
+ public static Response rejected(Request request, String reason) {
+ return new Response(request.id(), Status.REJECTED, reason);
+ }
+}
diff --git a/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java
new file mode 100644
index 000000000000..02654c213759
--- /dev/null
+++ b/microservices-load-shedding/src/main/java/com/iluwatar/loadshedding/ShedGuardedService.java
@@ -0,0 +1,81 @@
+/*
+ * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
+ *
+ * The MIT License
+ * Copyright © 2014-2022 Ilkka Seppälä
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+package com.iluwatar.loadshedding;
+
+import lombok.extern.slf4j.Slf4j;
+
+/**
+ * A service whose entry point is protected by a {@link LoadShedder}. Every request first asks the
+ * shedder for admission; requests that are shed receive a {@link Response.Status#REJECTED} response
+ * right away, admitted requests are handed to the {@link RequestHandler} and always release their
+ * capacity slot afterwards, even when the handler fails.
+ */
+@Slf4j
+public class ShedGuardedService {
+
+ private final String name;
+ private final LoadShedder shedder;
+ private final RequestHandler handler;
+
+ /**
+ * Creates a guarded service.
+ *
+ * @param name service name used in log output
+ * @param shedder admission controller protecting the service
+ * @param handler business logic executed for admitted requests
+ */
+ public ShedGuardedService(String name, LoadShedder shedder, RequestHandler handler) {
+ this.name = name;
+ this.shedder = shedder;
+ this.handler = handler;
+ }
+
+ /**
+ * Handles the request if capacity allows, otherwise fails fast.
+ *
+ * @param request incoming request
+ * @return the handler result, or a rejection when the request was shed
+ */
+ public Response handle(Request request) {
+ try {
+ shedder.acquire(request);
+ } catch (LoadShedException e) {
+ LOGGER.warn("[{}] shed {} ({}): {}", name, request.id(), request.priority(), e.getMessage());
+ return Response.rejected(request, e.getMessage());
+ }
+ LOGGER.info(
+ "[{}] admitted {} ({}), {}/{} in flight",
+ name,
+ request.id(),
+ request.priority(),
+ shedder.getInFlight(),
+ shedder.getMaxInFlight());
+ try {
+ return Response.accepted(request, handler.handle(request));
+ } finally {
+ shedder.release();
+ }
+ }
+}
diff --git a/microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/AppTest.java b/microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/AppTest.java
new file mode 100644
index 000000000000..3a3b6a8f9da6
--- /dev/null
+++ b/microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/AppTest.java
@@ -0,0 +1,177 @@
+/*
+ * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
+ *
+ * The MIT License
+ * Copyright © 2014-2022 Ilkka Seppälä
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+package com.iluwatar.loadshedding;
+
+import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertInstanceOf;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.time.Duration;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Semaphore;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicBoolean;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+
+class AppTest {
+
+ private static final Duration SHORT = Duration.ofMillis(10);
+ private static final Duration GENEROUS = Duration.ofSeconds(1);
+ private static final Request REQUEST = new Request("r", Priority.NORMAL, "place order");
+
+ @AfterEach
+ void clearInterruptFlag() {
+ Thread.interrupted();
+ }
+
+ @Test
+ void shouldLaunchApp() {
+ assertDoesNotThrow(() -> App.main(new String[] {}));
+ }
+
+ @Test
+ void shouldBeInstantiable() {
+ assertNotNull(new App(), "App should be instantiable");
+ }
+
+ @Test
+ void paymentProviderProcessesImmediatelyWhenHealthy() {
+ var entered = new Semaphore(0);
+ var handler =
+ App.simulatedPaymentProvider(
+ new AtomicBoolean(false), new CountDownLatch(1), entered, GENEROUS);
+ assertEquals("processed place order", handler.handle(REQUEST));
+ assertEquals(0, entered.availablePermits());
+ }
+
+ @Test
+ void paymentProviderResumesOnceRecovered() {
+ var entered = new Semaphore(0);
+ var recovered = new CountDownLatch(0);
+ var handler =
+ App.simulatedPaymentProvider(new AtomicBoolean(true), recovered, entered, GENEROUS);
+ assertEquals("processed place order", handler.handle(REQUEST));
+ assertEquals(1, entered.availablePermits());
+ }
+
+ @Test
+ void paymentProviderFailsWhenRecoveryTimesOut() {
+ var handler =
+ App.simulatedPaymentProvider(
+ new AtomicBoolean(true), new CountDownLatch(1), new Semaphore(0), SHORT);
+ var exception = assertThrows(IllegalStateException.class, () -> handler.handle(REQUEST));
+ assertEquals("payment provider never recovered", exception.getMessage());
+ }
+
+ @Test
+ void paymentProviderRestoresInterruptFlagWhenInterruptedWhileSlow() {
+ var entered = new Semaphore(0);
+ var handler =
+ App.simulatedPaymentProvider(
+ new AtomicBoolean(true), new CountDownLatch(1), entered, GENEROUS);
+ Thread.currentThread().interrupt();
+ var exception = assertThrows(IllegalStateException.class, () -> handler.handle(REQUEST));
+ assertInstanceOf(InterruptedException.class, exception.getCause());
+ assertTrue(Thread.interrupted());
+ assertEquals(1, entered.availablePermits());
+ }
+
+ @Test
+ void awaitEnteredReturnsOncePermitsAreAvailable() {
+ var entered = new Semaphore(2);
+ assertDoesNotThrow(() -> App.awaitEntered(entered, 2, GENEROUS));
+ assertEquals(0, entered.availablePermits());
+ }
+
+ @Test
+ void awaitEnteredFailsWhenNobodyEnters() {
+ var entered = new Semaphore(0);
+ var exception =
+ assertThrows(IllegalStateException.class, () -> App.awaitEntered(entered, 1, SHORT));
+ assertEquals("requests did not enter the service in time", exception.getMessage());
+ }
+
+ @Test
+ void resultReturnsCompletedResponse() throws InterruptedException {
+ var response = Response.accepted(REQUEST, "done");
+ assertEquals(response, App.result(CompletableFuture.completedFuture(response), GENEROUS));
+ }
+
+ @Test
+ void resultWrapsFailedWorker() {
+ var failed = CompletableFuture.failedFuture(new RuntimeException("boom"));
+ var exception = assertThrows(IllegalStateException.class, () -> App.result(failed, GENEROUS));
+ assertEquals("worker failed", exception.getMessage());
+ assertInstanceOf(RuntimeException.class, exception.getCause().getCause());
+ }
+
+ @Test
+ void resultWrapsWorkerThatNeverFinishes() {
+ var pending = new CompletableFuture();
+ var exception = assertThrows(IllegalStateException.class, () -> App.result(pending, SHORT));
+ assertEquals("worker failed", exception.getMessage());
+ assertInstanceOf(TimeoutException.class, exception.getCause());
+ }
+
+ @Test
+ void shutdownWaitsForIdleExecutor() throws InterruptedException {
+ var executor = Executors.newSingleThreadExecutor();
+ App.shutdown(executor, GENEROUS);
+ assertTrue(executor.isTerminated());
+ }
+
+ @Test
+ void shutdownForcesStopWhenWorkersIgnoreTheTimeout() throws InterruptedException {
+ var executor = Executors.newSingleThreadExecutor();
+ var started = new CountDownLatch(1);
+ var interrupted = new CountDownLatch(1);
+ var release = new CountDownLatch(1);
+ executor.execute(
+ () -> {
+ started.countDown();
+ while (release.getCount() > 0) {
+ try {
+ release.await();
+ } catch (InterruptedException e) {
+ // A stubborn worker that keeps going despite the interrupt.
+ interrupted.countDown();
+ }
+ }
+ });
+ assertTrue(started.await(1, TimeUnit.SECONDS));
+ App.shutdown(executor, SHORT);
+ assertTrue(executor.isShutdown());
+ assertTrue(interrupted.await(1, TimeUnit.SECONDS));
+ release.countDown();
+ assertTrue(executor.awaitTermination(1, TimeUnit.SECONDS));
+ }
+}
diff --git a/microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/LoadShedderTest.java b/microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/LoadShedderTest.java
new file mode 100644
index 000000000000..d95b83cbcd3a
--- /dev/null
+++ b/microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/LoadShedderTest.java
@@ -0,0 +1,187 @@
+/*
+ * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
+ *
+ * The MIT License
+ * Copyright © 2014-2022 Ilkka Seppälä
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+package com.iluwatar.loadshedding;
+
+import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import org.junit.jupiter.api.Test;
+
+class LoadShedderTest {
+
+ private static final int CAPACITY = 5;
+ private static final int LOW_LIMIT = 3;
+ private static final int RESERVE = 1;
+
+ private final LoadShedder shedder = new LoadShedder(CAPACITY, LOW_LIMIT, RESERVE);
+
+ private static Request request(Priority priority) {
+ return new Request("req-" + priority, priority, "test");
+ }
+
+ private void fill(int count) {
+ for (var i = 0; i < count; i++) {
+ shedder.acquire(request(Priority.CRITICAL));
+ }
+ }
+
+ @Test
+ void admitsEveryPriorityBelowLowPriorityLimit() {
+ fill(LOW_LIMIT - 1);
+ assertDoesNotThrow(() -> shedder.acquire(request(Priority.LOW)));
+ assertEquals(LOW_LIMIT, shedder.getInFlight());
+ assertEquals(LOW_LIMIT, shedder.getAccepted());
+ assertEquals(0, shedder.getTotalShed());
+ }
+
+ @Test
+ void shedsLowPriorityFirst() {
+ fill(LOW_LIMIT);
+ assertThrows(LoadShedException.class, () -> shedder.acquire(request(Priority.LOW)));
+ assertDoesNotThrow(() -> shedder.acquire(request(Priority.NORMAL)));
+ assertEquals(LOW_LIMIT + 1, shedder.getInFlight());
+ assertEquals(1, shedder.getShed(Priority.LOW));
+ assertEquals(0, shedder.getShed(Priority.NORMAL));
+ }
+
+ @Test
+ void keepsReserveForCriticalRequests() {
+ fill(CAPACITY - RESERVE);
+ assertThrows(LoadShedException.class, () -> shedder.acquire(request(Priority.NORMAL)));
+ assertDoesNotThrow(() -> shedder.acquire(request(Priority.CRITICAL)));
+ assertEquals(CAPACITY, shedder.getInFlight());
+ assertEquals(1, shedder.getShed(Priority.NORMAL));
+ assertEquals(0, shedder.getShed(Priority.CRITICAL));
+ }
+
+ @Test
+ void shedsCriticalRequestsAtHardCapacity() {
+ fill(CAPACITY);
+ assertThrows(LoadShedException.class, () -> shedder.acquire(request(Priority.CRITICAL)));
+ assertEquals(CAPACITY, shedder.getInFlight());
+ assertEquals(1, shedder.getShed(Priority.CRITICAL));
+ }
+
+ @Test
+ void releaseFreesCapacity() {
+ fill(LOW_LIMIT);
+ assertThrows(LoadShedException.class, () -> shedder.acquire(request(Priority.LOW)));
+ shedder.release();
+ assertDoesNotThrow(() -> shedder.acquire(request(Priority.LOW)));
+ assertEquals(LOW_LIMIT, shedder.getInFlight());
+ }
+
+ @Test
+ void countsShedRequestsPerPriority() {
+ fill(CAPACITY);
+ assertThrows(LoadShedException.class, () -> shedder.acquire(request(Priority.LOW)));
+ assertThrows(LoadShedException.class, () -> shedder.acquire(request(Priority.LOW)));
+ assertThrows(LoadShedException.class, () -> shedder.acquire(request(Priority.NORMAL)));
+ assertThrows(LoadShedException.class, () -> shedder.acquire(request(Priority.CRITICAL)));
+ assertEquals(CAPACITY, shedder.getAccepted());
+ assertEquals(2, shedder.getShed(Priority.LOW));
+ assertEquals(1, shedder.getShed(Priority.NORMAL));
+ assertEquals(1, shedder.getShed(Priority.CRITICAL));
+ assertEquals(4, shedder.getTotalShed());
+ }
+
+ @Test
+ void exceptionDescribesTheDecision() {
+ fill(LOW_LIMIT);
+ var request = new Request("low-42", Priority.LOW, "test");
+ var exception = assertThrows(LoadShedException.class, () -> shedder.acquire(request));
+ assertEquals("low-42", exception.getRequestId());
+ assertEquals(Priority.LOW, exception.getPriority());
+ assertTrue(exception.getMessage().contains("low-42"));
+ assertTrue(exception.getMessage().contains("LOW"));
+ }
+
+ @Test
+ void neverExceedsCapacityUnderConcurrentAdmission() throws InterruptedException {
+ var callers = 50;
+ var start = new CountDownLatch(1);
+ var done = new CountDownLatch(callers);
+ var admitted = new AtomicInteger();
+ var executor = Executors.newFixedThreadPool(callers);
+ try {
+ for (var i = 0; i < callers; i++) {
+ executor.execute(
+ () -> {
+ try {
+ start.await();
+ shedder.acquire(request(Priority.CRITICAL));
+ admitted.incrementAndGet();
+ } catch (LoadShedException expected) {
+ // shed
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ } finally {
+ done.countDown();
+ }
+ });
+ }
+ start.countDown();
+ assertTrue(done.await(10, TimeUnit.SECONDS));
+ } finally {
+ executor.shutdownNow();
+ }
+ assertEquals(CAPACITY, admitted.get());
+ assertEquals(CAPACITY, shedder.getInFlight());
+ assertEquals(callers - CAPACITY, shedder.getShed(Priority.CRITICAL));
+ }
+
+ @Test
+ void rejectsInvalidConfiguration() {
+ // maxInFlight must be positive
+ assertThrows(IllegalArgumentException.class, () -> new LoadShedder(0, 1, 0));
+ // criticalReserve must be between 0 and maxInFlight - 1
+ assertThrows(IllegalArgumentException.class, () -> new LoadShedder(5, 3, -1));
+ assertThrows(IllegalArgumentException.class, () -> new LoadShedder(5, 3, 5));
+ // lowPriorityLimit must be between 1 and maxInFlight - criticalReserve
+ assertThrows(IllegalArgumentException.class, () -> new LoadShedder(5, 0, 1));
+ assertThrows(IllegalArgumentException.class, () -> new LoadShedder(5, 5, 1));
+ }
+
+ @Test
+ void acceptsBoundaryConfiguration() {
+ assertDoesNotThrow(() -> new LoadShedder(5, 4, 1));
+ var noReserve = new LoadShedder(5, 5, 0);
+ fillWith(noReserve, Priority.LOW, 5);
+ assertEquals(5, noReserve.getInFlight());
+ assertThrows(LoadShedException.class, () -> noReserve.acquire(request(Priority.CRITICAL)));
+ }
+
+ private static void fillWith(LoadShedder target, Priority priority, int count) {
+ for (var i = 0; i < count; i++) {
+ target.acquire(request(priority));
+ }
+ }
+}
diff --git a/microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/ShedGuardedServiceTest.java b/microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/ShedGuardedServiceTest.java
new file mode 100644
index 000000000000..87c847196517
--- /dev/null
+++ b/microservices-load-shedding/src/test/java/com/iluwatar/loadshedding/ShedGuardedServiceTest.java
@@ -0,0 +1,80 @@
+/*
+ * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
+ *
+ * The MIT License
+ * Copyright © 2014-2022 Ilkka Seppälä
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+package com.iluwatar.loadshedding;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+import java.util.concurrent.atomic.AtomicBoolean;
+import org.junit.jupiter.api.Test;
+
+class ShedGuardedServiceTest {
+
+ private final LoadShedder shedder = new LoadShedder(2, 1, 0);
+
+ @Test
+ void returnsHandlerResultWhenAdmitted() {
+ var service = new ShedGuardedService("svc", shedder, request -> "done " + request.id());
+ var response = service.handle(new Request("a", Priority.NORMAL, "work"));
+ assertEquals(new Response("a", Response.Status.ACCEPTED, "done a"), response);
+ assertEquals(0, shedder.getInFlight());
+ }
+
+ @Test
+ void returnsRejectionWithoutInvokingHandlerWhenShed() {
+ var handlerCalled = new AtomicBoolean();
+ var service =
+ new ShedGuardedService(
+ "svc",
+ shedder,
+ request -> {
+ handlerCalled.set(true);
+ return "unexpected";
+ });
+ shedder.acquire(new Request("occupied", Priority.NORMAL, "work"));
+ var response = service.handle(new Request("b", Priority.LOW, "work"));
+ assertEquals(Response.Status.REJECTED, response.status());
+ assertEquals("b", response.requestId());
+ assertFalse(handlerCalled.get());
+ assertEquals(1, shedder.getInFlight());
+ assertEquals(1, shedder.getShed(Priority.LOW));
+ }
+
+ @Test
+ void releasesCapacityWhenHandlerFails() {
+ var service =
+ new ShedGuardedService(
+ "svc",
+ shedder,
+ request -> {
+ throw new IllegalStateException("boom");
+ });
+ assertThrows(
+ IllegalStateException.class,
+ () -> service.handle(new Request("c", Priority.NORMAL, "work")));
+ assertEquals(0, shedder.getInFlight());
+ }
+}
diff --git a/pom.xml b/pom.xml
index a71630d289d3..266c02fcb388 100644
--- a/pom.xml
+++ b/pom.xml
@@ -260,6 +260,7 @@
rate-limiting-pattern
fallback
onion-architecture
+ microservices-load-shedding