Add concurrency-limits example

examples/webserver/concurrency-limits/README.md

@@ -0,0 +1,139 @@
+# Helidon SE Concurrency Limits Example
+
+This example demonstrates the Concurrency Limits feature of Helidon WebServer. 
+For other approaches for rate limiting see the [Rate Limit Example](../ratelimit).
+
+The Concurrency Limits feature provides mechanisms to limit concurrent execution of incoming requests.
+Currently two algorithms are supported:
+
+1. Fixed: A semaphore based limit that supports queuing for a permit and timeout on the queue.
+2. AIMD: Additive-increase/multiplicative-decrease. Uses a feedback control algorithm that combines linear growth of the request limit when there is no congestion with an exponential reduction when congestion is detected.
+
+The example does the following:
+
+1. Defines two ports for accepting incoming requests: 8080 and 8088
+2. 8080 is configured to use the Fixed concurrency limit algorithm
+3. 8088 is configured to use the AIMD concurrency limit algorithm
+
+The server has two endpoints: `fixed/sleep` and `aimd/sleep`. Both sleep for the specified number of seconds to simulate a slow workload.
+
+These values are configured in [`application.yaml`](./src/main/resources/application.yaml)
+
+## Build and run
+
+Build and start the server:
+```shell
+mvn package
+java -jar target/helidon-examples-webserver-concurrencylimits.jar
+```
+
+## Exercise the application
+
+### Fixed Concurrency Limit
+
+The server is configured to process 6 requests concurrently with a backlog queue of depth 4. Therefore it can handle bursts of up to 10 requests at a time.
+
+Send a burst of 15 concurrent requests to the `fixed/sleep` endpoint (each requesting a sleep of 3 seconds):
+```shell
+curl -s  \
+  --noproxy '*' \
+  -o /dev/null \
+  --parallel \
+  --parallel-immediate \
+  -w "%{http_code}\n" \
+  "http://localhost:8080/fixed/sleep/3?c=[1-15]"
+```
+
+When the 15 concurrent requests hit the server:
+
+* 5 of the requests are rejected with a 503 because they exceed the size of the number of concurrent requests limit (6) plus the size of the queue (4).
+* 10 of the requests are accepted by the server
+* The first 6 of those return a 200 after sleeping for 3 seconds
+* The next 4 requests are then processed from the queue and return a 200 after sleeping for 3 more seconds.
+
+You will see this in the output:
+```
+503
+503
+503
+503
+503
+# Three second pause
+200
+200
+200
+200
+200
+200
+# Three second pause
+200
+200
+200
+200
+```
+
+### AIMD Concurrency Limit
+
+The AIMD limiter is [adaptive](https://en.wikipedia.org/wiki/Additive_increase/multiplicative_decrease)
+and will adjust according to the workload and responsiveness of the server.
+If the server successfully processes requests then the limiter will increase limits (up to a max limit).
+If the server starts to fail or timeout requests then the limiter will reduce limits (down to a min limit).
+
+In this example we set the initial and minimum limit to 6 and a max limit of 10 concurrent requests.
+We configure the timeout as 3 seconds. So requests that can't be serviced within 3 seconds fail -- which can lead to a reduction of the current limit.
+
+First execute 15 concurrent requests, each sleeping for 3 seconds.
+
+```shell
+curl -s  \
+  --noproxy '*' \
+  -o /dev/null \
+  --parallel \
+  --parallel-immediate \
+  -w "%{http_code}\n" \
+  "http://localhost:8088/aimd/sleep/3?c=[1-15]"
+```
+
+This will process 6 request (the currently configured limit), and fail 9:
+
+```
+503
+503
+503
+503
+503
+503
+503
+503
+503
+# Pause for 3 seconds
+200
+200
+200
+200
+200
+200
+```
+
+Repeat the curl command and you will see the same results. Because we have the AIMD timeout set to 3 seconds, and our sleep operation is taking 3 seconds, the current limit is never increased.
+
+Now repeat the curl command, but specify a sleep time of 2 seconds:
+
+```shell
+curl -s  \
+  --noproxy '*' \
+  -o /dev/null \
+  --parallel \
+  --parallel-immediate \
+  -w "%{http_code}\n" \
+  "http://localhost:8088/aimd/sleep/2?c=[1-15]"
+```
+
+As before the first invocation will process 6 requests. But as you
+repeat the curl command you will see more requests are
+successful until you hit the max concurrency limit of 10. Since
+each request is now taking 2 seconds (and not 3), the limiter
+is able to adapt and increase the current limit up to the maximum.
+
+Now go back and run the curl command a few times using a sleep time
+of 3, and you will see the limiter adapt again and lower the current limit.

examples/webserver/concurrency-limits/pom.xml

@@ -0,0 +1,86 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+    Copyright (c) 2024 Oracle and/or its affiliates.
+
+    Licensed under the Apache License, Version 2.0 (the "License");
+    you may not use this file except in compliance with the License.
+    You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <parent>
+        <groupId>io.helidon.applications</groupId>
+        <artifactId>helidon-se</artifactId>
+        <version>4.2.0-SNAPSHOT</version>
+        <relativePath/>
+    </parent>
+    <groupId>io.helidon.examples.webserver</groupId>
+    <artifactId>helidon-examples-webserver-concurrencylimits</artifactId>
+    <version>1.0.0-SNAPSHOT</version>
+    <name>Helidon Examples WebServer Concurrency Limits</name>
+
+    <properties>
+        <mainClass>io.helidon.examples.webserver.concurrencylimits.Main</mainClass>
+    </properties>
+
+    <dependencies>
+        <dependency>
+            <groupId>io.helidon.webserver</groupId>
+            <artifactId>helidon-webserver</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>io.helidon.config</groupId>
+            <artifactId>helidon-config-yaml</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>io.helidon.fault-tolerance</groupId>
+            <artifactId>helidon-fault-tolerance</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>io.helidon.logging</groupId>
+            <artifactId>helidon-logging-jul</artifactId>
+            <scope>runtime</scope>
+        </dependency>
+        <dependency>
+            <groupId>org.junit.jupiter</groupId>
+            <artifactId>junit-jupiter-api</artifactId>
+            <scope>test</scope>
+        </dependency>
+        <dependency>
+            <groupId>org.hamcrest</groupId>
+            <artifactId>hamcrest-all</artifactId>
+            <scope>test</scope>
+        </dependency>
+        <dependency>
+            <groupId>io.helidon.webserver.testing.junit5</groupId>
+            <artifactId>helidon-webserver-testing-junit5</artifactId>
+            <scope>test</scope>
+        </dependency>
+    </dependencies>
+
+    <build>
+        <plugins>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-dependency-plugin</artifactId>
+                <executions>
+                    <execution>
+                        <id>copy-libs</id>
+                    </execution>
+                </executions>
+            </plugin>
+        </plugins>
+    </build>
+</project>

examples/webserver/concurrency-limits/src/main/java/io/helidon/examples/webserver/concurrencylimits/Main.java

@@ -0,0 +1,75 @@
+/*
+ * Copyright (c) 2024 Oracle and/or its affiliates.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package io.helidon.examples.webserver.concurrencylimits;
+
+import io.helidon.config.Config;
+import io.helidon.logging.common.LogConfig;
+import io.helidon.webserver.WebServer;
+import io.helidon.webserver.WebServerConfig;
+import io.helidon.webserver.http.HttpRouting;
+
+/**
+ * The application main class.
+ */
+public class Main {
+
+    private static final System.Logger LOGGER = System.getLogger(Main.class.getName());
+
+    /**
+     * Cannot be instantiated.
+     */
+    private Main() {
+    }
+
+    /**
+     * Application main entry point.
+     * @param args command line arguments.
+     */
+    public static void main(String[] args) {
+
+        // load logging configuration
+        LogConfig.configureRuntime();
+
+        // initialize global config from default configuration
+        Config config = Config.create();
+        Config.global(config);
+
+        // Default port uses the fixed limiter
+        // "aimd" port uses the AIMD limiter
+        WebServerConfig webserverConfig = WebServer.builder()
+                .config(config.get("server"))
+                .routing(Main::fixedRouting)
+                .routing("aimd", Main::aimdRouting)
+                .buildPrototype();
+
+        WebServer webserver = webserverConfig.build().start();
+
+        LOGGER.log(System.Logger.Level.INFO, "WEB server is up! http://localhost:" + webserver.port() + "/fixed/sleep"
+                + " " + "http://localhost:" + webserver.port("aimd") + "/aimd/sleep");
+    }
+
+    /**
+     * Updates HTTP Routing.
+     */
+    static void fixedRouting(HttpRouting.Builder routing) {
+        routing.register("/fixed", new SleepService());
+    }
+
+    static void aimdRouting(HttpRouting.Builder routing) {
+        routing.register("/aimd", new SleepService());
+    }
+}

examples/webserver/concurrency-limits/src/main/java/io/helidon/examples/webserver/concurrencylimits/SleepService.java

@@ -0,0 +1,63 @@
+/*
+ * Copyright (c) 2024 Oracle and/or its affiliates.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package io.helidon.examples.webserver.concurrencylimits;
+
+import io.helidon.webserver.http.HttpRules;
+import io.helidon.webserver.http.HttpService;
+import io.helidon.webserver.http.ServerRequest;
+import io.helidon.webserver.http.ServerResponse;
+
+class SleepService  implements HttpService {
+
+    private static final System.Logger LOGGER = System.getLogger(SleepService.class.getName());
+
+    SleepService() {
+    }
+
+    @Override
+    public void routing(HttpRules rules) {
+        rules.get("/sleep/{seconds}", this::sleepHandler);
+    }
+
+    /**
+     * Sleep for a specified number of seconds.
+     * The optional path parameter controls the number of seconds to sleep. Defaults to 1
+     *
+     * @param request  server request
+     * @param response server response
+     */
+    private void sleepHandler(ServerRequest request, ServerResponse response) {
+        int seconds = request.path().pathParameters().first("seconds").asInt().orElse(1);
+        response.send(String.valueOf(sleep(seconds)));
+    }
+
+    /**
+     * Sleep current thread.
+     *
+     * @param seconds number of seconds to sleep
+     * @return number of seconds requested to sleep
+     */
+    private static int sleep(int seconds) {
+        LOGGER.log(System.Logger.Level.DEBUG, Thread.currentThread() + ": Sleeping for " + seconds + " seconds");
+        try {
+            Thread.sleep(seconds * 1_000L);
+        } catch (InterruptedException e) {
+            LOGGER.log(System.Logger.Level.WARNING, e);
+        }
+        return seconds;
+    }
+}

examples/webserver/concurrency-limits/src/main/java/io/helidon/examples/webserver/concurrencylimits/package-info.java

@@ -0,0 +1,17 @@
+/*
+ * Copyright (c) 2024 Oracle and/or its affiliates.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package io.helidon.examples.webserver.concurrencylimits;