Feature/s3 init docs (#66)

* 🔥 s3-client rm url override * 🔥 s3-client rm url override * 🎨 s3-client format * ✏️ s3-integration client starter fix method naming * 🎨 s3-integration format * ♻️ s3-integration extract rest * 🎨 s3-integration client fix pom order * ♻️ s3-integration service to rest-service * ♻️ s3-integration extract rest from client * ♻️ s3-integration rename client core module * ♻️ s3-integration client move java doc to interfaces * ♻️ s3-integration client move java doc to interfaces format * ♻️ s3 rest client refactor interface naming * 🎨 s3 client core format * ♻️ s3 mv rest to submodule * ♻️ s3 client fix reactive return mixed * ♻️ s3 client fix reactive return mixed * ♻️ s3 client simplify duplicate code DocumentStorageFileRepository * ✨ s3 init java client * ✨ s3 init java client starter * ♻️ s3 java client refactor interface naming * ♻️ s3 client rename FileService to FileValidationService * 🔥 s3 java client rm unused properties * 🐛 s3 rest client fix DocumentStorageFileRestRepository naming * ♻️ s3 mv java to submodule * 🔥 s3 rm unneeded pom comments * ✨ s3 rest client define default oauth client properties * ♻️ s3 rest service rename sso issuer url property * 📝 integrations init docs * 📝 s3 add usage to README * Update refarch-integrations/refarch-s3-integration/README.md Co-authored-by: Tobias Stadler <[email protected]> * 📝 s3 README rm muenchen references --------- Co-authored-by: Tobias Stadler <[email protected]>
it-at-m · Aug 14, 2024 · a3022e6 · a3022e6
1 parent c33bdb3
commit a3022e6
Show file tree

Hide file tree

Showing 8 changed files with 137 additions and 3 deletions.
diff --git a/README.md b/README.md
@@ -12,6 +12,7 @@ As counterpart there are also [refarch-templates](https://github.com/it-at-m/ref
 The usage of the different components is described in their corresponding README.md
 
 - [refarch-gateway](./refarch-gateway/README.md)
+- [refarch-integrations](./refarch-integrations/README.md)
 
 ## Contributing
 

diff --git a/refarch-integrations/README.md b/refarch-integrations/README.md
@@ -0,0 +1,16 @@
+# RefArch integrations
+
+Collection of different integration which can be used as is in RefArch projects.
+
+## Integrations
+
+- [s3-integration](./refarch-s3-integration/README.md): For CRUD operations on a s3 storage. Also used for file handling
+  in other integrations.
+
+## Naming conventions
+
+The different submodules of integrations follow following naming conventions:
+
+- `*-core`: Implementation of the base functionality of the integration in hexagonal architecture.
+- `*-starter`: Provides Beans of core services.
+- `*-service`: A Spring application using the starter. Is provided as container image. Should not be used as dependency.
diff --git a/refarch-integrations/refarch-s3-integration/README.md b/refarch-integrations/refarch-s3-integration/README.md
@@ -0,0 +1,70 @@
+# RefArch S3 integration
+
+Integration for CRUD operations on a s3 storage. Also used for file handling in other integrations.
+
+## Modules
+
+The S3 integration follows the [default naming conventions](../README.md#naming-conventions).
+
+Beside the default integration it contains different client libraries for accessing the integration. The client
+libraries are especially provided for usage in other integrations.
+
+- REST: The rest client uses the rest endpoints of the s3-rest-service to manage data in s3.
+- Java: The Java client directly uses the in ports of the s3-core.
+
+## Usage
+
+```xml
+
+<dependencies>
+    <!-- REST -->
+    <!-- requires refarch-s3-integration-rest-service -->
+    <dependency>
+        <groupId>de.muenchen.refarch</groupId>
+        <artifactId>refarch-s3-integration-rest-starter</artifactId>
+        <version>...</version>
+    </dependency>
+    <!-- or Java -->
+    <dependency>
+        <groupId>de.muenchen.refarch</groupId>
+        <artifactId>refarch-s3-integration-java-starter</artifactId>
+        <version>...</version>
+    </dependency>
+</dependencies>
+```
+
+## Configuration
+
+Following are the properties to configure the different modules. Some of them are custom defined and others are synonyms
+for spring package properties.
+Whether a property is an alias can be checked in the corresponding `application.yml` of each module.
+
+### s3-integration-rest-service
+
+| Property                          | Description                                                    | Example                                       |
+|-----------------------------------|----------------------------------------------------------------|-----------------------------------------------|
+| `refarch.s3.url`                  | Url of s3 endpoint to connect to.                              | `s3.example.com`                              |
+| `refarch.s3.bucket-name`          | Name of the bucket to connect to.                              | `refarch-bucket`                              |
+| `refarch.s3.access-key`           | Access key to use for connection.                              |                                               |
+| `refarch.s3.secret-key`           | Secret key to use for connection.                              |                                               |
+| `refarch.security.sso-issuer-url` | Issuer url of oAuth2 service used for securing rest endpoints. | `https://sso.example.com/auth/realms/refarch` |
+
+### s3-integration-java-client-starter
+
+| Property                                                   | Description                                           | Example                  |
+|------------------------------------------------------------|-------------------------------------------------------|--------------------------|
+| `refarch.s3.client.max-file-size` (optional)               | Single file limit for up- or downloading in byte.     | `10MB`                   |
+| `refarch.s3.client.max-batch-size` (optional)              | Limit for up- or downloading a list of files in byte. | `100MB`                  |
+| `refarch.s3.client.supported-file-extensions.*` (optional) | Map of allowed file extensions for up- and download.  | `pdf: "application/pdf"` |
+
+### s3-integration-rest-client-starter
+
+All properties of [s3-integration-java-client-starter](#s3-integration-rest-client-starter) and following:
+
+| Property                                 | Description                                                                | Example                                       |
+|------------------------------------------|----------------------------------------------------------------------------|-----------------------------------------------|
+| `refarch.s3.client.document-storage-url` | Url to the RefArch S3 integration service.                                 | `http://s3-integration-service:8080`          |
+| `refarch.s3.client.enable-security`      | Switch to enable or disable oAuth2 authentication against s3 service.      | `true`                                        |
+| `refarch.security.sso-issuer-url`        | Issuer url of oAuth2 service to use for authentication against s3 service. | `https://sso.example.com/auth/realms/refarch` |
+| `refarch.s3.client.client-id`            | Client id to be used for authentication.                                   | `refarch_client`                              |
+| `refarch.s3.client.client-secret`        | Client secret to be used for gathering client service account token.       |                                               |
diff --git a/...hen/refarch/integration/s3/client/configuration/S3IntegrationClientAutoConfiguration.java b/...hen/refarch/integration/s3/client/configuration/S3IntegrationClientAutoConfiguration.java
@@ -4,6 +4,7 @@
 import de.muenchen.refarch.integration.s3.client.api.FileApiApi;
 import de.muenchen.refarch.integration.s3.client.api.FolderApiApi;
 import de.muenchen.refarch.integration.s3.client.domain.model.SupportedFileExtensions;
+import de.muenchen.refarch.integration.s3.client.factory.YamlPropertySourceFactory;
 import de.muenchen.refarch.integration.s3.client.properties.S3IntegrationClientProperties;
 import de.muenchen.refarch.integration.s3.client.service.FileValidationService;
 import jakarta.annotation.PostConstruct;
@@ -16,10 +17,12 @@
 import org.springframework.context.annotation.Bean;
 import org.springframework.context.annotation.ComponentScan;
 import org.springframework.context.annotation.Configuration;
+import org.springframework.context.annotation.PropertySource;
 import org.springframework.security.oauth2.client.AuthorizedClientServiceOAuth2AuthorizedClientManager;
 import org.springframework.security.oauth2.client.OAuth2AuthorizedClientService;
 import org.springframework.security.oauth2.client.registration.ClientRegistrationRepository;
 import org.springframework.security.oauth2.client.web.reactive.function.client.ServletOAuth2AuthorizedClientExchangeFilterFunction;
+import org.springframework.util.ResourceUtils;
 import org.springframework.web.reactive.function.client.WebClient;
 
 @Configuration
@@ -30,6 +33,7 @@
 )
 @RequiredArgsConstructor
 @EnableConfigurationProperties(S3IntegrationClientProperties.class)
+@PropertySource(value = ResourceUtils.CLASSPATH_URL_PREFIX + "application.yml", factory = YamlPropertySourceFactory.class)
 @Slf4j
 public class S3IntegrationClientAutoConfiguration {
 

diff --git a/...ain/java/de/muenchen/refarch/integration/s3/client/factory/YamlPropertySourceFactory.java b/...ain/java/de/muenchen/refarch/integration/s3/client/factory/YamlPropertySourceFactory.java
@@ -0,0 +1,25 @@
+package de.muenchen.refarch.integration.s3.client.factory;
+
+import java.io.IOException;
+import java.util.Objects;
+import java.util.Properties;
+import org.springframework.beans.factory.config.YamlPropertiesFactoryBean;
+import org.springframework.core.env.PropertiesPropertySource;
+import org.springframework.core.env.PropertySource;
+import org.springframework.core.io.support.EncodedResource;
+import org.springframework.core.io.support.PropertySourceFactory;
+import org.springframework.lang.NonNull;
+
+public class YamlPropertySourceFactory implements PropertySourceFactory {
+
+    @Override
+    @NonNull
+    public PropertySource<?> createPropertySource(String name, EncodedResource resource) throws IOException {
+        YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean();
+        factory.setResources(resource.getResource());
+        Properties properties = factory.getObject();
+        return new PropertiesPropertySource(
+                Objects.requireNonNull(resource.getResource().getFilename()),
+                Objects.requireNonNull(properties));
+    }
+}
diff --git a/...ration-rest/refarch-s3-integration-rest-client-starter/src/main/resources/application.yml b/...ration-rest/refarch-s3-integration-rest-client-starter/src/main/resources/application.yml
@@ -0,0 +1,17 @@
+spring:
+  security:
+    oauth2:
+      client:
+        provider:
+          keycloak:
+            issuer-uri: ${refarch.security.sso-issuer-url}
+            user-info-uri: ${refarch.security.sso-issuer-url}/protocol/openid-connect/userinfo
+            jwk-set-uri: ${refarch.security.sso-issuer-url}/protocol/openid-connect/certs
+            user-name-attribute: user_name
+        registration:
+          s3:
+            provider: keycloak
+            authorization-grant-type: client_credentials
+            client-id: ${refarch.s3.client.client-id}
+            client-secret: ${refarch.s3.client.client-secret}
+            scope: email, profile, openid  # needed for userInfo endpoint
diff --git a/...gration-rest/refarch-s3-integration-rest-service/src/main/resources/application-local.yml b/...gration-rest/refarch-s3-integration-rest-service/src/main/resources/application-local.yml
@@ -1,5 +1,6 @@
-SSO_ISSUER_URL: http://keycloak:8100/auth/realms/local_realm
 refarch:
+  security:
+    sso-issuer-url: http://keycloak:8100/auth/realms/local_realm
   s3:
     bucket-name: test-bucket
     access-key: minio

diff --git a/...3-integration-rest/refarch-s3-integration-rest-service/src/main/resources/application.yml b/...3-integration-rest/refarch-s3-integration-rest-service/src/main/resources/application.yml
@@ -10,7 +10,7 @@ spring:
     oauth2:
       resourceserver:
         jwt:
-          issuer-uri: ${SSO_ISSUER_URL}
+          issuer-uri: ${refarch.security.sso-issuer-url}
 
 server:
   error:
@@ -36,7 +36,7 @@ management:
 
 security:
   oauth2:
-    resource.user-info-uri: ${SSO_ISSUER_URL}/protocol/openid-connect/userinfo
+    resource.user-info-uri: ${refarch.security.sso-issuer-url}/protocol/openid-connect/userinfo
 
 refarch:
   s3: