Skip to content

Commit

Permalink
Merge pull request #206 from fabiobrz/feat.k8s-take5
Browse files Browse the repository at this point in the history
PoC for Kubernetes provisioning support (via OLM)
  • Loading branch information
tommaso-borgato authored Nov 25, 2024
2 parents 9fb8400 + e3537fe commit c160b26
Show file tree
Hide file tree
Showing 256 changed files with 6,747 additions and 3,262 deletions.
76 changes: 76 additions & 0 deletions .github/workflows/kubernetes-e2e.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
name: Kubernetes E2E Tests

on:
workflow_dispatch:
push:
branches:
- main
pull_request:
paths-ignore:
- 'docs/**'
- 'ide-config/**'
- '**.md'

concurrency:
# Only run once for latest commit per ref and cancel other (previous) runs.
group: ci-e2e-intersmash-${{ github.ref }}
cancel-in-progress: true

jobs:
minikube:
name: K8S
runs-on: ubuntu-latest
if: github.repository == 'Intersmash/intersmash'
strategy:
fail-fast: false
matrix:
kubernetes: [v1.31.0] #, v1.30.0]
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Start minikube
uses: medyagh/setup-minikube@latest
with:
kubernetes-version: ${{ matrix.kubernetes }}
- name: Install operator-sdk
run: |
set -x
#
# operator-sdk must be installed manually on K8s
#
export ARCH=$(case $(uname -m) in x86_64) echo -n amd64 ;; aarch64) echo -n arm64 ;; *) echo -n $(uname -m) ;; esac)
export OS=$(uname | awk '{print tolower($0)}')
export OPERATOR_SDK_DL_URL=https://github.com/operator-framework/operator-sdk/releases/download/v1.37.0
curl -LO ${OPERATOR_SDK_DL_URL}/operator-sdk_${OS}_${ARCH}
chmod +x operator-sdk_${OS}_${ARCH} && sudo mv operator-sdk_${OS}_${ARCH} /usr/local/bin/operator-sdk
operator-sdk olm install
- name: Set up JDK 17
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: 'temurin'
cache: 'maven'
- name: Install and Run Integration Tests
run: |
set -x
#
mkdir local-repo
# pre-fetch dependencies
mvn clean install -pl kubernetes-client,core,provisioners,testsuite,testsuite/deployments,testsuite/deployments/deployments-provider -am --batch-mode -Dmaven.repo.local=./local-repo -DskipTests
# run K8s integration tests
mvn test -e --batch-mode -fae -Dmaven.repo.local=./local-repo -pl testsuite/integration-tests -Pts.k8s \
-Dintersmash.kubernetes.url=https://$(minikube ip):8443 \
-Dintersmash.kubernetes.route_domain=$(minikube ip) \
-Dintersmash.kubernetes.hostname=$(minikube ip) \
-Dintersmash.kubernetes.namespace=test-k8s-1 \
-Dintersmash.olm.operators.catalog_source=operatorhubio-catalog \
-Dintersmash.olm.operators.namespace=olm \
-Dintersmash.hyperfoil.operators.catalog_source=operatorhubio-catalog
- name: Upload artifacts
uses: actions/upload-artifact@v4
if: failure()
with:
name: logs-${{ matrix.kubernetes }}
path: |
**/log/*
**/target/surefire-reports/*
7 changes: 4 additions & 3 deletions .github/workflows/simple-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,17 @@ jobs:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v3
- name: Set up Maven
uses: stCarolas/[email protected]
with:
maven-version: 3.8.7
- uses: actions/checkout@v3
- name: Set up JDK 17
uses: actions/setup-java@v3
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: 'temurin'
cache: 'maven'
- name: Build with Maven
run: mvn -version ; mvn clean install -U -B -DskipTests ; mvn test -pl testsuite/deployments/deployments-provider,core,provisioners -am
run: mvn -version ; mvn clean install -U -B -DskipTests ; mvn test -B -pl testsuite/deployments/deployments-provider,kubernetes-client,core,provisioners -am
20 changes: 13 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,11 @@
Intersmash is a Java library that makes it easy to automate the
provisioning and execution of tests in cloud-native environments.
It helps the user prototype and test complex interoperability scenarios on
kubernetes compliant cloud-native environments and platforms, most notably OpenShift. (*Other Kubernetes implementations will be supported in the future.*)
kubernetes compliant cloud-native environments and platforms, most notably OpenShift.
Kubernetes support is being introduced. Support for other platforms (Bare-metal etc.) could be added
later on demand.



Intersmash is designed with these principles.
* It is integrated with JUnit.
Expand All @@ -16,7 +20,9 @@ the user can easily create an implementation and integrate it into the framework
to test on different Kubernetes compliant cloud implementations. This will
ensure application portability.

For usage examples, see [Getting Started](./docs/GETTING_STARTED.md).
## Next steps
- See how to [configure Intersmash for running tests on OpenShift and Kubernetes, or both](./docs/CLOUD_TARGET.md).
- For usage examples, see [Getting Started](./docs/GETTING_STARTED.md).

## Framework Design

Expand Down Expand Up @@ -145,8 +151,8 @@ Feel free to submit an issue in such a case, Intersmash welcomes community contr
| Infinispan | 15.0.5.Final | Or _default_ provided by the default Infinispan Operator `stable` channel |
| Red Hat Data Grid | 8.5.0.GA | Or _default_ provided by the Red Hat DataGrid Operator `stable` channel |
| | | |
| Kafka provided by Strimzi | 3.6.0 | Provided by the Strimzi Operator `stable` channel |
| Red Hat AMQ Streams | 3.6.0.redhat-00005 | Or _default_, as provided by the Red Hat AMQ Streams Operator `stable` channel |
| Kafka provided by Strimzi | 3.8.0 | Provided by the Strimzi Operator `stable` channel |
| Red Hat AMQ Streams | 3.8.0.redhat-00007 | Or _default_, as provided by the Red Hat AMQ Streams Operator `stable` channel |
| | | |
| Keycloak | 24.0.3 | Or _default_, as provided by default by the Keycloak Operator `fast` channel |
| Red Hat Build of keycloak (RHBK) | 24.0.3.redhat-00004 | Or _latest_ in the `:24` tag image stream, see registry.redhat.io/rhbk/keycloak-rhel9 |
Expand All @@ -156,7 +162,7 @@ Feel free to submit an issue in such a case, Intersmash welcomes community contr
| Red Hat JBoss EAP 8 | JBoss EAP 8.0.x (and XP 5.x) | |
| Red Hat JBoss EAP 7 | JBoss EAP 7.4.z (and XP 4.z) | |
| | | |
| Hyperfoil | 0.24.2 | |
| Hyperfoil | 0.24.2 | Supports provisioning via the Operator, both on **Kubernetes** and **OpenShift** |

Since multiple deliverables can be bound to a given service version, e.g.: container images, operator CRs, or Helm Charts,
more information can be found in [the provisioners' documentation](./provisioners/README.md), or in the resources there linked.
Expand All @@ -169,8 +175,8 @@ We welcome community contributions to other Kubernetes implementations.


## Future goals
* K8s support and Quarkus provisioning
* documentation and examples enhancements
* OpenShift AI provisioning
* Documentation and examples enhancements

see the [current milestone](https://github.com/Intersmash/intersmash/milestone/2) for a complete list of issues.

Expand Down
5 changes: 5 additions & 0 deletions core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,11 @@
<artifactId>http-client</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.jboss.intersmash</groupId>
<artifactId>intersmash-kubernetes-client</artifactId>
</dependency>
</dependencies>

<build>
Expand Down
25 changes: 15 additions & 10 deletions core/src/main/java/org/jboss/intersmash/IntersmashConfig.java
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,15 @@ public class IntersmashConfig {
private static final String DEPLOYMENTS_REPOSITORY_REF = "intersmash.deployments.repository.ref";

// Default Catalog for Operators
private static final String DEFAULT_OPERATOR_CATALOG_SOURCE_NAMESPACE = "openshift-marketplace";
private static final String KUBERNETES_OPERATOR_CATALOG_SOURCE_NAMESPACE = "olm";
private static final String OPENSHIFT_OPERATOR_CATALOG_SOURCE_NAMESPACE = "openshift-marketplace";
private static final String DEFAULT_OPERATOR_CATALOG_SOURCE_NAMESPACE = KUBERNETES_OPERATOR_CATALOG_SOURCE_NAMESPACE;
private static final String REDHAT_OPERATOR_CATALOG_SOURCE_NAME = "redhat-operators";
private static final String COMMUNITY_OPERATOR_CATALOG_SOURCE_NAME = "community-operators";
private static final String OPERATORHUB_IO_OPERATOR_CATALOG_SOURCE_NAME = "operatorhubio-catalog";
private static final String DEFAULT_OPERATOR_CATALOG_SOURCE_NAME = COMMUNITY_OPERATOR_CATALOG_SOURCE_NAME;
private static final String OLM_OPERATOR_CATALOG_SOURCE_NAME = "intersmash.olm.operators.catalog_source";
private static final String OLM_OPERATOR_CATALOG_SOURCE_NAMESPACE = "intersmash.olm.operators.namespace";

// Custom Catalogs for operators
private static final String INFINISPAN_OPERATOR_CATALOG_SOURCE_NAME = "intersmash.infinispan.operators.catalog_source";
Expand Down Expand Up @@ -125,15 +130,15 @@ public static String[] getKnownCatalogSources() {
}

public static String defaultOperatorCatalogSourceName() {
return DEFAULT_OPERATOR_CATALOG_SOURCE_NAME;
return XTFConfig.get(OLM_OPERATOR_CATALOG_SOURCE_NAME, DEFAULT_OPERATOR_CATALOG_SOURCE_NAME);
}

public static String defaultOperatorCatalogSourceNamespace() {
return DEFAULT_OPERATOR_CATALOG_SOURCE_NAMESPACE;
return XTFConfig.get(OLM_OPERATOR_CATALOG_SOURCE_NAMESPACE, DEFAULT_OPERATOR_CATALOG_SOURCE_NAMESPACE);
}

public static String infinispanOperatorCatalogSource() {
return XTFConfig.get(INFINISPAN_OPERATOR_CATALOG_SOURCE_NAME, DEFAULT_OPERATOR_CATALOG_SOURCE_NAME);
return XTFConfig.get(INFINISPAN_OPERATOR_CATALOG_SOURCE_NAME, defaultOperatorCatalogSourceName());
}

public static String infinispanOperatorIndexImage() {
Expand All @@ -149,7 +154,7 @@ public static String infinispanOperatorPackageManifest() {
}

public static String rhSsoOperatorCatalogSource() {
return XTFConfig.get(RHSSO_OPERATOR_CATALOG_SOURCE_NAME, DEFAULT_OPERATOR_CATALOG_SOURCE_NAME);
return XTFConfig.get(RHSSO_OPERATOR_CATALOG_SOURCE_NAME, defaultOperatorCatalogSourceName());
}

public static String rhSsoOperatorIndexImage() {
Expand All @@ -165,7 +170,7 @@ public static String rhSsoOperatorPackageManifest() {
}

public static String wildflyOperatorCatalogSource() {
return XTFConfig.get(WILDFLY_OPERATOR_CATALOG_SOURCE_NAME, DEFAULT_OPERATOR_CATALOG_SOURCE_NAME);
return XTFConfig.get(WILDFLY_OPERATOR_CATALOG_SOURCE_NAME, defaultOperatorCatalogSourceName());
}

public static String wildflyOperatorIndexImage() {
Expand All @@ -181,7 +186,7 @@ public static String wildflyOperatorPackageManifest() {
}

public static String kafkaOperatorCatalogSource() {
return XTFConfig.get(KAFKA_OPERATOR_CATALOG_SOURCE_NAME, DEFAULT_OPERATOR_CATALOG_SOURCE_NAME);
return XTFConfig.get(KAFKA_OPERATOR_CATALOG_SOURCE_NAME, defaultOperatorCatalogSourceName());
}

public static String kafkaOperatorIndexImage() {
Expand All @@ -197,7 +202,7 @@ public static String kafkaOperatorPackageManifest() {
}

public static String activeMQOperatorCatalogSource() {
return XTFConfig.get(ACTIVEMQ_OPERATOR_CATALOG_SOURCE_NAME, DEFAULT_OPERATOR_CATALOG_SOURCE_NAME);
return XTFConfig.get(ACTIVEMQ_OPERATOR_CATALOG_SOURCE_NAME, defaultOperatorCatalogSourceName());
}

public static String activeMQOperatorIndexImage() {
Expand All @@ -213,7 +218,7 @@ public static String activeMQOperatorPackageManifest() {
}

public static String hyperfoilOperatorCatalogSource() {
return XTFConfig.get(HYPERFOIL_OPERATOR_CATALOG_SOURCE_NAME, COMMUNITY_OPERATOR_CATALOG_SOURCE_NAME);
return XTFConfig.get(HYPERFOIL_OPERATOR_CATALOG_SOURCE_NAME, defaultOperatorCatalogSourceName());
}

public static String hyperfoilOperatorIndexImage() {
Expand Down Expand Up @@ -388,7 +393,7 @@ public static String getWildflyHelmChartsBranch() {
}

public static String keycloakOperatorCatalogSource() {
return XTFConfig.get(KEYCLOAK_OPERATOR_CATALOG_SOURCE_NAME, DEFAULT_OPERATOR_CATALOG_SOURCE_NAME);
return XTFConfig.get(KEYCLOAK_OPERATOR_CATALOG_SOURCE_NAME, defaultOperatorCatalogSourceName());
}

public static String keycloakOperatorIndexImage() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.jboss.intersmash.application.openshift;
package org.jboss.intersmash.application.k8s;

import java.util.List;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.jboss.intersmash.provision.openshift;
package org.jboss.intersmash.application.k8s;

import java.util.List;

Expand All @@ -25,8 +25,8 @@
public interface HasPods {

/**
* Return pods of the application.
* @return a list of related {@link Pod} instances
* Get the provisioned application service related Pods
* @return A list of related {@link Pod} instances
*/
List<Pod> getPods();
}
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.jboss.intersmash.application.openshift;
package org.jboss.intersmash.application.k8s;

import java.util.List;

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
/**
* Copyright (C) 2023 Red Hat, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.jboss.intersmash.application.k8s;

import org.jboss.intersmash.application.Application;

/**
* Interface representing the Application on Kubernetes.
*
* This interface is not supposed to be implemented by user Applications. See the "Mapping of implemented provisioners"
* section of Intersmash README.md file for the up-to-date list of supported end users Applications.
*/
public interface KubernetesApplication extends Application {
}
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,23 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.jboss.intersmash.application.openshift;
package org.jboss.intersmash.application.operator;

import java.util.Collections;
import java.util.List;

import org.jboss.intersmash.application.Application;
import org.jboss.intersmash.application.k8s.HasConfigMaps;
import org.jboss.intersmash.application.k8s.HasSecrets;

import io.fabric8.kubernetes.api.model.ConfigMap;
import io.fabric8.kubernetes.api.model.Secret;

/**
* This interface is not supposed to be implemented by user Applications. See the "Mapping of implemented provisioners"
* section of Intersmash README.md file for the up-to-date list of supported end users Applications.
*/
public interface OperatorApplication extends OpenShiftApplication, HasSecrets, HasConfigMaps {
public interface OperatorApplication extends Application, HasSecrets, HasConfigMaps {

@Override
default List<Secret> getSecrets() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
import org.jboss.intersmash.IntersmashConfig;
import org.jboss.intersmash.annotations.Intersmash;
import org.jboss.intersmash.annotations.Service;
import org.jboss.intersmash.application.openshift.OperatorApplication;
import org.jboss.intersmash.application.operator.OperatorApplication;
import org.junit.jupiter.api.extension.ConditionEvaluationResult;
import org.junit.jupiter.api.extension.ExecutionCondition;
import org.junit.jupiter.api.extension.ExtensionContext;
Expand All @@ -46,6 +46,7 @@ public ConditionEvaluationResult evaluateExecutionCondition(ExtensionContext con
if (intersmashes.length > 0) {
intersmash = intersmashes[0];
log.debug("Running: {}", context.getRequiredTestClass().getSimpleName());
// evaluate peculiar OpenShift/Kubernetes requirements
if (IntersmashConfig.isOcp3x(OpenShifts.admin())
&& Arrays.stream(intersmash.value()).anyMatch(isOperatorApplication)) {
return ConditionEvaluationResult.disabled("OLM is not available on OCP 3.x clusters, " +
Expand Down
Loading

0 comments on commit c160b26

Please sign in to comment.