In-source behave tests from OpenJDK for regression suite (cekit#61)

* In-source behave tests from OpenJDK for regression suite The GHA job up until now relied upon running the externally-defined behave tests for the OpenJDK container images (runtime variant). The external test suite is primarily designed to cover the OpenJDK image, and not to provide coverage of all of the behave-test-steps. Rework the job to operate on a local copy of these behave tests. (Also do a bit of spring-cleaning) Future work can adjust the feature files to provide better coverage for the steps defined in this repository. Signed-off-by: Jonathan Dowland <[email protected]> * Test to exercise 'File . should contain' See also PR cekit#60. Signed-off-by: Jonathan Dowland <[email protected]> --------- Signed-off-by: Jonathan Dowland <[email protected]>
jmtd · Nov 21, 2024 · 3442125 · 3442125
1 parent 85739e6
commit 3442125
Show file tree

Hide file tree

Showing 18 changed files with 621 additions and 63 deletions.
diff --git a/.github/workflows/ubi8-openjdk-17-runtime.yml b/.github/workflows/ubi8-openjdk-17-runtime.yml
diff --git a/.github/workflows/ubi9-openjdk-17-runtime.yml b/.github/workflows/ubi9-openjdk-17-runtime.yml
@@ -0,0 +1,40 @@
+name: run behave-test-steps regression tests
+on:
+  push:
+    branches: [ v1 ]
+  pull_request:
+    branches: [ v1 ]
+env:
+  OPENJDK_CONTAINER_SOURCE_TAG: ubi9-openjdk-containers-1.18
+  OPENJDK_CONTAINER:            ubi9/openjdk-17-runtime
+  OPENJDK_CONTAINER_VERSION:    1.18
+  OPENJDK_CONTAINER_DESCRIPTOR: ubi9-openjdk-17-runtime.yaml
+jobs:
+  openjdkci:
+    name: behave-test-steps regression tests
+    timeout-minutes: 60
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+    steps:
+      - uses: actions/checkout@v4
+      - name: fetch and tag openjdk runtime image
+        run: |
+          docker pull registry.access.redhat.com/${{ env.OPENJDK_CONTAINER }}:${{ env.OPENJDK_CONTAINER_VERSION }}
+          docker tag  registry.access.redhat.com/${{ env.OPENJDK_CONTAINER }}:${{ env.OPENJDK_CONTAINER_VERSION }} \
+            ${{ env.OPENJDK_CONTAINER }}:${{ env.OPENJDK_CONTAINER_VERSION }}
+
+      - name: Install CEKit
+        uses: cekit/[email protected]
+
+# even though we don't run any S2I tests for the runtime images, the test suite
+# will fail to start if the s2i binary is not present.
+      - name: mock s2i binary
+        run: sudo ln -s /usr/bin/echo /usr/bin/s2i
+
+      - name: Run behave tests
+        run: |
+          STEPSDIR=$(pwd)
+          git tag v1 # cekit hard-codes using ref v1
+          cd test
+          cekit -v test behave --steps-url file://$STEPSDIR
diff --git a/test/README.md b/test/README.md
@@ -0,0 +1,5 @@
+# Regression tests for behave-test-steps
+
+These features are used for regression testing behave-test-steps via GitHub
+Actions. Please see the `.github/workflows` directory to understand how they
+are applied.
diff --git a/test/image.yaml b/test/image.yaml
@@ -0,0 +1,3 @@
+# dummy descriptor, enough to start tests
+name: ubi9/openjdk-17-runtime
+version: 1.18
diff --git a/test/tests/features/file.feature b/test/tests/features/file.feature
@@ -0,0 +1,5 @@
+Feature: test has a shell redirection in it
+  Scenario: try running a container with redirection
+    Given container is started with command bash
+    Then run sh -c '/usr/local/s2i/run > /tmp/boot.log 2>&1' in container and detach
+    And  file /tmp/boot.log should contain java
diff --git a/test/tests/features/gc.feature b/test/tests/features/gc.feature
@@ -0,0 +1,61 @@
+@ubi9
+Feature: Openshift OpenJDK GC tests
+
+  Scenario: Check default GC configuration
+    Given container is started as uid 1000
+    Then container log should contain -XX:+UseParallelGC
+    And  container log should contain -XX:MinHeapFreeRatio=10
+    And  container log should contain -XX:MaxHeapFreeRatio=20
+    And  container log should contain -XX:GCTimeRatio=4
+
+  Scenario: Check GC_MIN_HEAP_FREE_RATIO GC configuration
+    Given container is started with env
+       | variable                         | value  |
+       | GC_MIN_HEAP_FREE_RATIO           | 5      |
+    Then container log should contain -XX:MinHeapFreeRatio=5
+
+  Scenario: Check GC_MAX_HEAP_FREE_RATIO GC configuration
+    Given container is started with env
+       | variable                         | value  |
+       | GC_MAX_HEAP_FREE_RATIO           | 50     |
+    Then container log should contain -XX:MaxHeapFreeRatio=50
+
+  Scenario: Check GC_TIME_RATIO GC configuration
+    Given container is started with env
+       | variable                         | value  |
+       | GC_TIME_RATIO                    | 5      |
+    Then container log should contain -XX:GCTimeRatio=5
+
+  Scenario: Check GC_ADAPTIVE_SIZE_POLICY_WEIGHT GC configuration
+    Given container is started with env
+       | variable                         | value  |
+       | GC_ADAPTIVE_SIZE_POLICY_WEIGHT   | 80     |
+    Then container log should contain -XX:AdaptiveSizePolicyWeight=80
+
+  Scenario: Check GC_MAX_METASPACE_SIZE GC configuration
+    Given container is started with env
+       | variable                 | value  |
+       | GC_MAX_METASPACE_SIZE    | 120    |
+    Then container log should contain -XX:MaxMetaspaceSize=120m
+
+  Scenario: Check GC_CONTAINER_OPTIONS configuration
+    Given container is started with env
+       | variable             | value        |
+       | GC_CONTAINER_OPTIONS | -XX:+UseG1GC |
+    Then container log should contain -XX:+UseG1GC
+    And container log should not contain -XX:+UseParallelGC
+
+  Scenario: Check GC_METASPACE_SIZE GC configuration
+    Given container is started with env
+       | variable             | value  |
+       | GC_METASPACE_SIZE    | 120    |
+    Then container log should contain -XX:MetaspaceSize=120m
+    And container log should not contain integer expression expected
+
+  Scenario: Check GC_METASPACE_SIZE constrained by GC_MAX_METASPACE_SIZE GC configuration
+    Given container is started with env
+       | variable                 | value |
+       | GC_METASPACE_SIZE        | 120   |
+       | GC_MAX_METASPACE_SIZE    | 90    |
+    Then container log should contain -XX:MaxMetaspaceSize=90m
+    And  container log should contain -XX:MetaspaceSize=90m
diff --git a/test/tests/features/general.feature b/test/tests/features/general.feature
@@ -0,0 +1,8 @@
+Feature: Miscellaneous general settings unit tests
+
+  @ubi9
+  Scenario: Check the attributes of /home/default using stat
+    When container is started with args
+    | arg     | value              |
+    | command | stat /home/default |
+    Then available container log should contain Access: (0770/drwxrwx---)
diff --git a/test/tests/features/java.runtime.feature b/test/tests/features/java.runtime.feature
@@ -0,0 +1,14 @@
+@ubi9/openjdk-11
+@ubi9/openjdk-17
+@ubi9/openjdk-21
+Feature: Openshift OpenJDK Runtime tests (OPENJDK-474)
+
+  Scenario: Ensure JAVA_ARGS is passed through, diagnostic options work correctly, JVM_ARGS not present in run script, OPENJDK-474 JAVA_ARGS not repeated
+    Given s2i build https://github.com/jboss-container-images/openjdk-test-applications from undertow-servlet
+       | variable         | value  |
+       | JAVA_ARGS        | unique |
+       | JAVA_DIAGNOSTICS | true   |
+    Then container log should contain /deployments/undertow-servlet.jar unique
+     And container log should contain -XX:NativeMemoryTracking=summary
+     And file /usr/local/s2i/run should not contain JVM_ARGS
+     And container log should not contain unique unique
diff --git a/test/tests/features/java.security.feature b/test/tests/features/java.security.feature
@@ -0,0 +1,7 @@
+@ubi9/openjdk-11
+@ubi9/openjdk-17
+@ubi9/openjdk-21
+Feature: Openshift S2I tests
+  Scenario: Check networkaddress.cache.negative.ttl has been set correctly
+    Given s2i build https://github.com/jboss-container-images/openjdk-test-applications from binary-cli-security-property
+    Then s2i build log should contain networkaddress.cache.negative.ttl=0