Merge pull request #39 from ArcGIS/issue14

#14 Redesigned test workflows
Esri · Apr 3, 2024 · 624c7d9 · 624c7d9
2 parents 75f37c9 + 0aa11a1
commit 624c7d9
Show file tree

Hide file tree

Showing 23 changed files with 254 additions and 147 deletions.
diff --git a/aws/README.md b/aws/README.md
@@ -38,7 +38,7 @@ Deployments configured to receive traffic from clients are referred to as *prima
 The following templates are available for AWS:
 
 * [arcgis-site-core](arcgis-site-core/README.md) - Provision core AWS resources for ArcGIS Enterprise site
-* [arcgis-enterprise-base-windows](arcgis-enterprise-base-windows/README.md) - Base ArcGIS Enterprise on Windows deployment operations
+* [arcgis-enterprise-base-windows](arcgis-enterprise-base-linux-windows/README.md) - Base ArcGIS Enterprise on Windows deployment operations
 * [arcgis-enterprise-base-linux](arcgis-enterprise-base-linux/README.md) - Base ArcGIS Enterprise on Linux deployment operations
 * [arcgis-enterprise-k8s](arcgis-enterprise-k8s/README.md) - ArcGIS Enterprise on Kubernetes deployment operations
 

diff --git a/aws/arcgis-enterprise-base-linux/workflows/enterprise-base-linux-aws-test.yaml b/aws/arcgis-enterprise-base-linux/workflows/enterprise-base-linux-aws-test.yaml
@@ -4,7 +4,6 @@ on:
   workflow_dispatch:
 
 env:
-  PYTHONPATH: ${{ github.workspace }}/aws/tests
   CONFIG_FILE: ${{ github.workspace }}/config/aws/arcgis-enterprise-base-linux/application.tfvars.json
 
 jobs:
@@ -13,7 +12,7 @@ jobs:
     runs-on: ubuntu-latest
     defaults:
       run:
-        working-directory: ${{ github.workspace }}/aws/tests
+        working-directory: ${{ github.workspace }}/enterprise-admin-cli
     permissions:
       contents: read
       pull-requests: write
@@ -22,20 +21,13 @@ jobs:
       uses: actions/checkout@v4
       with:
         ref: ${{ github.ref_name }}
-    - name: Setup Conda
-      uses: conda-incubator/setup-miniconda@v3
-      with:
-        auto-activate-base: true
-        activate-environment: ""    
-    - name: Install ArcGIS API for Python
-      id: install-arcgis
+    - name: Build Container Image
       run: |
-        conda install python=3.10 jupyter
-        conda install -c esri arcgis
-    - name: Run Test
-      id: test
+        docker build -t enterprise-admin-cli .
+    - name: Run Tests
       run: |
         DEPLOYMENT_FQDN=$(jq -r '.deployment_fqdn' $CONFIG_FILE)
+        ARCGIS_ENTERPRISE_CONTEXT=portal
         ADMIN_USERNAME=$(jq -r '.admin_username' $CONFIG_FILE)
         ADMIN_PASSWORD=$(jq -r '.admin_password' $CONFIG_FILE)
-        /usr/share/miniconda/bin/python arcgis-enterprise-base-test.py -a https://$DEPLOYMENT_FQDN/portal -u $ADMIN_USERNAME -p $ADMIN_PASSWORD
+        docker run enterprise-admin-cli python publish_csv.py -a https://$DEPLOYMENT_FQDN/$ARCGIS_ENTERPRISE_CONTEXT -u $ADMIN_USERNAME -p $ADMIN_PASSWORD
diff --git a/aws/arcgis-enterprise-base-windows/workflows/enterprise-base-windows-aws-test.yaml b/aws/arcgis-enterprise-base-windows/workflows/enterprise-base-windows-aws-test.yaml
@@ -4,7 +4,6 @@ on:
   workflow_dispatch:
 
 env:
-  PYTHONPATH: ${{ github.workspace }}/aws/tests
   CONFIG_FILE: ${{ github.workspace }}/config/aws/arcgis-enterprise-base-windows/application.tfvars.json
 
 jobs:
@@ -13,7 +12,7 @@ jobs:
     runs-on: ubuntu-latest
     defaults:
       run:
-        working-directory: ${{ github.workspace }}/aws/tests
+        working-directory: ${{ github.workspace }}/enterprise-admin-cli
     permissions:
       contents: read
       pull-requests: write
@@ -22,20 +21,13 @@ jobs:
       uses: actions/checkout@v4
       with:
         ref: ${{ github.ref_name }}
-    - name: Setup Conda
-      uses: conda-incubator/setup-miniconda@v3
-      with:
-        auto-activate-base: true
-        activate-environment: ""    
-    - name: Install ArcGIS API for Python
-      id: install-arcgis
+    - name: Build Container Image
       run: |
-        conda install python=3.10 jupyter
-        conda install -c esri arcgis
-    - name: Run Test
-      id: test
+        docker build -t enterprise-admin-cli .
+    - name: Run Tests
       run: |
         DEPLOYMENT_FQDN=$(jq -r '.deployment_fqdn' $CONFIG_FILE)
+        ARCGIS_ENTERPRISE_CONTEXT=portal
         ADMIN_USERNAME=$(jq -r '.admin_username' $CONFIG_FILE)
         ADMIN_PASSWORD=$(jq -r '.admin_password' $CONFIG_FILE)
-        /usr/share/miniconda/bin/python arcgis-enterprise-base-test.py -a https://$DEPLOYMENT_FQDN/portal -u $ADMIN_USERNAME -p $ADMIN_PASSWORD
+        docker run enterprise-admin-cli python publish_csv.py -a https://$DEPLOYMENT_FQDN/$ARCGIS_ENTERPRISE_CONTEXT -u $ADMIN_USERNAME -p $ADMIN_PASSWORD
diff --git a/aws/arcgis-enterprise-k8s/README.md b/aws/arcgis-enterprise-k8s/README.md
@@ -18,19 +18,15 @@ To enable the template's workflows, copy the .yaml files from the template's `wo
 
 ## Initial Deployment
 
-Initial deployment of ArcGIS Enterprise on Kubernetes includes: creating ingress controller creating ArcGIS Enterprise organization, and testing the deployment web services.
+Initial deployment of ArcGIS Enterprise on Kubernetes includes: provisioning container images, creating ingress controller creating ArcGIS Enterprise organization, and testing the deployment web services.
 
-> If pull through cache rules are not configured for Amazon ECR, The container images must be copied to the Amazon ECR private repositories.
+> The IAM principal used by the templates's workflows must have the EKS cluster administrator permissions. The IAM principal used to create the EKS cluster is granted the required permissions by site-k8s-cluster-aws workflow.
 
-> The IAM principal used by the templates's workflows must have the EKS cluster administrator permissions. The IAM principal used to create the EKS cluster has the cluster administrator permissions by default.
+### 1. Provisioning Container Images
 
-### 1. Copy Container Images to Amazon ECR
+GitHub Actions workflow **enterprise-k8s-aws-image** builds container image for [Enterprise Admin CLI](../../enterprise-admin-cli/README.md) and pushes it to private AWS Elastic Container Registry (ECR) repository. Optionally, if pull through cache is not enabled in the ECR, the workflow also copies container images of the ArcGIS Enterprise on Kubernetes version from DockerHub to the private ECR repositories.
 
-GitHub Actions workflow **enterprise-k8s-aws-image** copies the ArcGIS Enterprise on Kubernetes container images from DockerHub to private AWS Elastic Container Registry (ECR) repositories.
-
-In advance copying of images is not required if [pull through cache rules](https://docs.aws.amazon.com/AmazonECR/latest/userguide/pull-through-cache.html) are configured for Amazon ECR private repositories. But for some AWS regions, such as AWS GovCloud and China, pull through cache rules are not supported.
-
-The workflow uses [image-copy-ecr](image/README.md) script with [image.vars.json](../../config/aws/arcgis-enterprise-k8s/image.vars.json) config file.
+The workflow uses [shell scripts](image/README.md) with [image.vars.json](../../config/aws/arcgis-enterprise-k8s/image.vars.json) config file.
 
 Required IAM policies:
 
@@ -106,14 +102,12 @@ Instructions:
 
 GitHub Actions workflow **enterprise-k8s-aws-test** tests the ArcGIS Enterprise deployment.
 
-The python [test script](../tests/arcgis-enterprise-base-test.py) uses [ArcGIS API for Python](https://developers.arcgis.com/python/) to publish a CSV file to the Portal for ArcGIS URL. The portal domain name and admin credentials are retrieved from organization.tfvars.json properties file.
+The workflow uses publish-csv.py tool from the [Enterprise Admin CLI](../../enterprise-admin-cli/README.md) container image to test the deployment's service. The deployment FQDN and admin credentials are retrieved from [organization.tfvars.json](../../config/aws/arcgis-enterprise-k8s/organization.tfvars.json) config file.
 
 Instructions:
 
 1. Run enterprise-k8s-aws-test workflow using the branch.
 
-> Note: enterprise-k8s-aws-test requires the ArcGIS Enterprise deployment to be accessible from the GitHub Actions runner.
-
 ## Backups and Disaster Recovery
 
 TBD

diff --git a/aws/arcgis-enterprise-k8s/image/README.md b/aws/arcgis-enterprise-k8s/image/README.md
@@ -1,29 +1,38 @@
-# Copy Container Images from DockerHub to Amazon ECR
+# Scripts for Provisioning Container Images in Amazon ECR
 
 Amazon ECR is a fully managed container registry that provides a secure, scalable, and reliable registry for container images on AWS.
 
-Script image-copy-ecr copies the ArcGIS Enterprise on Kubernetes container images from Docker Hub to AWS Elastic Container Registry (ECR) repositories in specific AWS account and region.
-
 ## Requirements
 
-On the machine where the script is run, the following tools must be installed:
+On the machine where the scripts are run, the following tools must be installed:
 
 * [AWS CLI](https://aws.amazon.com/cli/)
 * [Docker](https://www.docker.com/)
 
 The AWS CLI must be configured with the appropriate AWS credentials and region must be set by AWS_DEFAULT_REGION environment variable.
 
-The DockerHub credentials must be set by environment variables `CONTAINER_REGISTRY_USER` and `CONTAINER_REGISTRY_PASSWORD`. The DockerHub container registry organization must be set by environment variable `CONTAINER_REGISTRY_ORG`.
+## build-admin-cli-image.sh
 
-`ECR_REPOSITORY_PREFIX` environment variable must be set to the prefix of the ECR repository name that matches the value "of ecr_repository_prefix" setting used for the k8s-cluster configuration.
+Builds container image for Enterprise Admin CLI and pushes it to private ECR repository in the AWS region.
+
+```bash
+chmod +x ./build-admin-cli-image.sh
+./build-admin-cli-image.sh <ECR repository name> <build context path>
+```
 
-The script requires at least 20GB of free disk space on the machine to temporary store the container images.
+## copy-docker-hub-images.sh
 
-## Usage
+Copies ArcGIS Enterprise for Kubernetes images from DockerHub registry to private Amazon ECR repositories in the AWS region.
+
+The DockerHub credentials must be set by environment variables `CONTAINER_REGISTRY_USER` and `CONTAINER_REGISTRY_PASSWORD`. The DockerHub container registry organization must be set by environment variable `CONTAINER_REGISTRY_ORG`.
+
+`ECR_REPOSITORY_PREFIX` environment variable must be set to the prefix of the ECR repository name that matches the value "of ecr_repository_prefix" setting used for the k8s-cluster configuration.
 
 ```bash
 chmod +x ./copy-docker-hub-images.sh
 ./copy-docker-hub-images.sh <manifest file path>
 ```
 
 The Esri-published version manifest is a JSON file that contains a list of images for a specific ArcGIS Enterprise on Kubernetes version, which are to be copied to ECR. This manifest file can be downloaded from a URL specified by the VERSION_MANIFEST_URL property in the `arcgis-enterprise/<version>/setup/.install/arcgis-enterprise/arcgis-enterprise.properties` file, located within the ArcGIS Enterprise on Kubernetes setup scripts for that particular version of ArcGIS Enterprise on Kubernetes.
+
+> The script requires at least 20GB of free disk space on the machine to temporary store the container images.
diff --git a/aws/arcgis-enterprise-k8s/image/build-admin-cli-image.sh b/aws/arcgis-enterprise-k8s/image/build-admin-cli-image.sh
@@ -0,0 +1,49 @@
+#!/bin/bash
+
+# This script builds container image for Enterprise Admin CLI and pushes it to
+# private ECR repository in the AWS region.
+#
+# On the machine where the script is executed:
+#
+# * AWS CLI and Docker must be installed
+# * AWS credentials must be configured for AWS CLI
+# * AWS region must be specified by AWS_DEFAULT_REGION environment variable
+
+set -e
+
+ECR_REPOSITORY_NAME=$1
+BUILD_CONTEXT_PATH=$2
+TAG=0.1.0
+
+AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
+ECR_REGISTRY_URL=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
+IMAGE_TAG=$ECR_REGISTRY_URL/$ECR_REPOSITORY_NAME:$TAG
+
+aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $ECR_REGISTRY_URL
+
+set +e
+
+aws ecr describe-repositories --repository-names $ECR_REPOSITORY_NAME
+
+# Create ECR repository if it does not exist   
+if [ $? -ne 0 ]
+then
+    aws ecr create-repository --repository-name $ECR_REPOSITORY_NAME --image-scanning-configuration scanOnPush=true --image-tag-mutability IMMUTABLE
+    echo "ECR repository '${ECR_REPOSITORY_NAME}' created."
+else
+    echo "ECR repository '${ECR_REPOSITORY_NAME}' already exists."
+fi    
+
+aws ecr describe-images --repository-name $ECR_REPOSITORY_NAME --image-ids imageTag=$TAG
+
+# Copy image to the ECR repository if it does not exist
+if [[ $? == 0 ]]; then
+    echo "Image $ECR_REPOSITORY_NAME:$TAG is already in the ECR repository"
+else
+    set -e
+
+    docker build -t $IMAGE_TAG $BUILD_CONTEXT_PATH
+    docker push $IMAGE_TAG
+
+    echo "Image $ECR_REPOSITORY_NAME:$TAG copied."
+fi
diff --git a/aws/arcgis-enterprise-k8s/image/copy-docker-hub-images.sh b/aws/arcgis-enterprise-k8s/image/copy-docker-hub-images.sh
@@ -1,7 +1,7 @@
 #!/bin/bash
 
 # This script copies ArcGIS Enterprise for Kubernetes images from DockerHub 
-# registry to Amazon ECR.
+# registry to private Amazon ECR repositories.
 #
 # Required environment variables:
 # CONTAINER_REGISTRY_ORG - The container registry organization

diff --git a/aws/arcgis-enterprise-k8s/image/copy-public-ecr-images.sh b/aws/arcgis-enterprise-k8s/image/copy-public-ecr-images.sh
diff --git a/aws/arcgis-enterprise-k8s/workflows/enterprise-k8s-aws-image.yaml b/aws/arcgis-enterprise-k8s/workflows/enterprise-k8s-aws-image.yaml
@@ -36,17 +36,22 @@ jobs:
           docker rmi node:16 node:18 node:20 > /dev/null 2>&1
       - name: Copy Docker Hub images
         run: |
-          ARCGIS_VERSION=$(jq -r '.arcgis_version' $CONFIG_FILE)
-          export CONTAINER_REGISTRY_ORG=$(jq -r '.container_registry_org' $CONFIG_FILE)
-          export ECR_REPOSITORY_PREFIX=$(jq -r '.ecr_repository_prefix' $CONFIG_FILE)
-          echo "ArcGIS Enterprise version: $ARCGIS_VERSION"
-          echo "Container registry org: $CONTAINER_REGISTRY_ORG"
-          echo "ECR repository prefix: $ECR_REPOSITORY_PREFIX"
-          MANIFEST_PATH=./manifests/$ARCGIS_VERSION.dat
-          chmod +x ./copy-docker-hub-images.sh
-          ./copy-docker-hub-images.sh $MANIFEST_PATH
-      - name: Copy Public ECR images
+          PULL_THROUGH_CACHE=$(jq -r '.pull_through_cache' $CONFIG_FILE)
+          if [ "$PULL_THROUGH_CACHE" == "true" ]; then
+            echo "Skip copying Docker Hub images because pull through cache is enabled."
+          else
+            ARCGIS_VERSION=$(jq -r '.arcgis_version' $CONFIG_FILE)
+            export CONTAINER_REGISTRY_ORG=$(jq -r '.container_registry_org' $CONFIG_FILE)
+            export ECR_REPOSITORY_PREFIX=$(jq -r '.ecr_repository_prefix' $CONFIG_FILE)
+            echo "ArcGIS Enterprise version: $ARCGIS_VERSION"
+            echo "Container registry org: $CONTAINER_REGISTRY_ORG"
+            echo "ECR repository prefix: $ECR_REPOSITORY_PREFIX"
+            MANIFEST_PATH=./manifests/$ARCGIS_VERSION.dat
+            chmod +x ./copy-docker-hub-images.sh
+            ./copy-docker-hub-images.sh $MANIFEST_PATH
+          fi
+      - name: Build Admin CLI Image
         run: |
-          chmod +x ./copy-public-ecr-images.sh
-          ./copy-public-ecr-images.sh
+          chmod +x ./build-admin-cli-image.sh
+          ./build-admin-cli-image.sh enterprise-admin-cli ${{ github.workspace }}/enterprise-admin-cli
   
diff --git a/aws/arcgis-enterprise-k8s/workflows/enterprise-k8s-aws-test.yaml b/aws/arcgis-enterprise-k8s/workflows/enterprise-k8s-aws-test.yaml
@@ -4,7 +4,9 @@ on:
   workflow_dispatch:
 
 env:
-  PYTHONPATH: ${{ github.workspace }}/aws/tests
+  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+  AWS_DEFAULT_REGION: ${{ vars.AWS_DEFAULT_REGION }}
   CONFIG_FILE: ${{ github.workspace }}/config/aws/arcgis-enterprise-k8s/organization.tfvars.json
 
 jobs:
@@ -13,7 +15,7 @@ jobs:
     runs-on: ubuntu-latest
     defaults:
       run:
-        working-directory: ${{ github.workspace }}/aws/tests
+        working-directory: ${{ github.workspace }}/enterprise-admin-cli
     permissions:
       contents: read
       pull-requests: write
@@ -22,21 +24,17 @@ jobs:
       uses: actions/checkout@v4
       with:
         ref: ${{ github.ref_name }}
-    - name: Setup Conda
-      uses: conda-incubator/setup-miniconda@v3
-      with:
-        auto-activate-base: true
-        activate-environment: ""    
-    - name: Install ArcGIS API for Python
-      id: install-arcgis
-      run: |
-        conda install python=3.10 jupyter
-        conda install -c esri arcgis
-    - name: Run Test
-      id: test
+    - name: Run Tests
       run: |
+        SITE_ID=$(jq -r '.site_id' $CONFIG_FILE)
         DEPLOYMENT_FQDN=$(jq -r '.deployment_fqdn' $CONFIG_FILE)
         ARCGIS_ENTERPRISE_CONTEXT=$(jq -r '.arcgis_enterprise_context' $CONFIG_FILE)
         ADMIN_USERNAME=$(jq -r '.admin_username' $CONFIG_FILE)
         ADMIN_PASSWORD=$(jq -r '.admin_password' $CONFIG_FILE)
-        /usr/share/miniconda/bin/python arcgis-enterprise-base-test.py -a https://$DEPLOYMENT_FQDN/$ARCGIS_ENTERPRISE_CONTEXT -u $ADMIN_USERNAME -p $ADMIN_PASSWORD
+        AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
+        ECR_REGISTRY_URL=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
+        ECR_REPOSITORY_NAME=enterprise-admin-cli
+        TAG=0.1.0
+        IMAGE_TAG=$ECR_REGISTRY_URL/$ECR_REPOSITORY_NAME:$TAG
+        aws eks update-kubeconfig --region $AWS_DEFAULT_REGION --name $SITE_ID
+        kubectl run -i --rm enterprise-k8s-aws-test --attach --image=$IMAGE_TAG --restart=Never --command -- python publish_csv.py -a https://$DEPLOYMENT_FQDN/$ARCGIS_ENTERPRISE_CONTEXT -u $ADMIN_USERNAME -p $ADMIN_PASSWORD
diff --git a/aws/arcgis-site-core/k8s-cluster/README.md b/aws/arcgis-site-core/k8s-cluster/README.md
@@ -72,7 +72,7 @@ thesubnet IDs are retrieved from the following SSM parameters:
 | container_registry_user | Source container registry user name | `string` | `null` | no |
 | ecr_repository_prefix | The repository name prefix to use when caching images from the source registry | `string` | `"docker-hub"` | no |
 | eks_version | The desired Kubernetes version for the EKS cluster | `string` | `"1.28"` | no |
-| enable_waf | Enable WAF addons for ALB | `bool` | `true` | no |
+| enable_waf | Enable WAF and Shield addons for ALB | `bool` | `true` | no |
 | key_name | EC2 key pair name | `string` | `null` | no |
 | node_groups | <p>EKS node groups configuration properties:</p>   <ul>   <li>name - Name of the node group</li>   <li>instance_type -Type of EC2 instance to use for the node group</li>   <li>root_volume_size - Size of the root volume in GB</li>   <li>desired_size - Number of nodes to start with</li>   <li>max_size - Maximum number of nodes in the node group</li>   <li>min_size - Minimum number of nodes in the node group</li>   <li>subnet_ids - List of subnet IDs to use for the node group (the first two private subnets are used by default)</li>   </ul> | ```list(object({ name = string instance_type = string root_volume_size = number desired_size = number max_size = number min_size = number subnet_ids = list(string) }))``` | ```[ { "desired_size": 4, "instance_type": "m6i.2xlarge", "max_size": 8, "min_size": 4, "name": "default", "root_volume_size": 1024, "subnet_ids": [] } ]``` | no |
 | pull_through_cache | Configure ECR pull through cache rules | `bool` | `true` | no |