Merge pull request #491 from sassoftware/staging

6.12.0 - September 21, 2023

Dockerfile

@@ -9,7 +9,7 @@ RUN apt-get update && apt-get upgrade -y \
   && update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 1
 
 FROM baseline as tool_builder
-ARG kubectl_version=1.25.9
+ARG kubectl_version=1.26.8
 
 WORKDIR /build
 

docs/CONFIG-VARS.md

@@ -23,6 +23,7 @@ Supported configuration variables are listed in the table below.  All variables
   - [PostgreSQL](#postgresql)
   - [CAS](#cas)
   - [CONNECT](#connect)
+  - [Workload Orchestrator](#workload-orchestrator)
   - [Miscellaneous](#miscellaneous)
   - [Third-Party Tools](#third-party-tools)
     - [Cert-manager](#cert-manager)
@@ -165,7 +166,7 @@ When V4_CFG_MANAGE_STORAGE is set to `true`, the `sas` and `pg-storage` storage
 | V4M_NODE_PLACEMENT_ENABLE | Whether to enable workload node placement for viya4-monitoring-kubernetes stack | bool | false | false | | cluster-logging, cluster-monitoring, viya-monitoring |
 | V4M_STORAGECLASS | StorageClass name | string | v4m | false | When V4_CFG_MANAGE_STORAGE is false, set to the name of your pre-existing StorageClass that supports ReadWriteOnce. | cluster-logging, cluster-monitoring, viya-monitoring |
 | V4M_ROUTING | Which routing type to use for viya4-monitoring-kubernetes applications | string | host-based | false | Supported values: [`host-based`, `path-based`] For host-based routing, the application name is part of the host name itself `https://dashboards.host.cluster.example.com/` For path-based routing, the host name is fixed and the application name is appended as a path on the URL `https://host.cluster.example.com/dashboards` | cluster-logging, cluster-monitoring |
-| V4M_CUSTOM_CONFIG_USER_DIR | Path to the viya4-monitoring-kubernetes top-level `USER_DIR` folder on the local file system. The `USER_DIR` folder can contain a top-level `user.env` file and `logging` and `monitoring` folders where your logging and monitoring `user.env` and customization yaml files are located. **NOTE**: viya4-monitoring does not validate `user.env` or yaml file content pointed to by this variable. It is recommended to use file content that has been verified ahead of time. | string | null | false | The following V4M configuration variables are ignored by viya4-monitoring when `V4M_CUSTOM_CONFIG_USER_DIR` is set: [`V4M_ROUTING`, `V4M_BASE_DOMAIN`, all `V4M_*_FQDN` variables,  all `V4M_*_PASSWORD` variables] [Additional documentation](https://go.documentation.sas.com/doc/en/obsrvcdc/v_001/obsrvdply/n0wgd3ju667sa9n1adnxs7hnsqt6.htm) describing the `USER_DIR` folder is available.| cluster-logging, cluster-monitoring
+| V4M_CUSTOM_CONFIG_USER_DIR | Path to the viya4-monitoring-kubernetes top-level `USER_DIR` folder on the local file system. The `USER_DIR` folder can contain a top-level `user.env` file and `logging` and `monitoring` folders where your logging and monitoring `user.env` and customization yaml files are located. **NOTE**: viya4-monitoring does not validate `user.env` or yaml file content pointed to by this variable. It is recommended to use file content that has been verified ahead of time. | string | null | false | The following V4M configuration variables are ignored by viya4-monitoring when `V4M_CUSTOM_CONFIG_USER_DIR` is set: [`V4M_ROUTING`, `V4M_BASE_DOMAIN`, all `V4M_*_FQDN` variables,  all `V4M_*_PASSWORD` variables] [Additional documentation](https://go.documentation.sas.com/doc/en/obsrvcdc/v_001/obsrvdply/n0wgd3ju667sa9n1adnxs7hnsqt6.htm) describing the `USER_DIR` folder is available.| cluster-logging, cluster-monitoring |
 
 #### Open Source Kubernetes
 
@@ -327,6 +328,16 @@ V4_CFG_POSTGRES_SERVERS:
 | V4_CFG_CONNECT_ENABLE_LOADBALANCER | Set up LoadBalancer to access SAS/CONNECT | bool | false | false | | viya |
 | V4_CFG_CONNECT_FQDN | FQDN that is assigned to access SAS/CONNECT | string | | false | Required when V4_CFG_TLS_MODE is not disabled and cert-manager is used to issue TLS certificates. This FQDN is added to the SAN DNS list of the issued certificates. | viya |
 
+## Workload Orchestrator
+
+| Name | Description | Type | Default | Required | Notes | Tasks |
+| :--- |------------:| ---: | ---: | ---: | ---: | ---: |
+| V4_WORKLOAD_ORCHESTRATOR_ENABLED | Enables the [SAS Workload Orchestrator](https://documentation.sas.com/?cdcId=itopscdc&cdcVersion=default&docsetId=dplyml0phy0dkr&docsetTarget=n08u2yg8tdkb4jn18u8zsi6yfv3d.htm#p1vo217m7ffso5n11vxwsyycw4tg) service and configures the required ClusterRole and ClusterRoleBinding used by the daemon. Setting this to false will disable SAS Workload Orchestrator service entirely | bool | true | false | This flag is only applicable for cadences 2023.08 and newer, this flag will perform no action on older cadences. | viya |
+
+The SAS Workload Orchestrator Service is used to manage workload started on demand through the launcher service. As of cadence 2023.08 this feature is now deployed by default. The SAS Workload Orchestrator daemons require information about resources on the nodes that can be used to run jobs. In order to obtain accurate resource information, it requires a ClusterRole and a ClusterRoleBinding to the SAS Workload Orchestrator service account which will be automatically configured by this project if you set `V4_WORKLOAD_ORCHESTRATOR_ENABLED` to true. 
+
+Additional documentation for the SAS Workload Orchestrator Service can be found here in the [SAS Viya Platform Operations documentation](https://documentation.sas.com/?cdcId=itopscdc&cdcVersion=default&docsetId=dplyml0phy0dkr&docsetTarget=n08u2yg8tdkb4jn18u8zsi6yfv3d.htm#p1vo217m7ffso5n11vxwsyycw4tg). 
+
 ## Miscellaneous
 
 | Name | Description | Type | Default | Required | Notes | Tasks |

docs/user/Dependencies.md

@@ -13,7 +13,7 @@ The following list details our dependencies and versions (~ indicates multiple p
 | ~              | docker           | >=20.10.10  |
 | ~              | git              | any         |
 | ~              | rsync            | any         |
-| ~              | kubectl          | 1.24 - 1.26 |
+| ~              | kubectl          | 1.25 - 1.27 |
 | ~              | Helm             | 3           |
 | pip3           | ansible          | 8.0.0       |
 | pip3           | openshift        | 0.13.1      |
@@ -48,7 +48,7 @@ As described in the [Docker Installation](./DockerUsage.md) section add addition
 ```bash
 # Override kubectl version
 docker build \
-	--build-arg kubectl_version=1.25.9 \
+	--build-arg kubectl_version=1.26.8 \
 	-t viya4-deployment .
 ```
 

roles/baseline/defaults/main.yml

@@ -35,12 +35,10 @@ ingressVersions:
     value: 23
     api:
       chartVersion: 4.3.0
-      appVersion: 1.4.0
   k8sMinorVersionFloor:
     value: 24
     api:
       chartVersion: 4.7.1
-      appVersion: 1.8.1
 
 ## Ingress-nginx - Ingress
 ##

roles/monitoring/files/azure-storageclass.yaml

@@ -12,7 +12,7 @@ metadata:
   name: v4m
 parameters:
   skuName: Standard_LRS
-provisioner: kubernetes.io/azure-disk
+provisioner: disk.csi.azure.com
 reclaimPolicy: Delete
 # Set binding mode to WaitForFirstConsumer to avoid
 # volume node affinity issues

roles/vdm/defaults/main.yaml

@@ -109,3 +109,6 @@ V4_CFG_MESSAGE_BROKER_ENABLE: false
 V4_CFG_MESSAGE_BROKER_HOST: null
 V4_CFG_MESSAGE_BROKER_NAME: null
 V4_CFG_MESSAGE_BROKER_PASSWORD: null
+
+## SAS Workload Orchestrator
+V4_WORKLOAD_ORCHESTRATOR_ENABLED: true

roles/vdm/tasks/main.yaml

@@ -190,6 +190,13 @@
     - uninstall
     - update
 
+- name: Include Workload Orchestrator
+  include_tasks: workload_orchestrator.yaml
+  tags:
+    - install
+    - uninstall
+    - update
+
 - name: Include Start Stop
   include_tasks: start_stop.yaml
   tags:

roles/vdm/tasks/workload_orchestrator.yaml

@@ -0,0 +1,53 @@
+# Copyright © 2020-2023, SAS Institute Inc., Cary, NC, USA. All Rights Reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+---
+
+- name: Workload Orchestrator - Configure SAS Workload Orchestrator Service Account
+  overlay_facts:
+    cadence_name: "{{ V4_CFG_CADENCE_NAME }}"
+    cadence_number: "{{ V4_CFG_CADENCE_VERSION }}"
+    existing: "{{ vdm_overlays }}"
+    add:
+      - { resources: overlays/sas-workload-orchestrator, min: "2023.08", vdm: false }
+  when:
+    - V4_WORKLOAD_ORCHESTRATOR_ENABLED
+  tags:
+    - install
+    - uninstall
+    - update
+
+- name: Workload Orchestrator - Disable the SAS Workload Orchestrator Service
+  overlay_facts:
+    cadence_name: "{{ V4_CFG_CADENCE_NAME }}"
+    cadence_number: "{{ V4_CFG_CADENCE_VERSION }}"
+    existing: "{{ vdm_overlays }}"
+    add:
+      - { transformers: examples/sas-workload-orchestrator/enable-disable/sas-workload-orchestrator-disable-patch-transformer.yaml, min: "2023.08", vdm: false }
+  when:
+    - not V4_WORKLOAD_ORCHESTRATOR_ENABLED
+  tags:
+    - install
+    - uninstall
+    - update
+
+- name: Workload Orchestrator - Remove the ClusterRoleBinding and ClusterRole
+  when:
+    - DEPLOY
+    - V4_WORKLOAD_ORCHESTRATOR_ENABLED
+    - V4_CFG_CADENCE_VERSION is version('2023.08', ">=") or V4_CFG_CADENCE_NAME|lower == "fast"
+  tags:
+    - uninstall
+  block:
+    - name: Workload Orchestrator - Remove ClusterRoleBinding
+      kubernetes.core.k8s:
+        kind: ClusterRoleBinding
+        name: "sas-workload-orchestrator-{{ NAMESPACE }}"
+        kubeconfig: "{{ KUBECONFIG }}"
+        state: absent
+    - name: Workload Orchestrator - Remove ClusterRole
+      kubernetes.core.k8s:
+        kind: ClusterRole
+        name: "sas-workload-orchestrator"
+        kubeconfig: "{{ KUBECONFIG }}"
+        state: absent