sap_ha_pacemaker_cluster: Stonith SBD enablement, stonith and repos dictionaries

roles/sap_ha_pacemaker_cluster/README.md

@@ -13,8 +13,8 @@ This Ansible Role provides:
 - setup and instantiation of Linux Pacemaker cluster (using `ha_cluster` Linux System Role)
 
 This Ansible Role has been tested for the following SAP Software Solution scenario deployments:
-- SAP HANA Scale-up High Availability
-- `Beta:` SAP NetWeaver (ABAP) AS ASCS and ERS High Availability
+- SAP HANA Scale-up High Availability (SAPHanaSR Classic and SAPHanaSR-angi)
+- SAP NetWeaver (ABAP) AS ASCS and ERS High Availability
 - `Experimental:` SAP NetWeaver (ABAP) AS PAS and AAS High Availability
 - `Experimental:` SAP NetWeaver (JAVA) AS SCS and ERS High Availability
 
@@ -71,8 +71,8 @@ The Ansible Control System (where Ansible is executed from) must have:
 - Ansible Core 2.9+
 - Access to dependency Ansible Collections and Ansible Roles:
   - **Upstream**:
-    - Ansible Collection [`community.sap_install` from Ansible Galaxy](https://galaxy.ansible.com/community/sap_install) version `1.3.0` or later
-    - Ansible Collection [`fedora.linux_system_roles` from Ansible Galaxy](https://galaxy.ansible.com/fedora/linux_system_roles) version `1.20.0` or later
+    - Ansible Collection [`community.sap_install` from Ansible Galaxy](https://galaxy.ansible.com/community/sap_install) version `1.4.1` or later
+    - Ansible Collection [`fedora.linux_system_roles` from Ansible Galaxy](https://galaxy.ansible.com/fedora/linux_system_roles) version `1.82.0` or later
   - **Supported (Downstream)** via Red Hat Ansible Automation Platform (AAP) license:
     - Ansible Collection [`redhat.sap_install` from Red Hat Ansible Automation Platform Hub](https://console.redhat.com/ansible/automation-hub/repo/published/redhat/sap_install) version `1.3.0` or later
     - Ansible Collection [`redhat.rhel_system_roles` from Red Hat Ansible Automation Platform Hub](https://console.redhat.com/ansible/automation-hub/repo/published/redhat/rhel_system_roles) version `1.20.0` or later
@@ -148,6 +148,13 @@ Additional minimum requirements depend on the type of cluster setup and on the t
 AWS access key to allow control of instances (for example for fencing operations).<br>
 Mandatory for the cluster nodes setup on AWS EC2 instances.<br>
 
+### sap_ha_pacemaker_cluster_aws_credentials_setup
+
+- _Type:_ `string`
+
+Set this parameter to 'true' to store AWS credentials into /root/.aws/credentials.<br>
+Required: `sap_ha_pacemaker_cluster_aws_access_key_id` and `sap_ha_pacemaker_cluster_aws_secret_access_key`<br>
+
 ### sap_ha_pacemaker_cluster_aws_region
 
 - _Type:_ `string`
@@ -259,19 +266,14 @@ Set this parameter to 'false' if the SAP HA interface should not be installed an
 Additional extra packages to be installed, for instance specific resource packages.<br>
 For SAP clusters configured by this role, the relevant standard packages for the target scenario are automatically included.<br>
 
-### sap_ha_pacemaker_cluster_fence_agent_minimal_packages
-
-- _Type:_ `list`
-- _Default:_ `['fence-agents-all']`
-
-The minimal set of fence agent packages that will be installed.<br>
-
 ### sap_ha_pacemaker_cluster_fence_agent_packages
 
 - _Type:_ `list`
 
 Additional fence agent packages to be installed.<br>
-This is automatically combined with `sap_ha_pacemaker_cluster_fence_agent_minimal_packages`.<br>
+This is automatically combined with default packages in:<br>
+`__sap_ha_pacemaker_cluster_fence_agent_packages_minimal`<br>
+`__sap_ha_pacemaker_cluster_fence_agent_packages_platform`<br>
 
 ### sap_ha_pacemaker_cluster_gcp_project
 
@@ -460,7 +462,7 @@ When set to "true" (default) a failover to secondary will be initiated on resour
 - _Default:_ `msl_SAPHana_<SID>_HDB<Instance Number>`
 
 Customize the cluster resource name of the SAP HANA DB resource master slave clone.<br>
-Master Slave clone is specific to SAPHana resource on SUSE.<br>
+Master Slave clone is specific to Classic SAPHana resource on SUSE (non-angi).<br>
 
 ### sap_ha_pacemaker_cluster_hana_resource_clone_name
 
@@ -849,6 +851,70 @@ sap_ha_pacemaker_cluster_resource_defaults:
 
 Disabling this variable enables to use Classic SAPHanaSR agents even on server, with SAPHanaSR-angi is available.<br>
 
+### sap_ha_pacemaker_cluster_sbd_devices
+
+- _Type:_ `list`
+
+Required if `sap_ha_pacemaker_cluster_sbd_enabled` is enabled.<br>
+Provide list of block devices for Stonith SBD agent<br>
+
+Example:
+
+```yaml
+sap_ha_pacemaker_cluster_sbd_devices:
+- /dev/disk/by-id/scsi-3600
+```
+
+### sap_ha_pacemaker_cluster_sbd_enabled
+
+- _Type:_ `bool`
+
+Set this parameter to 'true' to enable workflow to add Stonith SBD resource.<br>
+Stonith SBD resource has to be provided as part of `sap_ha_pacemaker_cluster_stonith_custom`.<br>
+Default SBD agents are: stonith:external/sbd for SLES and stonith:fence_sbd for RHEL<br>
+
+Example:
+
+```yaml
+sap_ha_pacemaker_cluster_sbd_devices:
+- /dev/disk/by-id/scsi-3600
+sap_ha_pacemaker_cluster_sbd_enabled: true
+sap_ha_pacemaker_cluster_stonith_custom:
+- agent: stonith:external/sbd
+  name: rsc_stonith_sbd
+  options:
+    pcmk_delay_max: 15
+```
+
+### sap_ha_pacemaker_cluster_sbd_options
+
+- _Type:_ `list`
+
+Optional if `sap_ha_pacemaker_cluster_sbd_enabled` is enabled.<br>
+Provide list SBD specific options that are added into SBD configuration file.<br>
+
+Example:
+
+```yaml
+sap_ha_pacemaker_cluster_sbd_options:
+- name: startmode
+  value: clean
+```
+
+### sap_ha_pacemaker_cluster_sbd_watchdog
+
+- _Type:_ `str`
+
+Optional if `sap_ha_pacemaker_cluster_sbd_enabled` is enabled.<br>
+Provide watchdog name to override default /dev/watchdog<br>
+
+### sap_ha_pacemaker_cluster_sbd_watchdog_modules
+
+- _Type:_ `list`
+
+Optional if `sap_ha_pacemaker_cluster_sbd_enabled` is enabled.<br>
+Provide list of watchdog kernel modules to be loaded (creates /dev/watchdog* devices).<br>
+
 ### sap_ha_pacemaker_cluster_stonith_custom
 
 - _Type:_ `list`
@@ -861,7 +927,7 @@ This definition override any defaults the role would apply otherwise.<br>
 - **name**<br>
     Name that will be used as the resource ID (name).
 - **options**<br>
-    The resource options listed in dictionary format, one option per line.<br>Requires the mandatory options for the particular stonith resource agent to be defined, otherwise the setup will fail.
+    The resource options listed in dictionary format, one option per line.<br>Requires the mandatory options for the particular stonith resource agent to be defined, otherwise the setup will fail.<br>Example: stonith:fence_sbd agent requires devices option with list of SBD disks.<br>Example: stonith:external/sbd agent does not require devices option, but `sap_ha_pacemaker_cluster_sbd_devices`.
 
 Example:
 

roles/sap_ha_pacemaker_cluster/defaults/main.yml

@@ -365,6 +365,7 @@ sap_ha_pacemaker_cluster_vip_nwas_abap_ers_resource_group_name: >-
 # sap_ha_pacemaker_cluster_aws_access_key_id
 # sap_ha_pacemaker_cluster_aws_secret_access_key
 # sap_ha_pacemaker_cluster_aws_region
+# sap_ha_pacemaker_cluster_aws_credentials_setup
 
 ## Google Cloud platform, Compute Engine Virtual Machines
 # sap_ha_pacemaker_cluster_gcp_project

roles/sap_ha_pacemaker_cluster/meta/argument_specs.yml

@@ -102,12 +102,6 @@ argument_specs:
               node_ip: 192.168.5.1
             - hana_site: DC02
 
-      sap_ha_pacemaker_cluster_fence_agent_minimal_packages:
-        type: list
-        default: ['fence-agents-all']
-        description:
-          - The minimal set of fence agent packages that will be installed.
-
       sap_ha_pacemaker_cluster_resource_defaults:
         type: dict
         default:
@@ -181,6 +175,8 @@ argument_specs:
               - The resource options listed in dictionary format, one option per line.
               - Requires the mandatory options for the particular stonith resource agent to be
                 defined, otherwise the setup will fail.
+              - "Example: stonith:fence_sbd agent requires devices option with list of SBD disks."
+              - "Example: stonith:external/sbd agent does not require devices option, but `sap_ha_pacemaker_cluster_sbd_devices`."
             required: true
 
         example:
@@ -194,6 +190,56 @@ argument_specs:
                 pcmk_host_list: node1,node2
                 power_wait: 3
 
+      sap_ha_pacemaker_cluster_sbd_enabled:
+        type: bool
+        description:
+          - Set this parameter to 'true' to enable workflow to add Stonith SBD resource.
+          - Stonith SBD resource has to be provided as part of `sap_ha_pacemaker_cluster_stonith_custom`.
+          - "Default SBD agents are: stonith:external/sbd for SLES and stonith:fence_sbd for RHEL"
+
+        example:
+          sap_ha_pacemaker_cluster_sbd_enabled: true
+          sap_ha_pacemaker_cluster_stonith_custom:
+            - name: rsc_stonith_sbd
+              agent: stonith:external/sbd
+              options:
+                pcmk_delay_max: 15
+          sap_ha_pacemaker_cluster_sbd_devices:
+            - /dev/disk/by-id/scsi-3600
+
+      sap_ha_pacemaker_cluster_sbd_devices:
+        type: list
+        description:
+          - Required if `sap_ha_pacemaker_cluster_sbd_enabled` is enabled.
+          - Provide list of block devices for Stonith SBD agent
+
+        example:
+          sap_ha_pacemaker_cluster_sbd_devices:
+            - /dev/disk/by-id/scsi-3600
+
+      sap_ha_pacemaker_cluster_sbd_options:
+        type: list
+        description:
+          - Optional if `sap_ha_pacemaker_cluster_sbd_enabled` is enabled.
+          - Provide list SBD specific options that are added into SBD configuration file.
+
+        example:
+          sap_ha_pacemaker_cluster_sbd_options:
+            - name: startmode
+              value: clean
+
+      sap_ha_pacemaker_cluster_sbd_watchdog:
+        type: str
+        description:
+          - Optional if `sap_ha_pacemaker_cluster_sbd_enabled` is enabled.
+          - Provide watchdog name to override default /dev/watchdog
+
+      sap_ha_pacemaker_cluster_sbd_watchdog_modules:
+        type: list
+        description:
+          - Optional if `sap_ha_pacemaker_cluster_sbd_enabled` is enabled.
+          - Provide list of watchdog kernel modules to be loaded (creates /dev/watchdog* devices).
+
       sap_ha_pacemaker_cluster_cluster_properties:
         type: dict
         default:
@@ -251,7 +297,9 @@ argument_specs:
         type: list
         description:
           - Additional fence agent packages to be installed.
-          - This is automatically combined with `sap_ha_pacemaker_cluster_fence_agent_minimal_packages`.
+          - "This is automatically combined with default packages in:"
+          - "`__sap_ha_pacemaker_cluster_fence_agent_packages_minimal`"
+          - "`__sap_ha_pacemaker_cluster_fence_agent_packages_platform`"
 
       sap_ha_pacemaker_cluster_hacluster_user_password:
         description:
@@ -321,7 +369,7 @@ argument_specs:
         default: "msl_SAPHana_<SID>_HDB<Instance Number>"
         description:
           - Customize the cluster resource name of the SAP HANA DB resource master slave clone.
-          - Master Slave clone is specific to SAPHana resource on SUSE.
+          - Master Slave clone is specific to Classic SAPHana resource on SUSE (non-angi).
 
       sap_ha_pacemaker_cluster_hanacontroller_resource_name:
         default: "rsc_SAPHanaCon_<SID>_HDB<Instance Number>"
@@ -798,6 +846,10 @@ argument_specs:
           - AWS secret key, paired with the access key for instance control.
           - Mandatory for the cluster setup on AWS EC2 instances.
 
+      sap_ha_pacemaker_cluster_aws_credentials_setup:
+        description:
+          - Set this parameter to 'true' to store AWS credentials into /root/.aws/credentials.
+          - "Required: `sap_ha_pacemaker_cluster_aws_access_key_id` and `sap_ha_pacemaker_cluster_aws_secret_access_key`"
 
       ##########################################################################
       # Platforms: GCP specific parameters

roles/sap_ha_pacemaker_cluster/tasks/construct_final_hacluster_vars.yml

@@ -29,6 +29,18 @@
 # __sap_ha_pacemaker_cluster_resource_primitives    ha_cluster_resource_primitives
 # __sap_ha_pacemaker_cluster_corosync_totem                  ha_cluster_totem
 
+# Combines SBD stonith options with ha_cluster if it was not imported as extra var.
+- name: "SAP HA Prepare Pacemaker - (ha_cluster) Include SBD config into 'ha_cluster'"  # noqa jinja[spacing]
+  when:
+    - __sap_ha_pacemaker_cluster_ha_cluster_stonith is defined and __sap_ha_pacemaker_cluster_ha_cluster_stonith | length > 0
+  ansible.builtin.set_fact:
+    __sap_ha_pacemaker_cluster_ha_cluster: >-
+      {%- if __sap_ha_pacemaker_cluster_ha_cluster is defined and __sap_ha_pacemaker_cluster_ha_cluster | length > 0 -%}
+      {{ __sap_ha_pacemaker_cluster_ha_cluster | combine(__sap_ha_pacemaker_cluster_ha_cluster_stonith) }}
+      {%- else -%}
+      {{ __sap_ha_pacemaker_cluster_ha_cluster_stonith }}
+      {%- endif -%}
+
 - name: "SAP HA Prepare Pacemaker - (ha_cluster) Define parameter 'ha_cluster'"
   when: __sap_ha_pacemaker_cluster_ha_cluster is defined
   ansible.builtin.set_fact:
@@ -110,3 +122,13 @@
   when: __sap_ha_pacemaker_cluster_corosync_totem is defined
   ansible.builtin.set_fact:
     ha_cluster_totem: "{{ __sap_ha_pacemaker_cluster_corosync_totem }}"
+
+- name: "SAP HA Prepare Pacemaker - (ha_cluster) Define parameter 'ha_cluster_sbd_options'"
+  when: __sap_ha_pacemaker_cluster_sbd_options is defined
+  ansible.builtin.set_fact:
+    ha_cluster_sbd_options: "{{ __sap_ha_pacemaker_cluster_sbd_options }}"
+
+- name: "SAP HA Prepare Pacemaker - (ha_cluster) Define parameter 'ha_cluster_sbd_enabled'"
+  when: __sap_ha_pacemaker_cluster_sbd_enabled is defined
+  ansible.builtin.set_fact:
+    ha_cluster_sbd_enabled: "{{ __sap_ha_pacemaker_cluster_sbd_enabled }}"

roles/sap_ha_pacemaker_cluster/tasks/construct_vars_common.yml

@@ -22,7 +22,25 @@
     __sap_ha_pacemaker_cluster_hacluster_user_password: "{{ sap_ha_pacemaker_cluster_hacluster_user_password }}"
   no_log: true  # secure the credential
 
+# sap_ha_pacemaker_cluster_ha_cluster   -> user-defined or default inherited from {{ ha_cluster }}
+- name: "SAP HA Prepare Pacemaker - Register sap_ha_pacemaker_cluster_ha_cluster"
+  when:
+    - __sap_ha_pacemaker_cluster_ha_cluster is not defined
+    - sap_ha_pacemaker_cluster_ha_cluster is defined
+  ansible.builtin.set_fact:
+    __sap_ha_pacemaker_cluster_ha_cluster: "{{ sap_ha_pacemaker_cluster_ha_cluster }}"
+
+- name: "SAP HA Prepare Pacemaker - Generate default sap_ha_pacemaker_cluster_ha_cluster"
+  when:
+    - not __sap_ha_pacemaker_cluster_ha_cluster is defined
+    - not sap_ha_pacemaker_cluster_ha_cluster is defined
+  ansible.builtin.set_fact:
+    __sap_ha_pacemaker_cluster_ha_cluster:
+      node_name: "{{ ansible_hostname }}"
+      pcs_address: "{{ ansible_default_ipv4.address }}"
 
+
+# Combine following extra packages together:
 # sap_ha_pacemaker_cluster_extra_packages             -> user-defined, empty by global default
 # __sap_ha_pacemaker_cluster_sap_extra_packages       -> included from vars/*
 # __sap_ha_pacemaker_cluster_platform_extra_packages  -> included from vars/platform*
@@ -36,17 +54,32 @@
       | unique | select() }}"
   # remove duplicates and empty elements
 
-# sap_ha_pacemaker_cluster_fence_agent_minimal_packages -> global default
-# sap_ha_pacemaker_cluster_fence_agent_packages         -> global default
-# __sap_ha_pacemaker_cluster_fence_agent_packages       -> internal default (vars/main.yml)
 
+# Combine following fence packages together:
+# __sap_ha_pacemaker_cluster_fence_agent_packages_minimal -> os default
+# __sap_ha_pacemaker_cluster_fence_agent_packages_platform -> platform defaults from dict
+# sap_ha_pacemaker_cluster_fence_agent_packages         -> user input or default []
+
+# __sap_ha_pacemaker_cluster_fence_agent_packages loaded from ha_cluster is not included,
+# because it would still not be used due to precedence.
+# TODO: Remove Tech debt conditionals in future.
 - name: "SAP HA Prepare Pacemaker - Combine fence agent packages lists"
   ansible.builtin.set_fact:
     __sap_ha_pacemaker_cluster_fence_agent_packages: "{{
-      (sap_ha_pacemaker_cluster_fence_agent_minimal_packages
-      + sap_ha_pacemaker_cluster_fence_agent_packages
-      + __sap_ha_pacemaker_cluster_fence_agent_packages)
+      (__sap_ha_pacemaker_cluster_fence_agent_packages_minimal_combined
+      + __sap_ha_pacemaker_cluster_fence_agent_packages_platform
+      + sap_ha_pacemaker_cluster_fence_agent_packages)
       | unique }}"
+  vars:
+    # Tech debt for sap_ha_pacemaker_cluster_fence_agent_minimal_packages
+    __sap_ha_pacemaker_cluster_fence_agent_packages_minimal_combined:
+      "{{ __sap_ha_pacemaker_cluster_fence_agent_packages_minimal
+       + sap_ha_pacemaker_cluster_fence_agent_minimal_packages
+        if (sap_ha_pacemaker_cluster_fence_agent_minimal_packages is defined
+          and sap_ha_pacemaker_cluster_fence_agent_minimal_packages | length > 0
+          and sap_ha_pacemaker_cluster_fence_agent_minimal_packages is iterable)
+        else __sap_ha_pacemaker_cluster_fence_agent_packages_minimal }}"
+
 
 # Prepare corosync totem variable with either:
 # - User provided sap_ha_pacemaker_cluster_corosync_totem if present