cluster-up, kind, common: Enable TopologyManager for kind-sriov (#1347)

SRIOV tests check topology alignment. Currently in kind/sriov kueblet does not attempt
to align resources.
In SRIOV alignment test, we call the function
hardware.LookupDeviceVCPUAffinity(). This function returns a slice of
aligned CPUs, i.e. (complex sentence ahead warning) CPU numbers,
that are assigned to the guest, which share a NUMA with SRIOV VFs, that are
also passed to the guest.
In other words: If an app is running in the guest, and using a network
interface, that NIC is physically close to its CPU, and it doesn't have
to cross the system bus to get to it.
Back to LookupDeviceVCPUAffinity(): if it finds that there is
no aligment, it doesn't err, it just returns an empty slice.
As the test is currently written, an empty list is fine i.e. no
alignment: it simply validates that the guest knows that there's no
alignment, by validating it in the cloud-init metadata file in the
guest.
This change adding a topology manager policy of single-numa-node, forces
alignment. If alignment is not achieved, scheduling will fail.
We will also assert in the test that the alignment slice is not empty.

Add topology manager[1] to kubelet config and set its policy to single-numa-node.
Together with cpu-manager policy=static, which we already set,
kubelet will reject a pod that it is unable to align.
[1] https://kubernetes.io/docs/tasks/administer-cluster/topology-manager/

Signed-off-by: Nir Dothan <[email protected]>

cluster-up/cluster/kind-sriov/provider.sh

@@ -62,6 +62,7 @@ function deploy_sriov() {
 function up() {
     cp $KIND_MANIFESTS_DIR/kind.yaml ${KUBEVIRTCI_CONFIG_PATH}/$KUBEVIRT_PROVIDER/kind.yaml
     export CONFIG_WORKER_CPU_MANAGER=true
+    export CONFIG_TOPOLOGY_MANAGER_POLICY="single-numa-node"
     kind_up
 
     configure_registry_proxy

cluster-up/cluster/kind/common.sh

@@ -311,20 +311,34 @@ EOF
     done
 }
 
-function _add_kubeadm_config_patches() {
-    if [ $KUBEVIRT_WITH_KIND_ETCD_IN_MEMORY == "true" ]; then
-        cat <<EOF >> ${KUBEVIRTCI_CONFIG_PATH}/$KUBEVIRT_PROVIDER/kind.yaml
+function _add_kubeadm_config_patches_header() {
+   cat <<EOF >> ${KUBEVIRTCI_CONFIG_PATH}/$KUBEVIRT_PROVIDER/kind.yaml
 kubeadmConfigPatches:
 - |
+EOF
+}
+
+function _add_kubeadm_config_patches() {
+  _add_kubeadm_config_patches_header
+  if [ $KUBEVIRT_WITH_KIND_ETCD_IN_MEMORY == "true" ]; then
+    cat <<EOF >> ${KUBEVIRTCI_CONFIG_PATH}/$KUBEVIRT_PROVIDER/kind.yaml
   kind: ClusterConfiguration
   metadata:
     name: config
   etcd:
     local:
       dataDir: $ETCD_IN_MEMORY_DATA_DIR
 EOF
-        echo "KIND cluster etcd data will be mounted to RAM on kind nodes: $ETCD_IN_MEMORY_DATA_DIR"
-    fi
+    echo "KIND cluster etcd data will be mounted to RAM on kind nodes: $ETCD_IN_MEMORY_DATA_DIR"
+  fi
+  if [[ -n "$CONFIG_TOPOLOGY_MANAGER_POLICY" ]]; then
+     cat <<EOF >> ${KUBEVIRTCI_CONFIG_PATH}/$KUBEVIRT_PROVIDER/kind.yaml
+  ---
+  kind: KubeletConfiguration
+  topologyManagerPolicy: ${CONFIG_TOPOLOGY_MANAGER_POLICY}
+  ---
+EOF
+  fi
 }
 
 function _setup_ipfamily() {
@@ -339,7 +353,9 @@ EOF
 
 function _prepare_kind_config() {
     _add_workers
-    _add_kubeadm_config_patches
+    if [[ "$KUBEVIRT_WITH_KIND_ETCD_IN_MEMORY" == "true" ||  -n "$CONFIG_TOPOLOGY_MANAGER_POLICY" ]]; then
+      _add_kubeadm_config_patches
+    fi
     _setup_ipfamily
     echo "Final KIND config:"
     cat ${KUBEVIRTCI_CONFIG_PATH}/$KUBEVIRT_PROVIDER/kind.yaml