New test: install the operator with debug enabled

[wainersm]

Test Case

Use case: As an user installing CoCo via operator I want to switch on the debugging messages so that I can debug errors on the installation.

Add a new test case to the operator tests to check the condition debug=true.

Split out of #247

Implementation

In order to switch debugging on/off , you should change the kustomization file on the corresponding ccruntime's overlay directory. For example, in the default ccruntime kustomization file:

diff --git a/config/samples/ccruntime/default/kustomization.yaml b/config/samples/ccruntime/default/kustomization.yaml
index b22bc16..1ebd13c 100644
--- a/config/samples/ccruntime/default/kustomization.yaml
+++ b/config/samples/ccruntime/default/kustomization.yaml
@@ -33,7 +33,7 @@ patches:
       value: "kata-qemu"
     - op: add
       path: /spec/config/debug
-      value: false
+      value: true
     - op: add
       path: /spec/config/environmentVariables
       value:

Questions:

• Once the operator is installed then it should be checked that kata-deploy emit debugging messages. How? Maybe grep'ing the messages of the installer pod?

[wainersm]

I'm not sure the debug is set correctly. kata-deploy expects an environment variable exported as DEBUG=true|false whereas the generated ccruntime deployment has spec/config/debug set:

[wmoschet@wainer-laptop operator]$ ./kustomize build config/samples/ccruntime/default/
apiVersion: confidentialcontainers.org/v1beta1
kind: CcRuntime
metadata:
  name: ccruntime-sample
spec:
  ccNodeSelector:
    matchLabels:
      node.kubernetes.io/worker: ""
  config:
    cleanupCmd:
    - /opt/kata-artifacts/scripts/kata-deploy.sh
    - reset
    debug: false
    defaultRuntimeClassName: kata-qemu
    environmentVariables:
    - name: INSTALL_OFFICIAL_CONTAINERD
      value: "false"
    installCmd:
    - /opt/kata-artifacts/scripts/kata-deploy.sh
    - install
    installDoneLabel:

<SNIP>

[wainersm]

I'm not sure the debug is set correctly. kata-deploy expects an environment variable exported as DEBUG=true|false whereas the generated ccruntime deployment has spec/config/debug set:

[wmoschet@wainer-laptop operator]$ ./kustomize build config/samples/ccruntime/default/
apiVersion: confidentialcontainers.org/v1beta1
kind: CcRuntime
metadata:
  name: ccruntime-sample
spec:
  ccNodeSelector:
    matchLabels:
      node.kubernetes.io/worker: ""
  config:
    cleanupCmd:
    - /opt/kata-artifacts/scripts/kata-deploy.sh
    - reset
    debug: false
    defaultRuntimeClassName: kata-qemu
    environmentVariables:
    - name: INSTALL_OFFICIAL_CONTAINERD
      value: "false"
    installCmd:
    - /opt/kata-artifacts/scripts/kata-deploy.sh
    - install
    installDoneLabel:

<SNIP>

Actually controllers/ccruntime_controller.go converts the spec/config/debug into an environment variable.

[portersrc]

Once the operator is installed then it should be checked that kata-deploy emit debugging messages. How? Maybe grep'ing the messages of the installer pod?

Is there a clean way to do this by checking for any logs with priority level == debug? If not, maybe that's a longer term goal to improve our logging.

[wainersm]

Hi @portersrc !

Once the operator is installed then it should be checked that kata-deploy emit debugging messages. How? Maybe grep'ing the messages of the installer pod?

Is there a clean way to do this by checking for any logs with priority level == debug? If not, maybe that's a longer term goal to improve our logging.

Just after opening this issue I looked at kata-deploy's code again, now I have a better understanding of what it enables in terms of debugging. It is not that kata-deploy script itself will emit debug messages, but rather it will enable debugging of the following components:

• Kata Container - set debug=true on various sections of the kata's configuration toml (https://github.com/kata-containers/kata-containers/blob/main/tools/packaging/kata-deploy/scripts/kata-deploy.sh#L163)
• Containerd (https://github.com/kata-containers/kata-containers/blob/main/tools/packaging/kata-deploy/scripts/kata-deploy.sh#L367)
• Cri-o (https://github.com/kata-containers/kata-containers/blob/main/tools/packaging/kata-deploy/scripts/kata-deploy.sh#L338)

[portersrc]

Good eyes. Well, one passing condition could be to check that all 3 of those files have debug enabled. It's a little error-prone, crude, etc., but that would work for now.

[wainersm]

Good eyes. Well, one passing condition could be to check that all 3 of those files have debug enabled. It's a little error-prone, crude, etc., but that would work for now.

Yep, I had the same thing in mind :)