Skip to content

Latest commit

 

History

History
226 lines (163 loc) · 8.91 KB

minikube-installation-guide.md

File metadata and controls

226 lines (163 loc) · 8.91 KB
Raw

Installing Kata Containers in Minikube

Introduction

Minikube is an easy way to try out a Kubernetes (k8s) cluster locally. It creates a single node Kubernetes stack in a local VM.

Kata Containers can be installed into a Minikube cluster using kata-deploy.

This document details the pre-requisites, installation steps, and how to check the installation has been successful.

Prerequisites

This installation guide has only been verified under a Minikube Linux installation, using the kvm2 driver.

Notes:

  • This installation guide may not work for macOS installations of Minikube, due to the lack of nested virtualization support on that platform.
  • This installation guide has not been tested on a Windows installation.
  • Kata under Minikube does not currently support Kata Firecracker (kata-fc). Although the kata-fc binary will be installed as part of these instructions, via kata-deploy, pods cannot be launched with kata-fc, and will fail to start.

Before commencing installation, it is strongly recommended you read the Minikube installation guide.

Checking for nested virtualization

For Kata Containers to work under a Minikube VM, your host system must support nested virtualization. If you are using a Linux system utilizing Intel VT-x and the kvm_intel driver, you can perform the following check:

$ cat /sys/module/kvm_intel/parameters/nested

If your system does not report Y from the nested parameter, then details on how to enable nested virtualization can be found on the KVM Nested Guests page

Alternatively, and for other architectures, the Kata Containers built in check command can be used inside Minikube once Kata has been installed, to check for compatibility.

Setting up Minikube

To enable Kata Containers under Minikube, you need to add a few configuration options to the default Minikube setup. You can easily accomplish this as Minikube supports them on the setup commandline. Minikube can be set up to use either CRI-O or containerd.

Here are the features to set up a CRI-O based Minikube, and why you need them:

what why
--bootstrapper=kubeadm As recommended for minikube CRI-o
--container-runtime=cri-o Using CRI-O for Kata
--enable-default-cni As recommended for minikube CRI-o
--memory 6144 Allocate sufficient memory, as Kata Containers default to 1 or 2Gb
--network-plugin=cni As recommended for minikube CRI-o
--vm-driver kvm2 The host VM driver

To use containerd, modify the --container-runtime argument:

what why
--container-runtime=containerd Using containerd for Kata

Notes:

  • Adjust the --memory 6144 line to suit your environment and requirements. Kata Containers default to requesting 2048MB per container. We recommended you supply more than that to the Minikube node.
  • Prior to Minikube/Kubernetes v1.14, the beta RuntimeClass feature also needed enabling with the following.
what why
--feature-gates=RuntimeClass=true Kata needs to use the RuntimeClass Kubernetes feature

The full command is therefore:

$ minikube start --vm-driver kvm2 --memory 6144 --network-plugin=cni --enable-default-cni --container-runtime=cri-o --bootstrapper=kubeadm

Note: For Kata Containers later than v1.6.1, the now default tcfilter networking of Kata Containers does not work for Minikube versions less than v1.1.1. Please ensure you use Minikube version v1.1.1 or above.

Check Minikube is running

Before you install Kata Containers, check that your Minikube is operating. On your guest:

$ kubectl get nodes

You should see your master node listed as being Ready.

Check you have virtualization enabled inside your Minikube. The following should return a number larger than 0 if you have either of the vmx or svm nested virtualization features available:

$ minikube ssh "egrep -c 'vmx|svm' /proc/cpuinfo"

Installing Kata Containers

You can now install the Kata Containers runtime components. You will need a local copy of some Kata Containers components to help with this, and then use kubectl on the host (that Minikube has already configured for you) to deploy them:

$ git clone https://github.com/kata-containers/kata-containers.git
$ cd kata-containers/tools/packaging/kata-deploy
$ kubectl apply -f kata-rbac/base/kata-rbac.yaml
$ kubectl apply -f kata-deploy/base/kata-deploy.yaml

This installs the Kata Containers components into /opt/kata inside the Minikube node. It can take a few minutes for the operation to complete. You can check the installation has worked by checking the status of the kata-deploy pod, which will be executing this script, and will be executing a sleep infinity once it has successfully completed its work. You can accomplish this by running the following:

$ podname=$(kubectl -n kube-system get pods -o=name | fgrep kata-deploy | sed 's?pod/??')
$ kubectl -n kube-system exec ${podname} -- ps -ef | fgrep infinity

NOTE: This check only works for single node clusters, which is the default for Minikube. For multi-node clusters, the check would need to be adapted to check kata-deploy had completed on all nodes.

Enabling Kata Containers

Note: Only Minikube/Kubernetes versions <= 1.13 require this step. Since version v1.14, the RuntimeClass is enabled by default. Performing this step on Kubernetes > v1.14 is however benign.

Now you have installed the Kata Containers components in the Minikube node. Next, you need to configure Kubernetes RuntimeClass to know when to use Kata Containers to run a pod.

$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/node-api/master/manifests/runtimeclass_crd.yaml > runtimeclass_crd.yaml

Register the runtime

Now register the kata qemu runtime with that class. This should result in no errors:

$ cd kata-containers/tools/packaging/kata-deploy/runtimeclasses
$ kubectl apply -f kata-runtimeClasses.yaml

The Kata Containers installation process should be complete and enabled in the Minikube cluster.

Testing Kata Containers

Launch a container that has been defined to run on Kata Containers. The enabling is configured by the following lines in the YAML file. See the Kubernetes Runtime Class Documentation for more details.

    spec:
      runtimeClassName: kata-qemu

Perform the following action to launch a Kata Containers based Apache PHP pod:

$ cd kata-containers/tools/packaging/kata-deploy/examples
$ kubectl apply -f test-deploy-kata-qemu.yaml

This may take a few moments if the container image needs to be pulled down into the cluster. Check progress using:

$ kubectl rollout status deployment php-apache-kata-qemu

There are a couple of ways to verify it is running with Kata Containers. In theory, you should not be able to tell your pod is running as a Kata Containers container. Careful examination can verify your pod is in fact a Kata Containers pod.

First, look on the node for a qemu running. You should see a QEMU command line output here, indicating that your pod is running inside a Kata Containers VM:

$ minikube ssh -- pgrep -a qemu

Another way to verify Kata Containers is running is to look in the container itself and check which kernel is running there. For a normal software container you will be running the same kernel as the node. For a Kata Container you will be running a Kata Containers kernel inside the Kata Containers VM.

First, examine which kernel is running inside the Minikube node itself:

$ minikube ssh -- uname -a

And then compare that against the kernel that is running inside the container:

$ podname=$(kubectl get pods -o=name | fgrep php-apache-kata-qemu | sed 's?pod/??')
$ kubectl exec ${podname} -- uname -a

You should see the node and pod are running different kernel versions.

Wrapping up

This guide has shown an easy way to setup Minikube with Kata Containers. Be aware, this is only a small single node Kubernetes cluster running under a nested virtualization setup. As such, it has limitations, but as a first introduction to Kata Containers, and how to install it under Kubernetes, it should suffice for initial learning and experimentation.