Skip to content

Latest commit

 

History

History
303 lines (206 loc) · 10.3 KB

mesos-docker.md

File metadata and controls

303 lines (206 loc) · 10.3 KB

WARNING WARNING WARNING WARNING WARNING

PLEASE NOTE: This document applies to the HEAD of the source tree

If you are using a released version of Kubernetes, you should refer to the docs that go with that version.

The latest 1.0.x release of this document can be found [here](http://releases.k8s.io/release-1.0/docs/getting-started-guides/mesos-docker.md).

Documentation for other releases can be found at releases.k8s.io.

Getting Started With Kubernetes on Mesos on Docker

The mesos/docker provider uses docker-compose to launch Kubernetes as a Mesos framework, running in docker with its dependencies (etcd & mesos).

Cluster Goals

  • kubernetes development
  • pod/service development
  • demoing
  • fast deployment
  • minimal hardware requirements
  • minimal configuration
  • entry point for exploration
  • simplified networking
  • fast end-to-end tests
  • local deployment

Non-Goals:

  • high availability
  • fault tolerance
  • remote deployment
  • production usage
  • monitoring
  • long running
  • state persistence across restarts

Cluster Topology

The cluster consists of several docker containers linked together by docker-managed hostnames:

Component Hostname Description
docker-grand-ambassador Proxy to allow circular hostname linking in docker
etcd etcd Key/Value store used by Mesos
Mesos Master mesosmaster1 REST endpoint for interacting with Mesos
Mesos Slave (x2) mesosslave1
mesosslave2		 Mesos agents that offer resources and run framework executors (e.g. Kubernetes Kublets)
Kubernetes API Server apiserver REST endpoint for interacting with Kubernetes
Kubernetes Controller Manager controller
Kubernetes Scheduler scheduler Schedules container deployment by accepting Mesos offers

Prerequisites

Required:

Optional:

  • Virtual Box - x86 hardware virtualizer
    • Required by Boot2Docker and Docker Machine

Install on Mac (Homebrew)

It's possible to install all of the above via Homebrew on a Mac.

Some steps print instructions for configuring or launching. Make sure each is properly set up before continuing to the next step.

brew install git
brew install caskroom/cask/brew-cask
brew cask install virtualbox
brew install docker
brew install boot2docker
boot2docker init
boot2docker up
brew install docker-compose

Install on Linux

Most of the above are available via apt and yum, but depending on your distribution, you may have to install via other means to get the latest versions.

It is recommended to use Ubuntu, simply because it best supports AUFS, used by docker to mount volumes. Alternate file systems may not fully support docker-in-docker.

Boot2Docker Config (Mac)

If on a mac using boot2docker, the following steps will make the docker IPs (in the virtualbox VM) reachable from the host machine (mac).

  1. Set the VM's host-only network to "promiscuous mode":

    boot2docker stop
VBoxManage modifyvm boot2docker-vm --nicpromisc2 allow-all
boot2docker start

    This allows the VM to accept packets that were sent to a different IP.

    Since the host-only network routes traffic between VMs and the host, other VMs will also be able to access the docker IPs, if they have the following route.

  2. Route traffic to docker through the boot2docker IP:

    sudo route -n add -net 172.17.0.0 $(boot2docker ip)

    Since the boot2docker IP can change when the VM is restarted, this route may need to be updated over time. To delete the route later: sudo route delete 172.17.0.0

Walkthrough

  1. Checkout source

    git clone https://github.com/GoogleCloudPlatform/kubernetes
cd kubernetes

    By default, that will get you the bleeding edge of master branch. You may want a release branch instead, if you have trouble with master.

  2. Build binaries

    KUBERNETES_CONTRIB=mesos hack/build-go.sh

    Alternatively, you can use make, if make is installed.

    Unless you're on linux, you'll also need to build the binaries for linux/amd64 (for the docker containers):

    KUBERNETES_CONTRIB=mesos build/run.sh hack/build-go.sh

    Breakdown:

    • KUBERNETES_CONTRIB=mesos - enables building of the contrib/mesos binaries
    • build/run.sh - executes a command in the build container
    • build-go.sh - builds the Go binaries for the current architecture (linux/amd64 when in a docker container)

  3. [Optional] Build docker images

    The following docker images are built as part of ./cluster/kube-up.sh, but it may make sense to build them manually the first time because it may take a while. In the future some of these may be hosted publicly, but you will always need to at least rebuild the Kubernetes-Mesos image when using locally built binaries.

    Test image includes all the dependencies required for running e2e tests.

    ./cluster/mesos/docker/test/build.sh

    Kubernetes-Mesos image includes the compiled linux binaries.

    ./cluster/mesos/docker/km/build.sh

  4. [Optional] Configure Mesos resources

    By default, the mesos-slaves are configured to offer a fixed amount of resources (cpus, memory, disk, ports). If you want to customize these values, update the MESOS_RESOURCES environment variables in ./cluster/mesos/docker/docker-compose.yml. If you delete the MESOS_RESOURCES environment variables, the resource amounts will be auto-detected based on the host resources, which will over-provision by > 2x.

    If the configured resources are not available on the host, you may want to increase the resources available to Docker Engine. You may have to increase you VM disk, memory, or cpu allocation in VirtualBox, Docker Machine, or Boot2Docker.

  5. Configure provider

    export KUBERNETES_PROVIDER=mesos/docker

    This tells cluster scripts to use the code within cluster/mesos/docker.

  6. Create cluster

    ./cluster/kube-up.sh

    If you manually built all the above docker images, you can skip that step during kube-up:

    MESOS_DOCKER_SKIP_BUILD=true ./cluster/kube-up.sh

    After deploying the cluster, ~/.kube/config will be created or updated to configure kubectl to target the new cluster.

  7. Explore examples

    To learn more about Pods, Volumes, Labels, Services, and Replication Controllers, start with the Kubernetes Walkthrough.

    To skip to a more advanced example, see the Guestbook Example

  8. Destroy cluster

    ./cluster/kube-down.sh

Addons

The kube-up for the mesos/docker provider will automatically deploy KubeDNS and KubeUI addons as pods/services.

Check their status with:

./cluster/kubectl.sh get pods --namespace=kube-system

KubeUI

The web-based Kubernetes UI is accessible in a browser through the API Server proxy: https://<apiserver>:6443/ui/.

By default, basic-auth is configured with user admin and password admin.

The IP of the API Server can be found using ./cluster/kubectl.sh cluster-info.

End To End Testing

Warning: e2e tests can take a long time to run. You may not want to run them immediately if you're just getting started.

While your cluster is up, you can run the end-to-end tests:

./cluster/test-e2e.sh

Notable parameters:

  • Increase the logging verbosity: -v=2
  • Run only a subset of the tests (regex matching): -ginkgo.focus=<pattern>

To build, deploy, test, and destroy, all in one command (plus unit & integration tests):

make test_e2e

Kubernetes CLI

When compiling from source, it's simplest to use the ./cluster/kubectl.sh script, which detects your platform & architecture and proxies commands to the appropriate kubectl binary.

ex: ./cluster/kubectl.sh get pods

Helpful scripts

  • Kill all docker containers

    docker ps -q -a | xargs docker rm -f

  • Clean up unused docker volumes

    docker run -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/lib/docker --rm martin/docker-cleanup-volumes

Analytics