Multi Node Kubernetes on CoreOS with Vagrant

This is good way to run a multi-node cluster quickly in a single host for dev/testing.   It can be enhanced with small VBox volumes and an additional VBox host network to implement core infrastructure parts like gluster and ingress.   It can be left running, vagrant suspend/resume can freeze/thaw the whole cluster in a few seconds if you want to work on something else and don't want it holding resources (like on a laptop). 

Here is the recipe, adapted from the official instructions here:

1. Install VirtualBox
2. Install Vagrant
   1. Check vagrant: "vagrant box list"
3. Install kubectl
   1. curl -O https://storage.googleapis.com/kubernetes-release/release/v1.5.2/bin/linux/amd64/kubectl
   2. install in /usr/local/bin or your favorite spot in your PATH
4. git clone https://github.com/coreos/coreos-kubernetes.git
5. cd multinode/vagrant
6. vagrant box add http://alpha.release.core-os.net/amd64-usr/current/coreos_production_virtualbox.json
   1. will download CoreOS for virtualbox
7. cp config.rb.sample to config.rb and edit:
   1. Pick the channel and node sizes and counts according to your laptop resources (primarily memory), here's one example:
      

      $update_channel="alpha"

      $controller_count=1

      $controller_vm_memory=512

      $worker_count=2

      $worker_vm_memory=1024

      $etcd_count=1

      $etcd_vm_memory=512

8. Good to check your networking for collisions on unrouted address ranges that you'll encounter at work, home, etc.
   1. The default VM network specs are in the Vagrantfile:  172.17.4/24

9. vagrant up --provider virtualbox

10. export KUBECONFIG="${KUBECONFIG}:$(pwd)/kubeconfig"

    1. Alt:   Add the config to any existing ~/.kube/config if you are working with multiple clusters and or namespaces (kubectl set-cluster, kubectl set-context, etc.)

11. Should be operational, some useful commands (use in the multinode/vagrant path)

    1. vagrant ssh c1 (or the other names) to get into coreos

    2. vagrant status

    3. vagrant suspend/resume

       1. Once in a while net doesn't come back, and kubectl says:  

          The connection to the server 172.17.4.101:443 was refused - did you specify the right host or port?

       2. In this case you can vagrant halt, vagrant up

    4. vagrant destroy (obliterates)

       1. Just vagrant up and restart afresh

Enjoy,

D