...
- Bring up a (virtual) Kubernetes cluster
- Deploy Labs Workbench
- Ideally, process should mimic deployment in other environments
- Ideally, user can create new Docker containers and publish all in local environment
Current environment (ndslabs-startup)
For developers, we achieve this with the ndslabs-startup repo. A simple shell script kube-up.sh brings up Kubernetes via Hyperkube. This requires access to /var/lib/docker and /var/lib/kubelet, which has proven problematic on MacOS and Windows with the newer Docker installs. A second shell script ndslabs-up.sh collects information from the user (domain name, IP address, email address) and uses kubectl to deploy a number of templated YAML files.
...
- Simple, scripted process gets things running simply and quickly
Cons:
- Doesn't currently work on MacOS/WindowsWindows – requires running a separate VM
- Repo is messy and has unused stuff (could be cleaned up)
- "Mustache" variable replacement is ridiculousinflexible.
- Doesn't include support for local DNS (can be added easily with bind container)
- Templates are duplicated in deploy-tools repository
...
Other options:
- Minikube (works on MacOS, Windows, Linux – doesn't require Docker; but also doesn't work on VMOpenStack VMs)
- Hyperkube (works well on VMs running Docker; doesn't work well on MacOS/Windows due to filesystem permission problems)
- Vagrant (works on MacOS via VirtualBox)
- Deploy-tools (could be used to deploy single-node without hyperkube)
Minikube
Minikube runs Kubernetes in a single VM .and sets up kubeconfig environment
- Requires virtualization environment, like such as VirtualBox
- Download binary and run
- minikube start creates a VM in VirtualBox (or other virtualization environment)
- Sets up kubeconfig and local environment to access VM via kubectl
- In general it's functional and easy. I've run into snags for our development (i.e., running apiserver binary locally, trying to connect to kubernetes via Minikube), but this is an advanced case.
Minikube on MacOS
- Install VirtualBox
- Download binary
- minikube start
- git clone ndslabs-up.shstartup
- Don't need kube-up.sh, since we're running Minikube
- ndslabs-up.sh
- Local DNS via bind works!
- IP detection, etcd, etc, won't work as-is
- Labeling won't work as is.
Minikube Windows 10
- Install VirtualBox (can also use Hyper-V, but not tested here)
- Disable Hyper-V: Settings > Turn Windows Features on or off; uncheck Hyper-V
- Download minikube for Windows
- minikube.exe start
- Will make changes to VirtualBox
- Works well on Windows
- Hit snags with bind for local DNS, but likely user error configuring Windows DNS.
- Minikube was much easier than trying to run Docker on Windows.
Linux/VM
- Minikube will work on Linux, but so far we haven't been able to use it on a VM (e.g., OpenStack). Hyperkube works for this.
...
Vagrant is a tool to manage virtual machines. The approach described below comes from CoreOS as a method to deploy a Kubernetes cluster via VirtualBox (or similar). Vagrant has multiple providers, so it might be possible to use the same Vagrant configuration to deploy a Kubernetes cluster on VirtualBox on your laptop and deploy a full cluster on OpenStack. However, it's not clear how useful this might be.
MacOS
The VirtualBox/Vagrant approach suggested by David works on MacOS. As with Minikube, this will not work on an OpenStack or other VM without running a different virtualization package (VirtualBox won't work in a VM). It's not clear that Vagrant will support this configuration.
...
- Domain name (dnsmasq)? For now add /etc/hosts entry
- IP address – IP of master node – same as used by kubeconfig
- ndslabs-up.sh
- kubectl label will change
- Email fails, but could output to log?
- No addons, currently
- Added bind service to support local DNS, based on instructions in
- http://www.damagehead.com/blog/2015/04/28/deploying-a-dns-server-using-docker/
- Worked like a charm, did have to add DNS server to network settings
Windows 10
- Install Virtualbox
- Install vagrant
- Install git
...