...
- Bring up a (virtual) Kubernetes cluster
- Deploy Labs Workbench
- Ideally, process should mimic deployment in other environments
- Ideally, user can create new Docker containers and publish all in local environment
A secondary use case is the single-node installation for a project. This has come up a few times, but for example:
- SEAD/IML-CZO want to run Labs Workbench – want a simple install process that runs on a single VM during initial evaluation. May never scale up.
Current environment (ndslabs-startup)
...
- Minikube will work on Linux, but so far we haven't been able to use it on a VM (e.g., OpenStack). Hyperkube works for this.
Hyperkube (ndslabs-startup)
Our current ndslabs-startup/kube-up.sh relies on Hyperkube (Kubernetes in Docker)
...
- gcr.io/google_containers/hyperkube-amd64
- The CoreOS project provides a process for launching Kubernetes on CoreOS, using another hyperkube image
- https://coreos.com/kubernetes/docs/latest/kubernetes-on-generic-platforms.html
- Uses quay.io/coreos/hyperkube image
- This actually supports different configuration options (i.e., insecure-port) that might be helpful for some installations.
MacOS
- On MacOS, removing the /var/lib/kubelet volume actually works via kube-up.sh
- Adding bind container makes DNS reasonably easy
Windows 10
- Docker for Windows requires using Hyper-V (not Virtual Box)
- So far unable to get Hyperkube to work under Windows 10/Hyper-V without Vm.
- Our current ndslabs-startup/kube-up relies on Hyperkube, which is basically Kubernetes in Docker.
VirtualBox VM
- You must follow the steps described here: The CoreOS project provides a process for launching Kubernetes on CoreOS, using another hyperkube imagehttps://coreos.com/kubernetesos/docs/latest/kubernetesbooting-on-generic-platforms.htmlUses quay.io/coreos/hyperkube image-virtualbox.html
- You must add a Host-only adapter to your VM instance to SSH in
- You must manually load the .iso containing your ssh key, as described in the guide above
- You will a DNS entry pointing 192.168.99.XXX to a hostname, like www.local.ndslabs.org
- This can be done by populating /etc/hosts, or using a tool such as Dnsmasq
- You can disable TLS (this is likely optional)
- Still minor bugs ongoing (for example: sending e-mail for signup throws a 503)
Openstack VM
- Process works well for OpenStack VMs
- Editing PATH is annoying, but otherwise fine.
Azure VM
- Deployed Ubuntu 16.10 VM via Azure portal
- Standard DS2_v2 (2 Cores, 7 GB memory). (Wow, that takes a really long time!)
- Add endpoint for 443 access
- Start Labs Workbench
- apt-install docker.io
- git clone ndslabs-startup
- kube-up.sh (+ bash_profile changes)
- Same SMTP problem as before – but manual registration worked
AWS VM
- Deployed CoreOS 1298.5 stable via AWS Management Console
- t2.micro, because it was free! (way too small)
- Expose HTTP/HTTPS ports via security group
- Create a DNS rule pointing to the public IP of this node
- SSH in and start Labs Workbench
- git clone ndslabs-startup
- kube-up.sh (+ bash_profile changes)
- ndslabs-up.sh
- Same SMTP problem as before
Vagrant
Vagrant is a tool to manage virtual machines. The approach described below comes from CoreOS as a method to deploy a Kubernetes cluster via VirtualBox (or similar). Vagrant has multiple providers, so it might be possible to use the same Vagrant configuration to deploy a Kubernetes cluster on VirtualBox on your laptop and deploy a full cluster on OpenStack. However, it's not clear how useful this might be.
MacOS
The VirtualBox/Vagrant approach suggested by David works on MacOS. As with Minikube, this will not work on an OpenStack or other VM without running a different virtualization package (VirtualBox won't work in a VM). It's not clear that Vagrant will support this configuration.
...
- Domain name (dnsmasq)? For now add /etc/hosts entry
- IP address – IP of master node – same as used by kubeconfig
- ndslabs-up.sh
- kubectl label will change
- Email fails, but could output to log?
- No addons, currently
- Added bind service to support local DNS, based on instructions in
- http://www.damagehead.com/blog/2015/04/28/deploying-a-dns-server-using-docker/
- Worked like a charm, did have to add DNS server to network settings
Azure VM
- Deployed Ubuntu 16.10 VM
- Standard DS2_v2 (2 Cores, 7 GB memory). Wow, that takes a really long time.
- apt-install docker.io
- Standart kube-up.sh
- Need to add endpoint for 443
Recommendations
- Support both Minikube and Hyperkube installs of Labs Workbench to support
- Refactor ndslabs-startup to work under both Minikube and Hyperkube installs.
- Fix PATH
- Improve kube-up (monitor status)
- Improve ndslabs-up (more optional components, mustache, etc)