...

1. Looks like a very involved process, despite being partially automated (i.e. creating instances, networks, etc is not automated)
2. Specific requirements that we may not meet (both for OpenStack and for the deployment machine)

Setup:

1. docker run -it -v /home/smana/kargoconf:/etc/kargo quay.io/smana/k8s-kargocli:latest /bin/bash

...

1. docker run -it -v /home/core/private:/root/SAVED_AND_SENSITIVE_VOLUME ndslabs/deploy-tools:latest

2. pip install python-netaddr    # This did not work for me

Deploying via kargo (redux)

I had reasonable success deploying with Kargo from my dev VM on Nebula.

• The default Docker container didn't contain the Python OpenStack client and dependencies, so I created my own
  • https://github.com/craig-willis/kargo
  • craigwillis/kargo:latest
• I tried deploying with Fedora 25 cloud and CoreOS, but apparently Kargo has a hard requirement that the VM image have both python and fedora installed. I ended up going with fedora and manually installing python and docker-ce on each node, which could be automated or added to the base image

  • https://docs.docker.com/engine/installation/linux/fedora/#install-using-the-repository

  • It looks like we might be able to use the greoup_vars/bootstrap_os option

• docker run -it -v `pwd`/kargoconf:/etc/kargo craigwillis/kargo bash

kargoconf/kargo.yml (these can also be flags)

# Common options
# ---------------
kargo_git_repo: "https://github.com/kubespray/kargo.git"
loglevel: "info"

# OpenStack options
# ---
os_auth_url: "http://nebula.ncsa.illinois.edu:5000/v2.0"
os_username: "me"
os_password: "my-password"
os_project_name: "NDSLabsDev"
masters_flavor: "m1.medium"
nodes_flavor: "m1.medium"
etcds_flavor: "m1.medium"
image: "fedora-25-cloud"
network: "NDSLabsDev"
sshkey: "my-key"

Then it just required two commands (both produce tons o' ansible output

kargo openstack --nodes 3

kargo deploy -k your.pem -u <os user i.e., fedora>

At this point, I have a 3-node kubernetes cluster with 2 masters and 2 etcds.  I manually assigned a public IP to one node, manually assigned a label (ndslabs-public-ip: true) and selector to the loadbalancer. Deployed via ndslabs-startup and all is good.

Pros:

• It works and is maintained by someone else
• It's almost identical to our ansible deploy process.

Cons:

• No /etc/hosts entries in container, so I must ssh to IP, but this could easily be fixed
• Kubectl doesn't work from kargo container, so I must ssh to node, but this could easily be fixed