Cluster Boot Sequence

1. Decide on cluster size. How many of each type of node?
   • kubernetes etcd
   • kubernetes master
   • kubernetes compute
   • kubernetes ingress
   • gluster storage
2. NDS-243 - Allocate desired nodes with Ansible playbook.
   • Wait for nodes to be online with Kubernetes running.
3. Label all nodes with their associated role/type from above
   • kubectl label nodes <nodeName> ndslabs-role=<nodeType>
   • Do we need a script to create these in bulk? File a new ticket if so...
     • NDS-251
4. NDS-223 - Deploy GlusterFS server Kubernetes service on nodes with ndslabs-role=storage
5. NDS-223 - Deploy GlusterFS client Kubernetes service on nodes with ndslabs-role=compute
6. NDS-214 - Deploy "default" Kubernetes LoadBalancer / Ingress rules for UI / API / cluster-wide services (i.e. TERRA toolserver)
7. NDS-244 - Deploy kube-system Kubernetes services
   • Grafana, dashboard, ELK, etc
8. NDS-238 - Depoy NDS Labs Kubernetes services
   • UI / API
   • TERRA toolserver?
9. Create any necessary NDS Labs projects using the API
   • Each project needs:
     • Public IP / CNAME
     • a storage volume (limited by the above quota)
     • resource limits/quotas
     • Project in etcd under /ndslabs/projects
     • Kubernetes Secret?
     • Kubernetes Namespace
     • Kubernetes LoadBalancer
     • Kubernetes Ingress ruleset
   • Do we need a script to create these in bulk? File a new ticket if so...
     • NDS-252

Ansible Overview

...

Overview

See NDS Labs Deploy Tools

Ansible configuration is held in FILES.deploy-tools/etc/ansible/ansible.cfg

/root contains most of the interesting bits:

• Playbooks - declare order and assignment of tasks / roles, defined elsewhere in /usr/local/lib/ndslabs/ansible

...

• Playbooks - declare order and assignment of tasks /roles
  • Tasks / Roles - declare the commands executed by each task / role
• Inventories - declarative file containing desired configuration
  • Servers - names and quantity of each node type
    • etcd: kubernetes / ndslabs key-value store dedicated node
    • glfs: glusterfs storage nodes
    • master: kubernetes master node
    • k8compute: compute / worker nodes
    • loadbal: load balancer node (requires public ip)
  • Groups
    • cluster1 - groups nodes and quantites
    • openstack - groups all openstack targets together (all nodes for now)
    • coreos - groups all coreos instances together (all nodes for now)
    • publicip - groups together all machines requiring a public ip (only load balancer for now)
  • Configuration
    • GlusterFS bricks / OpenStack volumes
    • OpenStack instance SSH key / image / flavor
• Group Vars - configuration options for each group
  
  • openstack
  • coreos
  • publicip

Configuration

...

Running Ansible

OpenRC File

...