Notes from NDS-765.
Create Trial Account
Azure offers a $200 free trial: portal.azure.com
Provision Kubernetes Container Service
Followed basic instructions provided in Get started with a Kubernetes cluster in Container Service:
Download the Azure CLI
curl -L https://aka.ms/InstallAzureCli | bash
Login via Azure CLI
az login
Create resource group
az group create -n acsrg1 -l "westus"
Create Kubernetes cluster. You're only given 4 vcpus in the trial account, so will need to create only 1 master and 1 agent and change the VM size:
az acs create --orchestrator-type=kubernetes --resource-group acsrg1 --name=ndslabs --dns-prefix=ndslabs --agent-vm-size=Standard_D1_v2 --master-count=1 --agent-count=1
Install the Kubernetes client (creates kube-config profile)
az acs kubernetes install-cli
At this point you have a running Kubernetes cluster with single master, single compute/agent. Azure provisions a number of resources (storage, network, loadbalancer, etc).
I was able to make minor modifications to the ndslabs-startup templates to get a basic Labs Workbench running almost immediately. I've captured the hardcoded configuration in my ndslabs-repo on the azure branch.
- Privileged pods:
- works as expected
- Ingress:
- works as expected
- Wildcard DNS/TLS
- Wildcard DNS worked fine via Google domains
- *.azure.ndslabs.org > public IP if ingress ilb
- TLS should work (disabled during testing)
- Access to Kubernetes API
- apiserver assumes token path. Added TOKEN_PATH configuration to entrypoint.sh
- Access to etcd:
- Requires dedicated etcd (etcd.yaml)
- SMTP server
- Problem: Azure requires SMTP mail relay. This will require code changes
- Workaround – create accounts via ndslabsctl
- Node labeling:
- works as expected
- Loadbalancer/Public IP
- works as expected (via kubectl expose)
- Storage:
- Broken by default
- Need to use AzureFile or GlusterFS
- AzureFile
- Need to test how/if this works
- GlusterFS requires deploying Gluster cluster
Addons
Access to Kubedash via kubectl proxy
- Monitoring
- Heapster installed, but no Grafana by default
- Logging
- No Kibana/Fluentd or cluster-wide logging services
- DNS
- yes
- Container registry as separate service
Creating an azureFile volume:
- Create secret with azurestorageaccountkey, azurestorageaccountname
- Create PersistentVolume of type azureFile referencing secret
- Attach volume to pod
Activity log:
kubectl create -f loadbalancer.yaml kubectl expose rc nginx-ilb-rc --port=80 --type=LoadBalancer # Wait for endpoint IP to be assigned kubectl get sv kubectl create -f default-ingress.yaml kubectl create -f default-backend.yaml kubectl create -f etcd.yaml kubectl create -f apiserver.yaml kubectl create -f webui.yaml kubectl logs ndslabs-apiserver-bmc8h kubectl delete -f apiserver.yaml kubectl label nodes k8s-agent-5133fb85-0 ndslabs-node-role=compute kubectl logs ndslabs-apiserver-qkcgj kubectl proxy kubectl get pods --all-namespaces