This will walk you through setting up GlusterFS backed by Kubernetes Persistent Volume Claims.
Get Started on GCE
More details: Labs Workbench on GKE
- https://console.cloud.google.com and sign up for GCE (free up to $300 or 365 days)
- Expand top-left menu and choose Container Engine > Container clusters
- Click "Create New Cluster"
- Select cluster parameters and click "Create"
- Wait for your cluster to come online
- Click "Connect" next to your cluster and copy the kubeconfig line into your clipboard
- Open a shell using the button at the top-right
- Paste the kubeconfig info from your clipboard
You should now be ready to start playing around with Kubernetes on GCE.
Running GlusterFS
The general process will be as follows:
- git clone https://github.com/bodom0015/gluster -b NDS-785 && cd gluster/templates/
- kubectl create -f claim1.json -f claim2.json
- kubectl get pvc,pv
- kubectl create -f gluster.svc.yaml -f gluster.rc.yaml
- export ENDPOINT_1=$(kubectl get ep glusterfs-cluster | grep -v ENDPOINTS | awk '{print $2}' | awk 'BEGIN { FS=","; }{print $1}' | awk 'BEGIN { FS=":"; }{print $1}')
- export ENDPOINT_2=$(kubectl get ep glusterfs-cluster | grep -v ENDPOINTS | awk '{print $2}' | awk 'BEGIN { FS=","; }{print $2}' | awk 'BEGIN { FS=":"; }{print $1}')
- kubectl exec -it `kubectl get pod | grep glfs-server-1 | awk '{print $1}'` sh
- gluster peer probe <endpoint ip>
- gluster volume create $VOLNAME $VOLSPEC <endpoint 1 from above>:/media/brick0 <endpoint 2 from above>:/media/brick0
- gluster volume start global
You are now running a 2-brick replicated GlusterFS setup on top of Kubernetes.
Test Case
To test out this shared filesystem, run two busybox YAMLs mounting the same GlusterFS volume:
apiVersion: v1 kind: Pod metadata: name: busybox1 namespace: default spec: containers: - image: busybox command: - sleep - "3600" volumeMounts: - name: "glfs" mountPath: "/var/glfs/global" imagePullPolicy: IfNotPresent name: busybox volumes: - name: glfs glusterfs: endpoints: glusterfs-cluster path: global restartPolicy: Always --- apiVersion: v1 kind: Pod metadata: name: busybox2 namespace: default spec: containers: - image: busybox command: - sleep - "3600" volumeMounts: - name: "glfs" mountPath: "/var/glfs/global" imagePullPolicy: IfNotPresent name: busybox volumes: - name: glfs glusterfs: endpoints: glusterfs-cluster path: global restartPolicy: Always
- kubectl create -f busybox.yaml
- kubectl get pods -o wide
- kubectl exec -it busybox1 sh
- echo 'Hello, GLFS!' >> /var/glfs/global/testing && exit
- kubectl exec -it busybox2 cat /var/glfs/global/testing