...
- Kubernetes support: Since the workbench is based on Kubernetes, storage options must be supported or supportable in the Kubernetes framework. This means that the storage can be mounted in a Pod/container.
- Support standard Kubernetes volume semantics and practices for multi-tenant clusters
- Distributed: Storage solution must support distribution across multiple nodes.
- Quotas: Storage solution must support per-project quotas
- Scalability: Storage solution must be scalable
- Reliability: Storage solution must be reliable, or support a balance in the reliability/performance trade-off
- Performance: Storage solution must be performant, or support a balance in the performance/reliability trade-off
- Security: Security between groups/users/projects in storage should follow standard parctices and polices - i.e. mirror standard linux user/group models
Storage options
We have explored the following storage options:
| Storage | Pros | Cons |
|---|---|---|
| Local storage | Supported by Kubernetes Simple Useful for single-node or development configurations | Not distributed Not scalable Does not support quotas |
| NFS | Supported by Kubernetes Distributed | Not scalable Performance requires tuning |
| OpenStack Cinder | Supportable by Kubernetes (with changes) | Requires change to Kubernetes Limits potential run-times to OpenStack |
| GlusterFS | Supported by Kubernetes for in-pod storage Scalable and adaptable - growth, performance, | Not supported directly on cloud-oriented OS's |
Creating a GlusterFS server cluster
The basic steps for creating a GlusterFS cluster on Nebula include:
...
Currently, we are planning to allocate one volume per project. Volumes , but we should anticipate the need for per-pod as it will likely be needed.
Volumes will be expandable by the admin and deleted by the admin tools when the project is deleted
...