Overview
The NDSLabs API is a RESTful interface to the NDSLabs system. The API is implemented via the NDSLabs API Server.
Key Concepts
- Project: A named configuration that relates an administrative user, likely represented by a client certificate, to a set of resource allocations (quotas), and re
sources (namespace, pods/services, volumes, configurations). - Service definition: A specification of a logical "service" that includes a name, description, storage requirements, configuration options, and relationshi
ps to other services. An NDSLabs "service" may be composed of multiple related containers. - Service library: A collection of service definitions and procedures for adding official/trusted as well as local/development services.
- Service instance: An instance of a service configured and running in a project namespace.
- Resource quota: A set of soft and hard limits assigned to a project (storage and compute).
- Volume: A named, allocated storage resource that counts against the project quota and can be mounted by one or more services.
A Simple Use Case
Using the CLI for simplicity: a project administrator wants to add a set of services to their project configuration:
| Command | What it does | Response |
|---|---|---|
| list services | Lists the services in the service library that can be added to this project. |
|
| list resources | Lists the storage resources available to this project | storage quota: 1TB |
| get service clowder | Gets the service specification | <spec> |
| add service clowder | Adds the specified service to the project | OK |
| get config clowder | Gets the configuration options available for the service | <config> |
| set config clowder smtp-host smtp.ncsa.illinois.edu | Sets a configuration option for the named services | OK |
| create volume clowder-vol 10GB | Creates and formats a volume named "clowder-vol" | OK |
| attach clowder-vol clowder | Makes the volume available to the service | OK |
| start clowder | Starts the service | <status> |
| status clowder | Returns the service status | <status> |
| endpoint clowder | Returns the service endpoint | <endpoint> |
| add service image-preview | Adds the specified service to the project | OK |
| link image-preview clowder | Links the specified services (in this case, by adding required rabbitmq) | OK |
| start image-preview | Starts the image preview service and updates the clowder service? | <status> |
| stop services | Stops everything | <status> |
| delete volume clowder-vol | Removes the specified volume | OK |
| delete clowder image-preview | Removes the services from the project | OK |
Entities
The following is a simple entity-relationship diagram intended to capture the entities, attributes, and relationships for the NDSLabs API.
Draft REST API
This is a first pass at a set of REST APIs that support the following workflows. For actor definitions, see NDS Labs Use Cases.
- Cluster admin can manage the "service library" (collection of definitions of available services, similar to the Charm store)
- Cluster admin can manage "projects" (a project is a namespace and set of storage/computation quotas)
- Project admin can view all available site services and storage resources.
- Project admin can manage deployed services (add/remove), associate services where necessary, and assign volumes to services.
Base path: /api/{version}/
| Path | Action | Project Admin | Cluster Admin | Notes |
|---|---|---|---|---|
| /services | List, add site-wide and project-specific services | GET, PUT | PUT | |
| /services/{service-id} | Get, update, delete site-wide services | GET | PUT, DELETE | |
| /projects | List, add projects | GET, PUT | ||
| /projects/{project-id} | Get, update, delete project | GET, PUT | DELETE | |
| /projects/{project-id}/serviceInstances | List, add project services | GET, PUT | ||
| /projects/{project-id}/serviceInstances/{instance-id} | Get, update, delete project service | GET, PUT, DELETE | ||
| /projects/{project-id}/serviceInstances/{instance-id}/status | Get, update, delete project services status | GET, PUT, DELETE | ||
| /projects/{project-id}/volumes | List, add project volumes | GET, PUT | ||
| /projects/{project-id}/volumes/{volume-id} | Get, update, delete project volumes | GET, PUT, DELETE |
A first draft of the spec is available via git on the NDS-108 branch
https://github.com/nds-org/nds-labs/blob/NDS-108/apiserver/api/swagger-spec/ndslabs.yaml
This can be opened using the Swagger editor demo http://editor.swagger.io/, if desired.
API Server (prototype)
The prototype NDSLabs API Server will be used by the CLI and GUI applications.
Components
We envision the following components:
- CLI
- API Server (either a Jetty application or Go server)
- etcd: Store for service library, project definitions, and services
- Kubernetes API client (for interaction with Kubernetes services, etc).
- Openstack client to handle volume allocation/initialization tasks
API Authentication
- For the CLI, we could follow the Kubernetes API server authentication model using client certificates. They use openssl/easyrsa to sign client certificates where the common name is the username (or in our case, project namespace).
- This is not useful for the GUI, which might require user authentication. Perhaps we can pre-generate a password?
Service definitions
How will we manage service definitions going forward so that new services can be added without requiring access to the nds-labs repository? We need to handle two cases: official or trusted services and services under development. The Docker model might suit us: official service definitions are stored in an SCM repo. A repo watcher polls HEAD for changes and imports new definitions into the service library (e.g., etcd). We can use pull requests to handle trust and verification. To handle development, we could store "draft" definitions in a way that are only accessible to a specific project. For example, the developer could issue "cli add service -f myservice.spec" and the service would be added to services/{namespace} or namespace/services. The "cli list services" would return both official and project-local definitions.