Overview
The NDSLabs API is a RESTful interface to the NDSLabs system. The API is implemented via the NDSLabs API Server.
Key Concepts
- Project: A named configuration that relates an administrative user, likely represented by a client certificate, to a set of resource allocations (quotas), and resources (namespace, pods/services, volumes, configurations).
- Stack: A set of services that are administratively related (i.e., started and stopped together). For example, "clowder" or "dataverse"
- Service definition: A specification of a logical "service" that includes a name, description, storage requirements, configuration options, and relationships to other services. An NDSLabs "service" may be composed of multiple related containers.
- Service library: A collection of service definitions and procedures for adding official/trusted as well as local/development services.
- Service instance: An instance of a service configured and running in a project namespace.
- Resource quota: A set of soft and hard limits assigned to a project (storage and compute).
- Volume: A named, allocated storage resource that counts against the project quota and can be mounted by one or more services.
A Simple Use Case
Using the CLI for simplicity: a project administrator wants to add a set of services to their project configuration:
Command | What it does | Response |
---|---|---|
list services | Lists the services in the service library that can be added to this project. |
|
list resources | Lists the storage resources available to this project | storage quota: 1TB |
get service clowder | Gets the service specification | <spec> |
add service clowder | Adds the specified service to the project | OK |
get config clowder | Gets the configuration options available for the service | <config> |
set config clowder smtp-host smtp.ncsa.illinois.edu | Sets a configuration option for the named services | OK |
create volume clowder-vol 10GB | Creates and formats a volume named "clowder-vol" | OK |
attach clowder-vol clowder | Makes the volume available to the service | OK |
start clowder | Starts the service | <status> |
status clowder | Returns the service status | <status> |
endpoint clowder | Returns the service endpoint | <endpoint> |
add service image-preview | Adds the specified service to the project | OK |
link image-preview clowder | Links the specified services (in this case, by adding required rabbitmq) | OK |
start image-preview | Starts the image preview service and updates the clowder service? | <status> |
stop services | Stops everything | <status> |
delete volume clowder-vol | Removes the specified volume | OK |
delete clowder image-preview | Removes the services from the project | OK |
Entities
The following is a simple entity-relationship diagram intended to capture the entities, attributes, and relationships for the NDSLabs API.
Gliffy Diagram | ||||||
---|---|---|---|---|---|---|
|
Draft REST API
For actor definitions, see NDS Labs Use Cases.
...
This can be opened using the Swagger editor demo http://editor.swagger.io/, if desired.
Authentication
We will likely use a JSON Web-Token approach (JWT)
The interceptor model described here https://thinkster.io/angularjs-jwt-auth also looks like a handy way to keep things alive without persisting stuff on the client.
- Password will be validated against the server-side "project" structure
Volumes
- If development environment, creating a volume is just a mkdir on some local path
- If production environment, creating a volume requires a call to the Openstack API (Create, Attach) then a local call to mkfs, then a call to the Openstack API (Detach)
API Server (prototype)
The prototype NDSLabs API Server will be used by the CLI and GUI applications.
Components
We envision the following components:
...
Gliffy Diagram | ||||
---|---|---|---|---|
|
API Authentication
- For the CLI, we could follow the Kubernetes API server authentication model using client certificates. They use openssl/easyrsa to sign client certificates where the common name is the username (or in our case, project namespace).
- This is not useful for the GUI, which might require user authentication. Perhaps we can pre-generate a password?
Service definitions
How will we manage service definitions going forward so that new services can be added without requiring access to the nds-labs repository? We need to handle two cases: official or trusted services and services under development. The Docker model might suit us: official service definitions are stored in an SCM repo. A repo watcher polls HEAD for changes and imports new definitions into the service library (e.g., etcd). We can use pull requests to handle trust and verification. To handle development, we could store "draft" definitions in a way that are only accessible to a specific project. For example, the developer could issue "cli add service -f myservice.spec" and the service would be added to services/{namespace} or namespace/services. The "cli list services" would return both official and project-local definitions.