Skip to end of metadata
Go to start of metadata

Overview

The NDSLabs API is a RESTful interface to the NDSLabs system. The API is implemented via the NDSLabs API Server.

Key Concepts

  • Project:  A named configuration that relates an administrative user to a set of resource allocations (quotas), and resources (namespace, pods/services, volumes, configurations).
  • Application (aka stack): A set of services that are administratively related (i.e., started and stopped together). For example, "clowder" or "dataverse"
  • Service specificationA specification of a logical "service" that includes a name, description, storage requirements, configuration options, and relationships to other services. An NDSLabs "service" may be composed of multiple related containers.
  • Service library: A collection of service definitions and procedures for adding official/trusted as well as local/development services.
  • Application instance: An instance of a service configured and running in a project namespace.
  • Resource quota: A set of soft and hard limits assigned to a project (storage and compute).
  • Volume mount: A named, allocated storage resource that counts against the project quota and can be mounted by one or more services.

A Simple Use Case

Using the CLI for simplicity: a project administrator wants to add a set of services to their project configuration:

CommandWhat it doesResponse
list servicesLists the services in the service library that can be added to this project.

clowder

image-preview

get service clowderGets the service specification<spec>
add service clowderAdds the specified service to the projectOK
set env smtp-host smtp.ncsa.illinois.eduSets a configuration option for the named servicesOK
start clowderStarts the service<status>
status clowderReturns the service status<status>
add service image-previewAdds the specified service to the projectOK
start image-previewStarts the image preview service and updates the clowder service?<status>
stop servicesStops everything<status>

Entities

The following is a simple entity-relationship diagram intended to capture the entities, attributes, and relationships for the NDSLabs API.

ndslabs-api-server-erd

REST API

For actor definitions, see NDS Labs Use Cases.

 

Base path: /api/

PathActionProject AdminCluster AdminNotes
/List available pathsGETGET 
/authenticate PUTPUTreturns token
/servicesList, add site-wide and project-specific servicesGET, PUTPUT 
/services/{service-id}Get, update, delete site-wide servicesGETPUT, DELETE 
/services/templates/{service-id}/{service|controller}Put, get, delete service templatesGETPUT, DELETE 
/projectsList, add projects GET, PUT 
/projects/{project-id}Get, update, delete projectGET, PUTDELETE 
/projects/{project-id}/serviceInstancesList, add project servicesGET, PUT  
/projects/{project-id}/serviceInstances/{instance-id}Get, update, delete project serviceGET, PUT, DELETE  
/projects/{project-id}/serviceInstances/{instance-id}/statusGet, update, delete project services statusGET, PUT, DELETE start, stop, restart
/projects/{project-id}/serviceInstance/{instance-id}/configGet, update, delete service configurationsGET, PUT, DELETE  
/projects/{project-id}/volumesList, add project volumesGET, PUT  
/projects/{project-id}/volumes/{volume-id}Get, update, delete project volumesGET, PUT, DELETE  
/projects/{project-id}/{stack-id}Get, update, delete service stackGET, PUT, DELETE  
/projects/{project-id}/{stack-id}/actions/{action}Start/Stop a stackGET,PUT  
/projects/{project-id}/{stack-id}/services/{service-id}Get, update, delete services in a stackGET, PUT, DELETE  
/versionGet server versionGETGET 
/refresh_tokenGet a new tokenGETGET 
/registerPost a new project POST 
/consoleGet a Websocket connection to a consoleGET  

 

 

The latest version of the API specification is available in Github:

https://github.com/nds-org/ndslabs/blob/master/apis/swagger-spec/ndslabs.yaml

This can be opened using the Swagger editor demo http://editor.swagger.io/, if desired.

Authentication

The API Server currently uses the JSON Web-Token (JWT) approach. The basic flow is as follows:

  • POST to /authenticate 
    • {"username": "demo", "password": "12345"}
  • response
    • {"token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE0NTY4NzE3ODgsImlkIjoiZGVtbyIsIm9yaWdfaWF0IjoxNDU2ODY5OTg4fQ.pJ2CQyqXDV675KrAtz3qVzwbM7k-tnZ28Pc0o81GtGU"}

  • For all subsequent requests, include the following header:
    • Authorization: Bearer TOKEN

  • To refresh the token, simply GET /refresh_token, this will retrieve an updated token

Volumes

  • If development environment, creating a volume is just a mkdir on some local path
  • If production environment, creating a volume requires a call to the Openstack API (Create, Attach) then a local call to mkfs, then a call to the Openstack API (Detach)

API Server (prototype)

The prototype NDSLabs API Server will be used by the CLI and GUI applications. 

Components

We envision the following components:

  • CLI
  • API Server (either a Jetty application or Go server)
  • etcd: Store for service library, project definitions, and services
  • Kubernetes API client (for interaction with Kubernetes services, etc).
  • Openstack client to handle volume allocation/initialization tasks

 

ndslabs-api-server-v1

 

API Authentication

  • For the CLI, we could follow the Kubernetes API server authentication model using client certificates. They use openssl/easyrsa to sign client certificates where the common name is the username (or in our case, project namespace). 
  • This is not useful for the GUI, which might require user authentication. Perhaps we can pre-generate a password?

Service definitions

How will we manage service definitions going forward so that new services can be added without requiring access to the nds-labs repository? We need to handle two cases: official or trusted services and services under development. The Docker model might suit us: official service definitions are stored in an SCM repo. A repo watcher polls HEAD for changes and imports new definitions into the service library (e.g., etcd). We can use pull requests to handle trust and verification.  To handle development, we could store "draft" definitions in a way that are only accessible to a specific project.  For example, the developer could issue "cli add service -f myservice.spec" and the service would be added to services/{namespace} or namespace/services. The "cli list services" would return both official and project-local definitions.

  • No labels