As part of CDDR project, we are revisiting the old "tool launcher" concept for new version of workbench.
It is expected that there is a 1:1 Clowder/WB pairing and don't currently plan to support multiple workbenches for a single Clowder. Current proposed architecture involves development on both workbench and Clowder side:
CLOWDER
- Implement API actions that can be queued for workbench
- Send dataset 123 to workbench
- Start container foo
- Stop container bar
- ...
- EST: 4 days for implementing API queue service
- 2 days for api endpoints to call service
- Basic workbench management page for users to see what they started or start new ones
- Is this necessary or can workbench UI work? for short term?
- Request to create new dataset from some Workbench directory?
- Connect to running container
- EST: 1 week for basic UI to call API actions
NDS LABS
- Move API from Go to Python Flask for easier development
- Implement new API endpoints needed by Clowder (not sure if some of these exist)
- get /containers/:id - return list of containers the user has running on WB
- ...
- EST: 2 days (see above)
- Implement sidecar shovel container
- Mounts the same data volumes as the container(s?) it is associated with
- Has a heartbeat service to ping RabbitMQ queue (or potentially Clowder)
- if send dataset action, either download from Clowder and copy into mounted volume, or copy/symlink from locally mounted disk to analysis folder
- can start/stop containers
- ...
- EST: 1-2 weeks
API Endpoints
Workbench (Go)
| endpoint | description |
|---|---|
authenticate check_token refresh_token | get token with 200 |
| services | get array of service definitions add a service |
| accounts | get list of accounts add an account |
| stacks | get list of stacks add a stack |
| logs/stackid | get log for stack |
start/stackid stop/stackid | start and stop existing stack |
| configs | service configuration options |
Clowder
| endpoint | description |
|---|---|
| api/workbench/transfer | queue an action for a remote container (transfer data) |
api/workbench/start api/workbench/stop |