As part of CDDR project, we are revisiting the old "tool launcher" concept for new version of workbench.
It is expected that there is a 1:1 Clowder/WB pairing and don't currently plan to support multiple workbenches for a single Clowder. Current proposed architecture involves development on both workbench and Clowder side:
CLOWDER
- Implement API actions that can be queued for workbench
- Send dataset 123 to workbench
- Start container foo
- Stop container bar
- ...
- EST: 4 days for implementing API queue service
- 2 days for api endpoints to call service
- Basic workbench management page for users to see what they started or start new ones
- Is this necessary or can workbench UI work? for short term?
- Request to create new dataset from some Workbench directory?
- Connect to running container
- EST: 1 week for basic UI to call API actions
NDS LABS
- Move API from Go to Python Flask for easier development
- Implement new API endpoints needed by Clowder (not sure if some of these exist)
- get /containers/:id - return list of containers the user has running on WB
- ...
- EST: 2 days (see above)
- Implement sidecar shovel container
- Mounts the same data volumes as the container(s?) it is associated with
- Has a heartbeat service to ping RabbitMQ queue (or potentially Clowder)
- if send dataset action, either download from Clowder and copy into mounted volume, or copy/symlink from locally mounted disk to analysis folder
- can start/stop containers
- ...
- EST: 1-2 weeks