Thoughts on generalizing workbench as Project X based on recent discussions.
Use Cases
Education and training
One of the clearest proven uses of the platform is for education and training purposes. Labs Workbench was used for:
- IASSIST 2016 for a workshop on integrating Dataverse and iRODS
- NDSC6 for a workshop for the development of Docker containers
- Phenome 2017 for a workshop on how to use the TERRA-REF reference data platform and tools
- Planned iSchool pilot for data curation educators
- Possible deployment by Big Data Hub on commercial provider, such as Microsoft Azure, AWS, GCE
Each environment is unique, but there are a few basic requirements:
- Custom catalog only of the tools needed for the training environment
- User accounts that can be created without requiring registration (e.g., batch import)
- Authentication that ties to existing systems (e.g., Shibboleth, Oauth)
- Short term scalable resources (e.g. 40 users, 4 hours) as well as longer-term stable resources (11 weeks, 24x7, maintenance allowed)
- Custom documentation and branding/skinning
- Custom data, API keys, etc accessible by users
- Configurable quotas (not one-size fits all)
- Ability to deploy a dedicated environment, scale it up, and tear it down. At the end of the workshop/semester, access can be revoked.
- Ability to backup/download data
- Ability to deploy system under a variety of architectures
- Ability to host/manage system at NDSC/SDSC/TACC
- Security/TLS/vulnerability assessment
Scalable analysis environment
We can also envision the platform working as a replacement for the TERRA-REF toolserver or as a DataDNS analysis service. In this case, the requirements are:
- Custom catalog of tools supported for the environment.
- User accounts that can be created without requiring registration (API)
- Authentication that ties to existing systems (e.g., Shibboleth, Oauth)
- Long-term stable and scalable resources. Ability to add/remove nodes as needed.
- Ability to terminate long-running containers to reclaim resources
- Custom documentation and branding, although the UI itself may be optional
- Ability to mount data stored on remote systems (e.g., ROGER) as read-only and possibly read-write scratch space
- Ability to add data to a running container, retrieved from a remote system?
- Clear REST API to
- List tools; list environments for a user; launch tools; stop tools;
- Security/TLS/vulnerability assessment
Platform for the development and deployment of research data portals
Another use case, really a re-purposing of the platform, is to support the development and deployment of research data portals – aka, the Zuhone case. Requirements include:
- Ability to develop data portal using common tools.
- Ability to deploy data portal services "near" datasets (e.g., ythub).
General requirements
- Backup
- Monitoring
Current features/components
- CoreOS
- Docker
- Kubernetes
- Gluster
- Deploy tools/NDS Labs Startup
- API Server
- Web UI
- Ingress controller
- Backup
- Monitoring/NRPE
- Specs
- Development environments.