Background
The NDS Labs "Workbench" service (under development) provides NDSC stakeholders with the ability to quickly launch and explore a variety of data management tools. Users select from a list of available services to configure and deploy instances of them. Workbench "services" are composed of a set of integrated Docker containers all deployed on a Kubernetes cluster.
In the following screenshot the user has starts a single instance of Dataverse which includes containers running Glassfish, PostgreSQL, Solr, Rserve, and TwoRavens (Apache + R). (The user is attached to a Kubernetes namespace and can start instances of multiple different services):
Currently, remote access to running services is implemented using Kubernetes "NodePort" mechanism. In essence, a given service (e.g., webserver) is mapped to a cluster-wide port in some configured range (default 30000-32767). Remote users access the running service on the specified port. In the above screenshot, the Dataverse web interface is accessible via http://141.142.210.130:30233. This solution has worked for development purposes but is 1) not scalable and 2) difficult to secure. We are exploring options to provide a scalable and secure solution to providing access to running services in the NDS Labs workbench for 10's-100's of users working with multiple instances of services, i.e. hundreds of service endpoints.
Requirements
- Ability for the user to securely access NDS Labs workbench services, which include web-based and TCP services.
- Services will be hosted on a cluster at "labs.nationaldataservice.org"
Options
Dynamic DNS-based solution
- Services would be accessed via DNS/name. For example: https://dataverse.demo.labs.nds.org"
- This would require:
- Wildcard DNS entry
- NDS-local DNS service with ability to update dynamically based on deployed services
- Wildcard SSL certificate
- Pro:
- Supports both HTTP and TCP services
- Con:
- Complex setup, requires wildcard DNS support and hosting a local DNS server
Path-based solution
- Services would be access via URL path. For example: https://labs.nds.org/demo/dataverse
- This would require
- Single DNS entry for labs.nds.org
- Single SSL certificate
- Pro:
- Simple
- Cons:
- Only supports web-based services that recognized paths. For example, cannot support remote access to iRODS server via iCommands on port 1247.
- Requires that every deployed service handle a path-based deployment, which could require custom configuration for each service.
NodePort
- Services would be accessible via the current NodePort strategy. For example: http://labs.nds.org:30233.
- Pro:
- Already supported
- Cons:
- Difficult to support per-service SSL
- Number of services scales up only to # of available ports.