Page History

Prototype status:

• Working ngingx nginx LB with kubernetes ingress controller integration
• LB runs under kubernetes as a system-service
• Instructions/test harnesses in
• The LB is unopinionated - it works at the system level with any K8s service, as long as the service conforms to standard K8s network model.   The requirements below are specific to NDSLabs test-drive/workbench but the LB is general-purpose and supportive of test-drive/workbench if test-drive/workbench are standard K8s services - assumed to be true.
• Tested with Vhost and path routing - basic testing not thorough
• Ingress interface based on K8s 1.2.0-alpha release - needs update
• Vhost/path routing verified

...

• When a new project is created, if the admin anticipates needing remote access to non-HTTP services, a static IP address and CNAME are assigned to the project.
• The load balancer routes requests to services configured in Kubernetes.  This means that the LB must be Namespace and service aware – which means monitoring Etcd or the Kubernetes API for changes.
• When a new HTTP service is added, load balancer config is updated to proxy via path
  • If no CNAME
    • paths are in the form: labs.nds.org/namespace/serviceId
  • If CNAME
    • paths are in the form namespace.labs.nds.org/serviceId
• When a new TCP service is added, load balancer config is updated to proxy via port – only if project has CNAME/IP:
  • namespace.labs.nds.org:port
• For GUI and API, paths are labs.nds.org/ labs.nds.org/api respectively
• Load balancer must be resilient – if restarted, previous configuration is maintained.  Possibly in failover configuration.

Preliminary Design

Based on the prototype, we will move forward with the Kubernetes ingress-based nginx load balancer model. The current version from the Kubernetes contrib repo works based on preliminary tests.

• Load balancer node: A VM node will serve as the dedicated load-balancer node and run the Nginx LB replication controller using node labels
• Nginx ingress controller: The nginx ingress controller is deployed as a replication controller
• DNS:
  • "A" record points to load balancer node (e.g., test.ndslabs.org A 141.142.210.172)
  • Per-project wildcard CNAME (e.g., "*.demo.ndslabs.org. CNAME test.ndslabs.org)
• Per-service Ingress resource:  
  • For each exposed service endpoint, an ingress rule will be created 
    • host: <service>.<namespace>.ndslabs.org
    • path: "/"
    • backend:
      • serviceName: <service name>
      • servicePort: <service port>
  • These resources will be created/updated/deleted with the associated service
  • The <service> value in the host will be the stack service ID (e.g., srz4wj-clowder)
• GUI/CLI: Instead of NodePort URLs, change to use the LB URL
• TLS:  Add TLS termination support