Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Preconditions: none
  • Test steps:
    • ndslabsctl

    • ndslabsctl -h

    • ndslabsctl --help

  • Expected results
    • Any of these commands should display the help/usage information

...

Account administration

Add

...

account (admin only)

    • Test steps
      • ndslabsctl add project account <name> <password>
      • ndslabsctl add project account -f <project<account.json>
      • ndslabsctl add project account --file <project<account.json>
    • Expected results
      • Prompted for admin password
      • Project Account is added. User can login with specified name and password
    • Alternate flows
      • Project Account exists with same name (409 conflict)
      • Incorrect admin password (401 unauthorized)
    • Sample Data:
      • Code Block
        languagejs
        {
            "id": "demo",
            "name": "testdemo account",
             "description": "testacctdemo account description",
             "namespace": "testdemo",
             "password": "12345612345",
             	"storageQuota": "10
        }

Delete project (admin only)

    • Test steps
      • ndslabsctl delete project <project>
    • Expected results
      • Prompted for admin password
      • Project is deleted
    • Alternate flows:
      • Project does not exist (404 not found) 
      • Incorrect admin password (401 unauthorized)

...

      • ",
        	"resourceLimits": {
        		"cpuMax": "2",
        		"cpuDefault": "1",
        		"memMax": "8Gi",
        		"memDefault": "100Mi",
        		"storageQuota": "10"
        	}
        }

Delete account (admin only)

    • Test steps
      • ndslabsctl account project <account>
    • Expected results
      • Prompted for admin password
      • Account is deleted
    • Alternate flows:
      • Account does not exist (404 not found) 
      • Incorrect admin password (401 unauthorized)

Get account details (admin or current project only)

    • Preconditions
      • Admin is logged in or current project account user is logged in
    • Test steps
      • ndslabsctl get project <project>account <account>
    • Expected results
      • Displays project details account details (as json)
      • Password is redacted
    • Alternate flows:
      • Project Account does not exist (404)
      • Project is Account is not current project (401 unauthorized)

List

...

accounts

  • Test steps
    • ndslabsctl list projectsaccounts
  • Expected results
    • Prompted for admin password
    • Projects Accounts are listed
  • Alternate flows:
    • Incorrect admin password (401 unauthorized)

Authentication/authorization

Login

  • Preconditions:

    • Project Account has been created

  • Test steps:
    • ndslabsctl login <user>
    • Prompted for password
  • Expected results:
    • Login succeeded
  • Alternative flows
    • Invalid username / password (401 unauthorized)
  • Notes:
    • Login information is stored in ~/.apictl/.passwd. This files contains the currently logged in user and an authentication token. Removing this file is the equivalent of logging out
  • Possible Issues:
    • It is possible to execute login while already logged in.
      • This may be intentional to allow you to easily switch namespaces.
      • Won't fix, for now

...

    • Preconditions
      • User is logged in
    • Test steps
      • ndslabsctl get service <key>
    • Expected results
      • Displays service spec (as json).  Defaults to user catalog
    • Alternate flows:
      • Service does not exist (404)

List services 

  • Preconditions
    • User is logged in
  • Test steps
    • ndslabsctl list services
    • ndslabsctl list services -c system
    • ndslabsctl list services -c user
    • ndslabsctl list services -c all
  • Expected results
    • Services are listed for the specified catalog (Default: user)
  • Alternate flows:
    • User not logged in (401 unauthorized)

Add service

...

    • Test steps
      • ndslabsctl add service -f <service.json> ndslabsctl add service --file <service.json>-c <catalog>
    • Expected results
      • Prompted for admin password
      • Service is added to the specified catalog (default: user). Only admin user can add to system catalog
      • "ndslabsctl list services" now shows added service
    • Alternate flows
      • Service exists with same key (409 conflict)
        • Problem: value is replaced in etcd instead of error thrown - this may be intentional
          • Should not replace service if being used in a stack
          • Prompt for confirmation on replace?
      • Incorrect admin password (401 unauthorized)
    • Sample Data:
      • Code Block
        languagejs
        {
            "key": "whalesay",
            "label": "Docker Whalesay Example",
            "image": "ndslabs/cowsay-php",
            "description": "An example of a custom spec loaded into NDSLabs",
            "isStack": true,
            "isService": true,
            "isPublic": true,
            "isStandalone": true,
            "ports": [
                { "port": 80, "protocol": "http" }
            ]
        }

Delete

...

service 

    • Test steps
      • ndslabsctl delete service <key> -c <catalog>
    • Expected results
      • Prompted for admin password
      • Service is deleted from the specified catalog
      • Default catalog is user
      • Admin user required to delete from system catalog
      • "ndslabsctl list services" no longer shows deleted service
    • Alternate flows:
      • Incorrect admin password (401 unauthorized)
      • Service is in use by projects (409)
      • Service required by other services (409)
      • No such service (404)

...

    • Preconditions
      • Logged in
      • Stack exists
      • Stack started
      • Config exists
    • Test case:
      • ndslabsctl logs <sid> 
      • ndslabsctl logs <sid> --lines=<lines>
    • Expected results:
      • Logs are displayed
      • Last n log lines are displayed
    • Alternative flow
      • Not logged in (401)
      • Stack doesn't exist (404)
        • Problem: currently throws 500 error
      • Stack is not running (404)
        • Problem: No output given, appears to be no-op?

Console access

    • Preconditions
      • Logged in
      • Stack exists
      • Stack started
    • Test case
      • ndslabsctl console <ssid>
    • Alternative flow
      • Not logged in (401)
      • Stack service doesn't exist (404)
      • Stack not running (404)

API Server Startup

Kubernetes token authentication

    • Preconditions
      • Kubernetes started with token authentication enabled for kube-apiserver
      • Tokens are 
    • Test case
      • Configure apiserver.conf Kubernetes section
      • Start NDS Labs (e.g., ndslabs-up)
      • Tail the apiserver log and look for successful startup
    • Expected results
      • Successful server startup (i.e., you do not see the error "Kubernertes API not availble"
    • Alternative flow
      • Invalid token path
      • Token authentication not supported by kube-apiserver

 

Resource management

Create project with resource limits

...

      • Stack is not running (404)
        • Problem: No output given, appears to be no-op?

Console access

    • Preconditions
      • Logged in
      • Stack exists
      • Stack started
    • Test case
      • ndslabsctl console <ssid>
    • Alternative flow
      • Not logged in (401)
      • Stack service doesn't exist (404)
      • Stack not running (404)

API Server Startup

Kubernetes token authentication

    • Preconditions
      • Kubernetes started with token authentication enabled for kube-apiserver
      • Tokens are 
    • Test case
      • Configure apiserver.conf Kubernetes section
      • Start NDS Labs (e.g., ndslabs-up)
      • Tail the apiserver log and look for successful startup
    • Expected results
      • Successful server startup (i.e., you do not see the error "Kubernertes API not availble"
    • Alternative flow
      • Invalid token path
      • Token authentication not supported by kube-apiserver

 

Resource management

Add account with resource limits

    • Test steps
      • ndslabsctl add account -f <account.json>
    • Expected results
      • Prompted for admin password
      • Account is added
      • kubectl describe quota --namespace=demo returns specified quotas
      • kubectl describe limitranges --namespace=demo returns specified limit ranges
    • Sample Data:
    Code Block
    { 
        "id": "demo",
        "name": "demo project",
        "description": "demo project description",
        "namespace": "demo",
        "password": "12345",
    	"storageQuota": 10,
    	"resourceLimits": {
    		"cpuMax": "2",
    		"cpuDefault": "1",
    		"memMax": "8Gi",
    		"memDefault": "100Mi"
    	}
    }
    • 5/19: OK (NDS-205)

     

Start stack with resource limits

    • Prerequisites:
      • Test services have been added
      • User is logged in
    • Test steps
      • Add "Dataverse" service
    • Expected results
      • Services start as expected
      • For each pod, "kubectl get pod <pod> --namespace=<ns> -o wide" shows a "resources" section, such as:

        Code Block
                        "resources": {
                            "limits": {
                                "cpu": "100m",
                                "memory": "500Mi"
                            },
                            "requests": {
                                "cpu": "100m",
                                "memory": "500Mi"
                            }
                        }
      • Resources and limits match those specified on the service spec
    • 5/19: OK (NDS-205)

Start stack: Too big

    • Prerequisites:
      • Test services have been added
      • User is logged in
    • Test steps
      • Add "Test - Too Big" service
    • Expected results
      • Stack service is in error state
    • 5/19: OK (NDS-205)


Start stack: Memory Hog

    • Prerequisites:
      • Test services have been added
      • User is logged in
    • Test steps
      • Add "Test - Memory Hog" service
    • Expected results
      • Stack service starts, but quickly enters error state
      • Log shows 
        • Reason="OOMKilled"
    • 5/19: OK (NDS-205)

Start stack: No such image

    • Prerequisites:
      • Test services have been added
      • User is logged in
    • Test steps
      • Add "Test - No Image" service
    • Expected results
      • Stack service starts, but quickly enters error state
      • Log shows 
        • Reason=Failed, Message=Failed to pull image "xyzzy": Error: image library/xyzzy not found

    • 5/19: OK (NDS-205)

Start stack: Bad dependency

    • Prerequisites:
      • Test services have been added
      • User is logged in
    • Test steps
      • Add "Test - Bad dependency" service
    • Expected results
      • "No image" service is in error state, "Bad dependency" service is in "stopped" state
      • Log shows 
        • Reason=Failed, Message=Failed to pull image "xyzzy": Error: image library/xyzzy not found

    • 5/19: OK (NDS-205)

Ingress/LoadBalancer support

Single-node, NodePort, no TLS

This first test case demonstrates how to configure a single-node instance using NodePort without TLS;

    • Prerequisites
      • Kubernetes is running
      • Assumes single-node installation
    • Test steps
      • Modify apiserver.yaml
        • CORS_ORIGIN_ADDR = http://<public ip>:30000
        • INGRESS = NodePort
        • PREFX=/api
      • Modify gui.yaml
        • APISERVER_HOST = <public ip>
        • APISERVER_PORT = 30001
        • UI_BASE_PATH = /ui
      • kubectl create -f gui.yaml
      • kubectl create -f apiserver.yaml
      • kubectl label nodes 127.0.0.1 ndslabs-role=compute
      • Open browser to: http://<public ip>:30000/ui/

Single-node, LoadBalancer + TLS

    • Prerequisites
      • Kubernetes is running
      • Assumes single-node installation
      • Wildcard CNAME entry for *.test.ndslabs.org pointing to single node IP
    • Test steps
      • Generate wildcard TLS certificate and key
Code Block
openssl genrsa 2048 > certs/ndslabs.key
openssl req -new -x509 -nodes -sha1 -days 3650 -key certs/ndslabs.key > certs/ndslabs.cert
#[enter *.test.ndslabs.org for the Common Name]
cat certs/ndslabs.cert certs/ndslabs.key > certs/ndslabs.pem
chmod 400 certs/ndslabs.key certs/ndslabs.pem
        • git clone https://github.com/craig-willis/ndslabs-startup
        • cd ndslabs-startup
        • ./ndslabs-up.sh
          • Domain name: test.ndslabs.org
          • Internal IP: accept default
        • Open browser to:
        • A few kubectl checks
          • kubectl get secrets --namespace=default 
            • ndslabs-tls-secret
          • kubectl get ingress --namespace=default
            • ndslabs-ingress with rules for /api and /ui
          • kubectl get secret --namespace=demo 
            • demo-tls-secret
          • kubectl get ingress --namespace=demo
            • stack-clowder-ingress with rule for clowder "/"
        • Stop the stack
          • Confirm ingress deleted
            • kubectl get ingress --namespace=demo
        • Delete project
          • ndslabsctl delete project demo
          • Confirm secret deleted
            • kubectl get secret --namespace=demo 

Multi-node, LoadBalancer + TLS

    • Prerequisites
      • Kubernetes is running 
      • Assumes multi-node installation
      • Wildcard CNAME entry for *.test.ndslabs.org pointing to load balancer node
      • Assumes loadbalancer and default-backend services already deployed
    • Test steps
      • Generate wildcard TLS certificate and key
Code Block
openssl genrsa 2048 > certs/ndslabs.key
openssl req -new -x509 -nodes -sha1 -days 3650 -key certs/ndslabs.key > certs/ndslabs.cert
#[enter *.test.ndslabs.org for the Common Name]
cat certs/ndslabs.cert certs/ndslabs.key > certs/ndslabs.pem
chmod 400 certs/ndslabs.key certs/ndslabs.pem
      • git clone https://github.com/craig-willis/ndslabs-startup
      • cd ndslabs-startup
      • ./ndslabs-up-multinode.sh
      • Open browser to:
      • A few kubectl checks
        • kubectl get secrets --namespace=default 
          • ndslabs-tls-secret
        • kubectl get ingress --namespace=default
          • ndslabs-ingress with rules for /api and /ui
        • kubectl get secret --namespace=demo 
          • demo-tls-secret
        • kubectl get ingress --namespace=demo
          • stack-clowder-ingress with rule for clowder "/"
      • Stop the stack
        • Confirm ingress deleted
          • kubectl get ingress --namespace=demo
      • Delete project
        • ndslabsctl delete project demo
        • Confirm secret deleted
          • kubectl get secret --namespace=demo