You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 35 Next »

Overview

This page defines test cases for the NDSLabs command line utility (ndslabsctl). 

See also

Starting the API server

  • eval $(docker run --rm -t ndslabs/system-shell usage docker)
  • kube-up.sh
  • If needed, download the latest copy of ndslabsctl
  • If testing a new version of API server
    • vi /usr/local/lib/ndslabs/apiserver
    • Change latest -> test
    • docker rmi ndslabs/apiserver:test
  • ndslabs-up.sh
  • "ndslabsctl list projects" to confirm things are working

Global preconditions

  • Kubernetes is running
  • API Server is running either as a Kubernetes service or as a standalone Docker container 
  • ndslabsctl has been installed and is in your path

Help

  • Preconditions: none
  • Test steps:

    • ndslabsctl

    • ndslabsctl -h

    • ndslabsctl --help

  • Expected results
    • Any of these commands should display the help/usage information

Project administration

Add project (admin only)

    • Test steps
      • ndslabsctl add project <name> <password>
      • ndslabsctl add project -f <project.json>
      • ndslabsctl add project --file <project.json>
    • Expected results
      • Prompted for admin password
      • Project is added. User can login with specified name and password
    • Alternate flows
      • Project exists with same name (409 conflict)
      • Incorrect admin password (401 unauthorized)
    • Sample Data:
      • {
     "name": "test",
     "description": "testacct",
     "namespace": "test",
     "password": "123456",
     "storageQuota": 10
}

Delete project (admin only)

    • Test steps
      • ndslabsctl delete project <project>
    • Expected results
      • Prompted for admin password
      • Project is deleted
    • Alternate flows:
      • Project does not exist (404 not found) 

      • Incorrect admin password (401 unauthorized)

Get project details (admin or current project only)

    • Preconditions
      • Admin is logged in or current project user is logged in
    • Test steps
      • ndslabsctl get project <project>
    • Expected results
      • Displays project details (as json)
      • Password is redacted
    • Alternate flows:
      • Project does not exist (404)
      • Project is not current project (401 unauthorized)

List projects

  • Test steps
    • ndslabsctl list projects
  • Expected results
    • Prompted for admin password
    • Projects are listed
  • Alternate flows:
    • Incorrect admin password (401 unauthorized)

Authentication/authorization

Login

  • Preconditions:

    • Project has been created

  • Test steps:
    • ndslabsctl login <user>
    • Prompted for password
  • Expected results:
    • Login succeeded
  • Alternative flows
    • Invalid username / password (401 unauthorized)
  • Notes:
    • Login information is stored in ~/.apictl/.passwd. This files contains the currently logged in user and an authentication token. Removing this file is the equivalent of logging out
  • Possible Issues:
    • It is possible to execute login while already logged in.
      • This may be intentional to allow you to easily switch namespaces.
      • Won't fix, for now

Logout

  • Preconditions
    • User is logged in
  • Test steps
    • ndslabsctl logout
    • ndslabsctl list services
  • Expected results:
    • List command results in "unauthorized"

Change password

  • Preconditions:
    • User is logged in
  • Test steps
    • ndslabsctl login <user>
    • ndslabsctl passwd
      • Enter current password
      • Enter new password
      • Confirm new password
  • Alternate flows:
    • User not logged in (401 unauthorized)
    • Invalid password
    • Password mismatch yield error message: "Passwords do not match"
  • Expected results
    • Password is changed

Service administration

Get service details

    • Preconditions
      • User is logged in
    • Test steps
      • ndslabsctl get service <key>
    • Expected results
      • Displays service spec (as json)
    • Alternate flows:
      • Service does not exist (404)

List services 

  • Preconditions
    • User is logged in
  • Test steps
    • ndslabsctl list services
  • Expected results
    • Services are listed
  • Alternate flows:
    • User not logged in (401 unauthorized)

Add service (admin only)

    • Test steps
      • ndslabsctl add service -f <service.json>
      • ndslabsctl add service --file <service.json>
    • Expected results
      • Prompted for admin password
      • Service is added
      • "ndslabsctl list services" now shows added service
    • Alternate flows
      • Service exists with same key (409 conflict)
        • Problem: value is replaced in etcd instead of error thrown - this may be intentional
          • Should not replace service if being used in a stack
          • Prompt for confirmation on replace?
      • Incorrect admin password (401 unauthorized)
    • Sample Data:
      • {
    "key": "whalesay",
    "label": "Docker Whalesay Example",
    "image": "ndslabs/cowsay-php",
    "description": "An example of a custom spec loaded into NDSLabs",
    "isStack": true,
    "isService": true,
    "isPublic": true,
    "isStandalone": true,
    "ports": [
        { "port": 80, "protocol": "http" }
    ]
}

Delete service (admin only)

    • Test steps
      • ndslabsctl delete service <key>
    • Expected results
      • Prompted for admin password
      • Service is deleted
      • "ndslabsctl list services" no longer shows deleted service
    • Alternate flows:
      • Incorrect admin password (401 unauthorized)
      • Service is in use by projects (409)
      • Service required by other services (409)
      • No such service (404)

Volume administration

Create volume

    • Preconditions
      • Logged in
      • Sufficient quota remaining (not implemented)
      • Service exists with no existing attachments
      • Service not running
    • Test steps
      • ndslabsctl add volume <vol name> <vol size GB>
        • Creates a detached volume
      • ndslabsctl add volume <vol name> <vol size GB> <sid>
        • Creates and attaches volume
    • Expected results
      • Volume is created
      • Volume is created and attached
    • Alternate flows
      • Not logged in (401)
      • Insufficient quota (409) (not implemented)
      • Duplicate name (409)
        • NDS-181 now allows duplicate volume names
      • Stack service already has a volume attached (409)
      • Stack service doesn't exist (404)

Attach volume

    • Preconditions:
      • Logged in 
      • Volume exists
      • Service exists with no existing attachments
      • Service not running
    • Test steps
      • ndslabsctl attach <vol name> <sid>
    • Expected results
      • Volume is attached to specified service
    • Alternate flows
      • Not logged in (401)
      • Volume doesn't exist (404)
      • Service doesn't exist (404)
      • Service is running (409)
      • Volume is attached (409)
        • Problem: Reattaches volume to new service without throwing error
    • Issues:
      • It is possible to attach a single volume to any kind of service. This may be intentional.

List volumes

    • Preconditions
      • Logged in
      • Volumes exist
    • Test steps
      • ndslabsctl list volumes
    • Expected results
      • Volume details are displayed
    • Alternate flows
      • Not logged in (401)
      • No volumes exist (200)

Detach volume

    • Preconditions
      • Logged in
      • Volume exists and is attached
      • Stack not running
    • Test steps
      • ndslabsctl detach <vol name>
    • Expected results
      • Volume is detached
    • Alternative flows
      • Not logged in (401)
      • Volume doesn't exist (404)
      • Stack is running (409)
        • Problem: Was able to detach from running stack without error
        • Resolved
      • Volume isn't attached (409)
        • Problem: Was able to detach multiple times without error
        • Resolved

Delete volume

    • Preconditions
      • Logged in
      • Volume exists
      • Stack not running
    • Test case:
      • ndslabsctl delete <vol name>
    • Expected results:
      • Volume is deleted
    • Alternative flows
      • Not logged in (401)
      • Volume doesn't exist (404)
      • Stack is running (409)

Stack administration

Add stack

    • Preconditions
      • Logged in
      • Service exists
    • Test steps
      • ndslabsctl add stack <key> <name> 
      • ndslabsctl add stack <key> <name> --opt=<comma-separated list of optional services>
    • Expected results
      • Stack is added
    • Alternate flows
      • Not logged in (401)
      • Service doesn't exist (404)
      • Duplicate stack name (409)
        • NDS-181 now allows duplicate stack names

List stacks

List stacks

    • Preconditions:
      • Logged in
      • Stack has been added
    • Test steps
      • ndslabsctl list stacks
    • Expected results
      • Summary of stacks is displayed (name, services, status, SIDs)
    • Alternate flow:
      • Not logged in (401)
      • No stacks (empty list)

Delete stack

    • Preconditions:
      • Logged in
      • Stack exists
      • Stack not running
    • Test steps
      • ndslabsctl delete stack <sid>
      • Stack is deleted (201)
    • Alternate flows
      • Not logged in (401)
      • Stack doesn't exist (404)
      • Stack is running (409)
      • Attached volumes are orphaned

Get stack details

    • Preconditions:
      • Logged in
      • Stack exists
    • Test steps
      • ndslabsctl get stack <sid>
    • Expected results
      • Stack details, including endpoints and config, are displayed
    • Alternate flows
      • Not logged in (401)
      • Stack doesn't exist (404)

Start stack

    • Preconditions
      • Logged in
      • Stack exists and started
    • Test steps
      • ndslabsctl start <sid>
    • Expected results
      • All stack services (service, replication controller, pods) are started and ready
    • Alternate flows
      • Not logged in (401)
      • Stack doesn't exist (404)
      • Stack not started or stopping (409)

Stop stack

    • Preconditions
      • Stack started
    • Test steps
      • ndslabsctl stop <sid>
    • Expected results
      • Stack service, replication controller, pods are stopped (200)
    • Alternative flow
      • Not logged in (401)
      • Stack does not exist (404)
      • Stack not started (200)

Configuration

List configuration options for spec

    • Preconditions
      • Logged in
      • Service exists
    • Test case:
      • ndslabsctl list configs <service key(s)>
    • Expected results:
      • Possible service configuration options are displayed
    • Alternative flow
      • Not logged in (401)
      • Service does not exist (404)
        • Problem: currently throws 500 error
        • Resolved – if one fails, all fail
    • Issues:
      • Help text references "service ids", should be "service keys" for consistency
      • Resolved

Get stack service configuration

    • Preconditions
      • Logged in
      • Stack exists
    • Test case:
      • ndslabsctl get stack <stack id>
    • Expected results:
      • Current service configuration options and settings are displayed
    • Alternative flow
      • Not logged in (401)
      • Stack does not exist (404)
        • Problem: currently throws 500 error
        • Works for me

Set stack service configuration option

    • Preconditions
      • Logged in
      • Stack exists
      • Config exists
    • Test case:
      • ndslabsctl set <stack service id> <name> <value>
    • Expected results:
      • Specified configuration setting os overriden for the service
    • Alternative flow
      • Not logged in (401)
      • Stack doesn't exist (404)
        • Problem: currently throws 500 error
        • Resolved

      • Config doesn't exist or is not overridable (409)

        • Problem: No output given, appears to be no-op?
        • Resolved

Other

View stack service logs

    • Preconditions
      • Logged in
      • Stack exists
      • Stack started
      • Config exists
    • Test case:
      • ndslabsctl logs <sid> 
      • ndslabsctl logs <sid> --lines=<lines>
    • Expected results:
      • Logs are displayed
      • Last n log lines are displayed
    • Alternative flow
      • Not logged in (401)
      • Stack doesn't exist (404)
        • Problem: currently throws 500 error
      • Stack is not running (404)
        • Problem: No output given, appears to be no-op?

Console access

    • Preconditions
      • Logged in
      • Stack exists
      • Stack started
    • Test case
      • ndslabsctl console <ssid>
    • Alternative flow
      • Not logged in (401)
      • Stack service doesn't exist (404)
      • Stack not running (404)

API Server Startup

Kubernetes token authentication

    • Preconditions
      • Kubernetes started with token authentication enabled for kube-apiserver
      • Tokens are 
    • Test case
      • Configure apiserver.conf Kubernetes section
      • Start NDS Labs (e.g., ndslabs-up)
      • Tail the apiserver log and look for successful startup
    • Expected results
      • Successful server startup (i.e., you do not see the error "Kubernertes API not availble"
    • Alternative flow
      • Invalid token path
      • Token authentication not supported by kube-apiserver

 

Resource management

Add project with resource limits

    • Test steps
      • ndslabsctl add project -f <project.json>
    • Expected results
      • Prompted for admin password
      • Project is added
      • kubectl describe quota --namespace=demo returns specified quotas
      • kubectl describe limitranges --namespace=demo returns specified limit ranges
    • Sample Data:
    { 
    "id": "demo",
    "name": "demo project",
    "description": "demo project description",
    "namespace": "demo",
    "password": "12345",
	"storageQuota": 10,
	"resourceLimits": {
		"cpuMax": "2",
		"cpuDefault": "1",
		"memMax": "8Gi",
		"memDefault": "100Mi"
	}
}
    • 5/19: OK (NDS-205)

     

Start stack with resource limits

    • Prerequisites:
      • Test services have been added
      • User is logged in
    • Test steps
      • Add "Dataverse" service
    • Expected results
      • Services start as expected

      • For each pod, "kubectl get pod <pod> --namespace=<ns> -o wide" shows a "resources" section, such as:

                        "resources": {
                    "limits": {
                        "cpu": "100m",
                        "memory": "500Mi"
                    },
                    "requests": {
                        "cpu": "100m",
                        "memory": "500Mi"
                    }
                }
      • Resources and limits match those specified on the service spec
    • 5/19: OK (NDS-205)

Start stack: Too big

    • Prerequisites:
      • Test services have been added
      • User is logged in
    • Test steps
      • Add "Test - Too Big" service
    • Expected results
      • Stack service is in error state
    • 5/19: OK (NDS-205)


Start stack: Memory Hog

    • Prerequisites:
      • Test services have been added
      • User is logged in
    • Test steps
      • Add "Test - Memory Hog" service
    • Expected results
      • Stack service starts, but quickly enters error state
      • Log shows 
        • Reason="OOMKilled"
    • 5/19: OK (NDS-205)

Start stack: No such image

    • Prerequisites:
      • Test services have been added
      • User is logged in
    • Test steps
      • Add "Test - No Image" service
    • Expected results
      • Stack service starts, but quickly enters error state
      • Log shows 

        • Reason=Failed, Message=Failed to pull image "xyzzy": Error: image library/xyzzy not found

    • 5/19: OK (NDS-205)

Start stack: Bad dependency

    • Prerequisites:
      • Test services have been added
      • User is logged in
    • Test steps
      • Add "Test - Bad dependency" service
    • Expected results
      • "No image" service is in error state, "Bad dependency" service is in "stopped" state
      • Log shows 

        • Reason=Failed, Message=Failed to pull image "xyzzy": Error: image library/xyzzy not found

    • 5/19: OK (NDS-205)

Ingress/LoadBalancer support

Single-node, NodePort, no TLS

This first test case demonstrates how to configure a single-node instance using NodePort without TLS;

    • Prerequisites
      • Kubernetes is running
      • Assumes single-node installation
    • Test steps
      • Modify apiserver.yaml
        • CORS_ORIGIN_ADDR = http://<public ip>:30000
        • INGRESS = NodePort
        • PREFX=/api
      • Modify gui.yaml
        • APISERVER_HOST = <public ip>
        • APISERVER_PORT = 30001
        • UI_BASE_PATH = /ui
      • kubectl create -f gui.yaml
      • kubectl create -f apiserver.yaml
      • kubectl label nodes 127.0.0.1 ndslabs-role=compute
      • Open browser to: http://<public ip>:30000/ui/

Single-node, LoadBalancer + TLS

These steps have all been implemented in https://github.com/craig-willis/ndslabs-startup/ndslabs-up.sh

    • Prerequisites
      • Kubernetes is running
      • Assumes single-node installation
      • Wildcard CNAME entry for *.test.ndslabs.org
    • Test steps
      • Generate wildcard TLS certificate and key
openssl genrsa 2048 > certs/ndslabs.key
openssl req -new -x509 -nodes -sha1 -days 3650 -key certs/ndslabs.key > certs/ndslabs.cert
#[enter *.test.ndslabs.org for the Common Name]
cat certs/ndslabs.cert certs/ndslabs.key > certs/ndslabs.pem
chmod 400 certs/ndslabs.key certs/ndslabs.pem

 

    • Create secret in default namespace (ndslabs-startup/create-tls-secret.sh)

 

kubectl create secret generic ndslabs-tls-secret --from-file=tls.crt=certs/ndslabs.cert --from-file=tls.key=certs/ndslabs.key --namespace=default

 

    • Create loadbalancer, default backend and default ingress rules
      • kubectl create -f ndslabs/loadbalancer.yaml
      • kubectl create -f ndslabs/default-backend.yaml
      • kubectl create -f ndslabs/default-ingress.yaml
    • Modify apiserver.yaml
    • Modify gui.yaml
      • APISERVER_SECURE = true
      • APISERVER_PORT = 443
      • UI_BASE_PATH = /ui
    • Create the NDSLabs apiserver and gui services
      • kubectl create -f gui.yaml
      • kubectl create -f apiserver.yaml
    • Open browser to https://www.test.ndslabs.org/ui/
      • Configure basic Clowder, launch stack 
      • Confirm Clowder endpoint link points to LoadBalanacer URI

 

 

 

  • No labels