Table of Contents |
---|
Overview
This page defines test cases for the NDSLabs command line utility (ndslabsctl).
Table of Contents |
---|
See also
Starting the API server
- eval $(docker run --rm -t ndslabs/system-shell usage docker)
- git clone https://github.com/craig-willis/ndslabs-startup
- cd ndslabs-startup
- ./kube-up.sh
- ./ndslabs-up.sh
...
- If needed, download the latest
...
- copy of
...
- ndslabsctl
...
- wget
...
...
- /ndslabsctl-linux-amd64 -O /usr/local/bin/ndslabsctl
- chmod +x ndslabsctl
- If testing a new version of API server
- vi /usr/local/lib/ndslabs/apiserver
- Change latest -> test
- docker rmi ndslabs/apiserver:test
- ndslabs-up.sh
- "ndslabsctl list projects" to confirm things are working
If the API server is running under docker or Kubernetes, you will need to specify the server address. This can be done on the commandline using the --server flag or by creating a .ndslabsctl.yaml file in your home directory. For example:
APISERVER=`kubectl describe svc ndslabs-apiserver | grep ^IP | awk '{print $2}'`
...
Global preconditions
- Kubernetes is running
- API Server is running either as a Kubernetes service or as a standalone Docker container
- ndslabsctl has been installed and is in your path
...
- Preconditions: none
- Test steps:
ndslabsctl
ndslabsctl -h
ndslabsctl --help
- Expected results
- Any of these commands should display the help/usage information
Authentication/authorization
Login
Preconditions:
Project has been created or admin user
- Test steps:
- ndslabsctl login admin
- ndslabsctl login <user>
- Prompted for password
- Expected results:
- Login succeeded
- Alternative flows
- Invalid username or password results in login error
- Notes:
- Login information is stored in ~/.apictl/.passwd. This files contains the currently logged in user and an authentication token. Removing this file is the equivalent of logging out
Logout
- Preconditions
- User is logged in
- Test steps
- ndslabsctl logout
- ndslabsctl list services
- Expected results:
- List command results in "unauthorized"
Change password
- Preconditions:
- User is logged in
- Test steps
- ndslabsctl login <user>
- ndslabsctl passwd
- Enter current password
- Enter new password
- Confirm new password
- Expected results
- Password is changed
Project administration
...
Account administration
Add account (admin only)
- Preconditions:
- Admin is logged in
- Test steps
- ndslabsctl add
project - Test steps
- account <name> <password>
- ndslabsctl add
project - account -f
<project- <account.json>
- ndslabsctl add
project - account --file
<project- <account.json>
- Expected results
Project - Prompted for admin password
- Account is added. User can login with specified name and password
- Alternate flows
Project - Account exists with same name (409 conflict)
Logged in user is not - Incorrect admin password (401 unauthorized)
- Sample
...
- Data:
Code Block language js { "id": "demo", "name": "
- Data:
...
demo account",
...
"description": "
...
demo account description", "namespace": "
...
demo",
...
"password": "12345", "storageQuota": "10", "resourceLimits": { "cpuMax": "2", "cpuDefault": "1", "memMax": "8Gi", "memDefault": "100Mi", "storageQuota": "10" } }
Delete account
...
(admin only)
- Preconditions
- Admin is logged in
- Test steps
- ndslabsctl
delete - Test steps
- account project
<project>- <account>
- Expected results
Project - Prompted for admin password
- Account is deleted
- Alternate flows:
Project User is not admin - Account does not exist (404 not found)
- Problem: currently results in 500
Incorrect admin password (401 unauthorized)
Get
...
account details (admin or current project only)
- Preconditions
- Admin is logged in or current
project - Preconditions
- account user is logged in
- Test steps
- ndslabsctl get
project <project>- account <account>
- Expected results
- Displays
project details - account details (as json)
- Password is redacted
- Alternate flows:
Project - Account does not exist (404)
User is not admin and project is not - Account is not current project (401 unauthorized)
List
...
- Admin is logged in
accounts
- Test steps
- ndslabsctl list
...
- accounts
- Expected results
...
- Prompted for admin password
- Accounts are listed
...
- Alternate flows:
...
- Incorrect admin password (401 unauthorized)
Authentication/authorization
Login
Preconditions:
Account has been created
- Test steps:
- ndslabsctl login <user>
- Prompted for password
- Expected results:
- Login succeeded
- Alternative flows
- Invalid username / password (401 unauthorized)
- Notes:
- Login information is stored in ~/.apictl/.passwd. This files contains the currently logged in user and an authentication token. Removing this file is the equivalent of logging out
- Possible Issues:
- It is possible to execute login while already logged in.
- This may be intentional to allow you to easily switch namespaces.
- Won't fix, for now
- It is possible to execute login while already logged in.
Logout
- Preconditions
- User is logged in
- Test steps
- ndslabsctl logout
- ndslabsctl list services
- Expected results:
- List command results in "unauthorized"
Change password
- Preconditions:
- User is logged in
- Test steps
- ndslabsctl login <user>
- ndslabsctl passwd
- Enter current password
- Enter new password
- Confirm new password
- Alternate flows:
- User not logged in (401 unauthorized)
- Invalid password
- Password mismatch yield error message: "Passwords do not match"
- Expected results
- Password is changed
Service administration
Get service details
- Preconditions
- User is logged in
- Test steps
- ndslabsctl get service <key>
- Expected results
- Displays service spec (as json). Defaults to user catalog
- Alternate flows:
- Service does not exist (404)
- Service does not exist (404)
- Preconditions
List services
- Preconditions
- User is logged in
- Test steps
- ndslabsctl list services
- ndslabsctl list services -c system
- ndslabsctl list services -c user
- ndslabsctl list services -c all
- Expected results
- Services are listed for the specified catalog (Default: user)
- Alternate flows:
- User not logged in (401 unauthorized)
Add service
...
Stack administration
- Admin user can create / start / stop stacks. We will need to revisit the workflow for admin.
Add stack
- Preconditions
- Logged in
- Service exists
- Test steps
- ndslabsctl add stack <key> <name>
- ndslabsctl add stack <key> <name> --opt=<comma-separated list of optional services>
- Expected results
- Stack is added
- Alternate flow
- Not logged in (401)
- Service doesn't exist (404)
- Duplicate stack name (409)
List stacks
List stacks
- Preconditions:
- Logged in
- Stack has been added
- Test steps
- ndslabsctl list stacks
- Expected results
- Summary of stacks is displayed (name, services, status, SIDs)
- Alternate flow:
- Not logged in (401)
- No stacks (empty list)
Delete stack
- Preconditions:
- Logged in
- Stack exists
- Stack not running
- Test steps
- ndslabsctl delete stack <sid>
- Stack is deleted (201)
- Alternate flow
- Not logged in (401)
- Stack doesn't exist (404)
- Stack is running (409)
- Attached volumes are orphaned
Get stack details
- Preconditions:
- Logged in
- Stack exists
- Test steps
- ndslabsctl get stack <sid>
- Expected results
- Stack details, including endpoints and config, are displayed
- Alternate flow
- Not logged in (401)
- Stack doesn't exist (404)
Start stack
- Preconditions
- Logged in
- Stack exists and started
- Test steps
- ndslabsctl start <sid>
- Expected results
- All stack services (service, replication controller, pods) are started and ready
- Alternate flow
- Not logged in (401)
- Stack doesn't exist (404)
- Stack not started or stopping (409)
Stop stack
- Preconditions
- Stack started
- Test steps
- ndslabsctl stop <sid>
- Expected results
- Stack service, replication controller, pods are stopped (200)
- Alternative flow
- Not logged in (401)
- Stack does not exist (404)
- Stack not started (200)
Service administration
Add service (admin only)
- Preconditions:
- Admin is logged in
- Test steps
- ndslabsctl add service -f <service.json>
ndslabsctl add service --file <service.json>- Test steps
- -c <catalog>
- Expected results
- Prompted for admin password
- Service is added
. User can add service/stack.- to the specified catalog (default: user). Only admin user can add to system catalog
- "ndslabsctl list services" now shows added service
- Alternate flows
- Service exists with same key (409 conflict)
- Service exists with same key (409 conflict)
Logged in user is not admin - Problem: value is replaced in etcd instead of error thrown - this may be intentional
- Should not replace service if being used in a stack
- Prompt for confirmation on replace?
- Problem: value is replaced in etcd instead of error thrown - this may be intentional
- Incorrect admin password (401 unauthorized)
- Notes:
- See ndslabs-specs repos for sample services
Delete service (admin only)
- Sample Data:
Code Block language js { "key": "whalesay", "label": "Docker Whalesay Example", "image": "ndslabs/cowsay-php", "description": "An example of a custom spec loaded into NDSLabs", "isStack": true, "isService": true, "isPublic": true, "isStandalone": true, "ports": [ { "port": 80, "protocol": "http" } ] }
- Sample Data:
Delete service
...
- Preconditions
- Admin is logged in
- Test steps
- ndslabsctl delete service <key> -c <catalog>
- Expected results
- Prompted for admin password
- Service is deleted from the specified catalog
- Default catalog is user
- Admin user required to delete from system catalog
- "ndslabsctl list services" no longer shows deleted service
- Alternate flows:
- Test steps
- Project does not exist (404 not found) User is not admin
- Incorrect admin password (401 unauthorized
)Not logged in (401- )
- Not admin user (401)
- Service is in use by projects (409)
- Service required by other services (409)
- No such service (404)
Get service details
- Preconditions
- User is logged in
- Test steps
- ndslabsctl get service <key>
- Expected results
- Displays service spec (as json)
- Alternate flows:
- Service does not exist (404
List services
...
- User is logged in
...
- Services are listed
Volume administration
...
Create volume
- Preconditions
- Logged in
- Sufficient quota remaining (not implemented)
- Service exists with no existing attachments
- Service not running
- Test steps
- ndslabsctl
create - Preconditions
- add volume <vol name> <vol size GB>
- Creates a detached volume
- ndslabsctl
- add volume <vol name> <vol size GB>
create - add volume <vol name> <vol size GB> <sid>
- Creates and attaches volume
- add volume <vol name> <vol size GB> <sid>
- Expected results
- Volume is created
- Volume is created and attached
- Alternate
flow- flows
- Not logged in (401)
- Insufficient quota (409) (not implemented)
Duplicate name (409)- NDS-181 now allows duplicate volume names
- Stack service already has a volume attached (409)
- Stack service doesn't exist (404)
- flows
Attach volume
- Preconditions:
- Logged in
- Volume
exits- Preconditions:
- exists
- Service exists with no existing attachments
- Service not running
- Test steps
- ndslabsctl attach <vol name> <sid>
- Expected results
- Volume is attached to specified service
- Alternate
flow- flows
- Not logged in (401)
- Volume doesn't exist (404)
- Service doesn't exist (404)
- Service is running (409)
- Volume is attached (409)
- Problem: Reattaches volume to new service without throwing error
- Issues:
- It is possible to attach a single volume to any kind of service. This may be intentional.
- flows
List volumes
- Preconditions
- Logged in
- Volumes exist
- Test steps
- ndslabsctl list volumes
- Expected results
- Volume details are displayed
Alterate flow- Preconditions
- Alternate flows
- Not logged in (401)
- No volumes exist (200)
- Alternate flows
Detach volume
- Preconditions
- Logged in
- Volume exists and is attached
- Stack not running
- Test steps
- ndslabsctl detach <vol name>
- Expected results
- Volume is detached
- Alternative
flow- Preconditions
- flows
- Not logged in (401)
- Volume doesn't exist (404)
- Stack is running (409)
- Problem: Was able to detach from running stack without error
- Resolved
- Volume isn't attached (409)
- Problem: Was able to detach multiple times without error
- Resolved
- flows
Delete volume
- Preconditions
- Logged in
- Volume exists
- Stack not running
- Test case:
- ndslabsctl delete <vol name>
- Expected results:
- Volume is deleted
- Alternative flows
- Not logged in (401)
- Volume doesn't exist (404)
- Stack is running (409)
- Preconditions
Stack administration
Add stack
- Preconditions
- Logged in
- Service exists
- Test steps
- ndslabsctl add stack <key> <name>
- ndslabsctl add stack <key> <name> --opt=<comma-separated list of optional services>
- Expected results
- Stack is added
- Alternate flows
- Not logged in (401)
- Service doesn't exist (404)
Duplicate stack name (409)- NDS-181 now allows duplicate stack names
- Preconditions
List stacks
List stacks
- Preconditions:
- Logged in
- Stack has been added
- Test steps
- ndslabsctl list stacks
- Expected results
- Summary of stacks is displayed (name, services, status, SIDs)
- Alternate flow:
- Not logged in (401)
- No stacks (empty list)
- Preconditions:
Delete stack
- Preconditions:
- Logged in
- Stack exists
- Stack not running
- Test steps
- ndslabsctl delete stack <sid>
- Stack is deleted (201)
- Alternate flows
- Not logged in (401)
Volume - Preconditions:
- Stack doesn't exist (404)
- Stack is running (409)
- Attached volumes are orphaned
Get stack details
- Preconditions:
- Logged in
- Stack exists
- Test steps
- ndslabsctl get stack <sid>
- Expected results
- Stack details, including endpoints and config, are displayed
- Alternate flows
- Not logged in (401)
- Stack doesn't exist (404)
- Preconditions:
Start stack
- Preconditions
- Logged in
- Stack exists and started
- Test steps
- ndslabsctl start <sid>
- Expected results
- All stack services (service, replication controller, pods) are started and ready
- Alternate flows
- Not logged in (401)
- Stack doesn't exist (404)
- Stack not started or stopping (409)
- Preconditions
Stop stack
- Preconditions
- Stack started
- Test steps
- ndslabsctl stop <sid>
- Expected results
- Stack service, replication controller, pods are stopped (200)
- Alternative flow
- Not logged in (401)
- Stack does not exist (404)
- Stack not started (200)
- Preconditions
Configuration
List configuration options for spec
- Preconditions
- Logged in
- Service exists
- Test case:
- ndslabsctl list configs <service key(s)>
- Expected results:
- Possible service configuration options are displayed
- Alternative flow
- Not logged in (401)
- Service does not exist (404)
- Problem: currently throws 500 error
- Resolved – if one fails, all fail
- Issues:
- Help text references "service ids", should be "service keys" for consistency
- Resolved
- Preconditions
Get stack service configuration
- Preconditions
- Logged in
- Stack exists
- Test case:
- ndslabsctl get
configs <sid>- Preconditions
- stack <stack id>
- Expected results:
- Current service configuration options and settings are displayed
- Problem: resulting in 500 now
- Alternative flow
- Not logged in (401)
- Stack
doesn't - Alternative flow
- does not exist (404)
- Problem: currently throws 500 error
- Works for me
- does not exist (404)
Set stack service configuration option
- Preconditions
- Logged in
- Stack exists
- Config exists
- Test case:
- ndslabsctl set
<sid> - Preconditions
- <stack service id> <name> <value>
- Expected results:
- Specified configuration setting os overriden for the service
- Alternative flow
- Not logged in (401)
- Stack doesn't exist (404)
- Problem: currently throws 500 error
- Resolved
Config doesn't exist or is not overridable (409)
- Problem: No output given, appears to be no-op?
- Resolved
Other
View stack service logs
- Preconditions
- Logged in
- Stack exists
- Stack started
- Config exists
- Test case:
- ndslabsctl logs <sid>
- ndslabsctl logs <sid> --lines=<lines>
- Expected results:
- Logs are displayed
- Last n log lines are displayed
- Alternative flow
- Not logged in (401)
- Stack doesn't exist (404)
- Problem: currently throws 500 error
- Stack is not running (404)
- Problem: No output given, appears to be no-op?
- Preconditions
Console access
- Preconditions
- Logged in
- Stack exists
- Stack started
- Test case
- ndslabsctl console <ssid>
- Alternative flow
- Not logged in (401)
- Stack service doesn't exist (404)
- Stack not running (404)
- Preconditions
API Server Startup
Kubernetes token authentication
- Preconditions
- Kubernetes started with token authentication enabled for kube-apiserver
- Tokens are
- Test case
- Configure apiserver.conf Kubernetes section
- TokenPath=/run/secrets/kubernetes.io/serviceaccount/token
- Start NDS Labs (e.g., ndslabs-up)
- Tail the apiserver log and look for successful startup
- Configure apiserver.conf Kubernetes section
- Expected results
- Successful server startup (i.e., you do not see the error "Kubernertes API not availble"
- Alternative flow
- Invalid token path
- Token authentication not supported by kube-apiserver
- Preconditions
Resource management
Add account with resource limits
- Test steps
- ndslabsctl add account -f <account.json>
- Expected results
- Prompted for admin password
- Account is added
- kubectl describe quota --namespace=demo returns specified quotas
- kubectl describe limitranges --namespace=demo returns specified limit ranges
- Sample Data:
Code Block { "id": "demo", "name": "demo project", "description": "demo project description", "namespace": "demo", "password": "12345", "storageQuota": 10, "resourceLimits": { "cpuMax": "2", "cpuDefault": "1", "memMax": "8Gi", "memDefault": "100Mi" } }
- 5/19: OK (NDS-205)
- Test steps
Start stack with resource limits
- Prerequisites:
- Test services have been added
- User is logged in
- Test steps
- Add "Dataverse" service
- Expected results
- Services start as expected
For each pod, "kubectl get pod <pod> --namespace=<ns> -o wide" shows a "resources" section, such as:
Code Block "resources": { "limits": { "cpu": "100m", "memory": "500Mi" }, "requests": { "cpu": "100m", "memory": "500Mi" } }
- Resources and limits match those specified on the service spec
- 5/19: OK (NDS-205)
- Prerequisites:
Start stack: Too big
- Prerequisites:
- Test services have been added
- User is logged in
- Test steps
- Add "Test - Too Big" service
- Expected results
- Stack service is in error state
- 5/19: OK (NDS-205)
- Prerequisites:
Start stack: Memory Hog
- Prerequisites:
- Test services have been added
- User is logged in
- Test steps
- Add "Test - Memory Hog" service
- Expected results
- Stack service starts, but quickly enters error state
- Log shows
- Reason="OOMKilled"
- 5/19: OK (NDS-205)
- Prerequisites:
Start stack: No such image
- Prerequisites:
- Test services have been added
- User is logged in
- Test steps
- Add "Test - No Image" service
- Expected results
- Stack service starts, but quickly enters error state
- Log shows
Reason=Failed, Message=Failed to pull image "xyzzy": Error: image library/xyzzy not found
- 5/19: OK (NDS-205)
- Prerequisites:
Start stack: Bad dependency
- Prerequisites:
- Test services have been added
- User is logged in
- Test steps
- Add "Test - Bad dependency" service
- Expected results
- "No image" service is in error state, "Bad dependency" service is in "stopped" state
- Log shows
Reason=Failed, Message=Failed to pull image "xyzzy": Error: image library/xyzzy not found
- 5/19: OK (NDS-205)
- Prerequisites:
Ingress/LoadBalancer support
Single-node, NodePort, no TLS
This first test case demonstrates how to configure a single-node instance using NodePort without TLS;
- Prerequisites
- Kubernetes is running
- Assumes single-node installation
- Test steps
- Modify apiserver.yaml
- CORS_ORIGIN_ADDR = http://<public ip>:30000
- INGRESS = NodePort
- PREFX=/api
- Modify gui.yaml
- APISERVER_HOST = <public ip>
- APISERVER_PORT = 30001
- UI_BASE_PATH = /ui
- kubectl create -f gui.yaml
- kubectl create -f apiserver.yaml
- kubectl label nodes 127.0.0.1 ndslabs-role=compute
- Open browser to: http://<public ip>:30000/ui/
- Configure basic Clowder, launch stack
- Confirm Clowder endpoint link points to NodePort (http://publicip:NodePort)
- Modify apiserver.yaml
- Prerequisites
Single-node, LoadBalancer + TLS
- Prerequisites
- Kubernetes is running
- Assumes single-node installation
- Wildcard CNAME entry for *.test.ndslabs.org pointing to single node IP
- Test steps
- Generate wildcard TLS certificate and key
- Prerequisites
Code Block |
---|
openssl genrsa 2048 > certs/ndslabs.key
openssl req -new -x509 -nodes -sha1 -days 3650 -key certs/ndslabs.key > certs/ndslabs.cert
#[enter *.test.ndslabs.org for the Common Name]
cat certs/ndslabs.cert certs/ndslabs.key > certs/ndslabs.pem
chmod 400 certs/ndslabs.key certs/ndslabs.pem |
- git clone https://github.com/craig-willis/ndslabs-startup
- cd ndslabs-startup
- ./ndslabs-up.sh
- Domain name: test.ndslabs.org
- Internal IP: accept default
- Open browser to:
- https://www.test.ndslabs.org/ui/ (Accept certificate)
- Configure basic Clowder, launch stack
- Confirm endpoint URI link points to ingress host (https://stackservice.test.ndslabs.org)
- A few kubectl checks
- kubectl get secrets --namespace=default
- ndslabs-tls-secret
- kubectl get ingress --namespace=default
- ndslabs-ingress with rules for /api and /ui
- kubectl get secret --namespace=demo
- demo-tls-secret
- kubectl get ingress --namespace=demo
- stack-clowder-ingress with rule for clowder "/"
- kubectl get secrets --namespace=default
- Stop the stack
- Confirm ingress deleted
- kubectl get ingress --namespace=demo
- Confirm ingress deleted
- Delete project
- ndslabsctl delete project demo
- Confirm secret deleted
- kubectl get secret --namespace=demo
Multi-node, LoadBalancer + TLS
- Prerequisites
- Kubernetes is running
- Assumes multi-node installation
- Wildcard CNAME entry for *.test.ndslabs.org pointing to load balancer node
- Assumes loadbalancer and default-backend services already deployed
- Test steps
- Generate wildcard TLS certificate and key
- Prerequisites
Code Block |
---|
openssl genrsa 2048 > certs/ndslabs.key
openssl req -new -x509 -nodes -sha1 -days 3650 -key certs/ndslabs.key > certs/ndslabs.cert
#[enter *.test.ndslabs.org for the Common Name]
cat certs/ndslabs.cert certs/ndslabs.key > certs/ndslabs.pem
chmod 400 certs/ndslabs.key certs/ndslabs.pem |
- git clone https://github.com/craig-willis/ndslabs-startup
- cd ndslabs-startup
- ./ndslabs-up-multinode.sh
- Domain name: test.ndslabs.org
- Internal IP: accept default
- Open browser to:
- https://www.test.ndslabs.org/ui/ (Accept certificate)
- Configure basic Clowder, launch stack
- Confirm endpoint URI link points to ingress host (https://stackservice.test.ndslabs.org)
- A few kubectl checks
- kubectl get secrets --namespace=default
- ndslabs-tls-secret
- kubectl get ingress --namespace=default
- ndslabs-ingress with rules for /api and /ui
- kubectl get secret --namespace=demo
- demo-tls-secret
- kubectl get ingress --namespace=demo
- stack-clowder-ingress with rule for clowder "/"
- kubectl get secrets --namespace=default
- Stop the stack
- Confirm ingress deleted
- kubectl get ingress --namespace=demo
- Confirm ingress deleted
- Delete project
- ndslabsctl delete project demo
- Confirm secret deleted
- kubectl get secret --namespace=demo