Testing Matrices

There are typically two approaches to testing:

• "Break" Testing
• Scenario/Usability Testing

Break Testing

Break Testing takes the approach of finding multiple ways to break the system and in this case it would be via the UI so all supported browsers would be tested in each use case.  The number of permutations of tests becomes exponential when you add several browsers to test on each field within the UI:

• Name field - tested for:
  • special characters
  • length
  • spaces
  • combinations of all
    • Explorer
    • Firefox
    • Chrome
    • Safari
      • what versions?
        • etc.

And that is only one field tested.

Scenario/Usability Testing

Brown Dog Test-Plan-Scenario.xlsx

In our case - due to the limited amount of time available before the Beta Release itself for Brown Dog, we have the option to choose Scenario Use testing.  Basically this means we are trusting that the person who will be using the system initially is not focused on ill intent and wants the system to work for them. 

Goal of Scenario Testing would be to walk through a typical set of steps a user would take to perform a particular function and no more:

• Create Account or Log In
  • User/Developer signs up on BrownDog webpage
    • http://browndog.ncsa.illinois.edu/
  • User receives email with more information and link to survey
  • User fills out survey - are we doing survey in current account validation setting?
  • Account is activated
  • Log in should work for BDFiddle and Tools Catalog
• Go to BDFiddle
  • Perform a Conversion and a Extraction
    • Convert a png image to tif image - should we allow random selection? we need to test more than png to tif?
    • Run extraction service with all test files (Examples link is at top of page - it links to example test files: https://browndog.ncsa.illinois.edu/bdfiddle/examples.php
  • etc
  • View output for usability
  • Take code snippet and execute locally in own code
  • Create a Jupyter Notebook
• Upload a tool to the Tools Catalog
  • Creating a python script that convert png image to jpg image
  • Creating a python script that finds images which contain a word "internet" from set of given images
  • etc
• Use a tool from the Tool Catalog
  • Excel Converter? Luigi's tool?
• Use the BD-API in local code - need to make sure person who tests has the skill set
  • Who is able to program with certain languages that BD supports (such as python, matlab, R, curl?, etc)
    • BD students
    • External volunteers
    • ISDA developers who have not used BD
• Possible Additional Tasks
  
  • Bookmarklet
  • File Manager Extension
  • Creating a python script that convert png image to jpg image
  • Creating a python script that finds images which contain a word "internet" from set of given images
  • Use Command Line Interface to Access Brown Dog
  • etc

With each section containing the steps to perform each Scenario.

Results

If a typical user can perform the above standard functions the beta release of Brown Dog would be a viable product.  We could then work on Break Testing after the Beta is released.