...
Note: In the below steps, <version_branch>
should be replaced with the version branch of Daffodil you are working on based on the Branch Workflow.
- Once the code has been reviewed, checked in and the JIRA ticket has been updated as Resolved by Developer, the ticket will be assigned to the Test lead.
- The Test Lead will assign the JIRA ticket to the appropriate Tester.
- If the ticket pertains to a new feature, the JIRA bug is completed by Developer, status is "Resolved"
JIRA bug is assigned to tester
Tester makes note of requirements associated with the ticket and finds any other requirements (in Testlink) that are relevant. - The Tester writes appropriate test cases to provide adequate test coverage for all of the requirements.
- Tester makes sure the requirements are mapped to the current Spin in Testlink, and maps them if they are not already:
- Click Edit on the requirement pane
- Check the Spin:
- Hit SaveClick Save.
Tester creates a new bug branch
Code Block $ git fetch --prune origin $ git checkout -b <relevant_branch_name> --track origin/<version_branch>
- Tester verifies any existing developer-created tests in scala-new and uses tests to better understand feature.
- Tester moves developer-created tests from scala-new to scala.
- Tester creates more tests to ensure completeness of test coverage.
Completing test development includes repeating the following steps many times until you are satisfied with your test coverage:
Code Block language bash $ # edit files $ git add <files> $ git commit $ git fetch --prune origin $ git rebase origin/<version_branch> $ sbt test
When creating tests it is helpful in sbt to only run a single test (or small group of tests) at a time:
From the sbt prompt:
Code Block > test-only [test class] -- --tests=<regex>
For example:
Code Block > test-only edu.illinois.ncsa.daffodil.section15.choice_groups.TestChoice -- --tests=test_choiceDelim\d+
Please read A Note About Git Commit Messages for the correct commit message format. Please follow those standards, with the additional requirement that the last line should contain only the JIRA bug number preceded by an empty line, e.g. "\nDFDL-123". If there is more than one bug related to the commit, separate them with commas.
- If a bug is found:
- Make sure any associated tests are in scala-debug, not scala
- Reopen JIRA ticket and provide the following:
- Expected result and actual result. Include error and traceback if applicable.
- Name and location of failing test(s)
- Assign ticket to original developer
- Once a Test Case has been written (See Step 16), under Test Execution in Testlink, mark test as “Failed”
- Click bug button and enter JIRA ticket number (e.g. DFDL-123)
When finished creating tests, tester ensures changes work with the latest upstream
Code Block $ git fetch --prune origin $ git rebase origin/<version_branch>
Tester squashes commits into a reasonable patchset.
Code Block $ git rebase -i origin/<version_branch>
Please pay attention to commit message format
Tester ensures all tests pass
Code Block $ sbt test
Tester sets <version_branch> branch to bug branch and pushes
Code Block language bash $ git checkout <version_branch> $ git reset --hard <relevant_branch_name> $ git push origin <version_branch> # do not use the --force option here
If commit was a bug fix or a test on <version_branch> and a newer version branch exists:
Tester merges the change into the newer version branch, and fixes any conflicts:
Code Block language bash $ git checkout <new_version_branch> $ git merge <version_branch>
Tester ensures all tests pass
Code Block $ sbt test
If tests fail due to incorrect conflict resolution, changes should be made and amended to the merge commit until tests pass:
Code Block language bash # make changes $ git add <files> $ git commit --amend
Tester pushes changes
Code Block language bash $ git push origin <new_version_branch> # do not use the --force option here
Tester deletes bug branch
Code Block $ git branch -D <relevant_branch_name>
- Tester enters new tests into Testlink by performing the following steps:
- Navigate to the Test Specification area in Testlink
- Select the current Spin and creates a new test
- Map the test case to the corresponding requirement(s).
- Add test to the Test Plan
- Repeat steps b through d for each test, including developer-created tests.
- Execute tests in the Test Execution tab.
- If no bugs have been found through the course of testing, tester prepares ticket for closing, performing the following tasks:
- Reference the new tests in the ticket
- Add comment to ticket stating that there is now sufficient test coverage
- Close ticket and leave assigned to yourself.
...