Date: Fri, 29 Mar 2024 03:31:47 -0500 (CDT)
Message-ID: <1356683308.310.1711701107256@os-confluence.ncsa.illinois.edu>
Subject: Exported From Confluence
MIME-Version: 1.0
Content-Type: multipart/related;
boundary="----=_Part_309_1691427123.1711701107256"
------=_Part_309_1691427123.1711701107256
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Content-Location: file:///C:/exported.html
Developer Workflows
Developer Workflows
All documentation pertaining to how developers can contribute to=
NDS Labs.
New to NDS Labs?
Start here: New Developer Workflow
Develop Workflows
- JIRA Workflows: =
Issue and project tracking workflows
- Git Workflows: Forking workflow with fe=
ature branches=20
- Fork repo (if applicable)=20
- Press "Fork" in GitHub UI
- Clone repo to make changes locally (if applicable)
- Ensure correct branch and sync with upstream before making additional c=
hanges=20
- git checkout master
- git pull upstream master
- Create a branch named after the Story (for e=
xample <=
a href=3D"https://opensource.ncsa.illinois.edu/jira/browse/NDS-174" class=
=3D"jira-issue-key">Getting issue details... STATUS )=20
- Make any necessary modifications locally on your branch
- Stage any modified files for commit=20
- git add path/to/modified/file.ext
- Commit any modifications to your local branch with a comment=20
- git commit -m "A comment about this commit"
- Push any local commits back up to your remote branch (your forked repo)=
=20
- When you are satisfied with your set of commits, create a Pull Request =
(PR) to view the diff=20
- Press "Pull Request" in GitHub UI
- Be sure to select the correct base and compare branches=20
- Select nds-org/ndslabs as the base fork
- Select master as the base branch
- Select your personal fork (USERNAME/ndslabs) as t=
he head fork
- Select your personal Story branch as the compare b=
ranch
- Scroll down and click on the "Files Changed" tab to briefly review your=
own Pull Request=20
- Ensure that all changes made on this branch were intentional
- If you are unsure about any specific code segments, comment in-line on =
the PR to ask for clarification
- If you are unsure about any general concepts changed or introduced, com=
ment in the section at the bottom of the PR
- Name your Pull Request after the Story / bra=
nch (i.e. "NDS-174: User can access console of running service via CLI=
")
- Enter a short description of any modifications, additions, or removals =
from the codebase=20
- If applicable, include a Test Case that the reviewer should run before =
merging the Pull Request
- Click "Create Pull Request"
- Docker Workflows: Push any necessary=
test images to Docker Hub=20
- Build test image=20
- docker build -t ndslabs/apiserver:dev .
- Tag test image with Story id (i.e. NDS-174)=
=20
- docker tag ndslabs/apiserver:dev ndslabs/apiserver:NDS-174
- Push test image to Docker Hub
- docker push ndslabs/apiserver:NDS-174
- Kubernetes Workflows: Sometimes u=
sed in testing new services or the API server
An Example
https://github.com/bodom0015/developer-workflo=
w
More Detail
Release Workflows
New Unstab=
le "Test" Release
Prerequisites
- Pull request has been created containing the changes to be reviewed / t=
ested
- Ensure that associated JIRA ticket contains a test case
- Ensure that your current code passes the test case that you have writte=
n
- Ensure documentation in Confluence is up-to-date
- Checkout your feature branch=20
- Sync with upstream=20
- git pull upstream master
- git push origin master
- Update any relevant documentation in GitHub
Developer'=
s Process (Semi-automatic)
- "Start Progress" on one of your assigned tickets (assign a new one if y=
ou have none assigned)
- If you haven't already, fork the upstr=
eam repository (you will only need to do this once per repository)=20
- Set up an automatic build of your new fork on =
DockerHub
- Configure the build to build all new branches from GitHub (choose "Bran=
ch" and leave the branch name blank)
- Push these new branch builds to a Docker image tag of the same name (si=
mply leave the tag name blank)
- Once saved,
- Clone your fork onto your local ma=
chine
- Create / switch to a development branch (named after one the JIRA ticke=
t associated with the work being done, i.e. NDS-XXX)
- Make any necessary changes to fulfill the JIRA ticket
- Commit all associated changes and push them to GitHub=20
- Any new changes pushed to any branch on your GitHub fork will be automa=
tically built into an image of the same name on DockerHub
- Mark ticket as "In Review" and assign to an available Tester
- Wait for the ticket to be assigned back to you
- Review the Tester's results=20
- If the Tester encountered problems, choose Review Rejected go back to #5 and address them
- If the Tester submitted comments or feedback, do your best to address t=
heir concerns or comment back to come to consensus
- If both Developer and Tester
Tester's Process (Man=
ual)
- Ensure that all associated auto-build images have completed their build=
s before beginning testing=20
- Links to these images should be provided with the test case.
- Run through the test case described in the associated ticket(s)=20
- A test case should be provided in the comments of each ticket, where ap=
propriate. If it is not, send it back to the Developer.
- The test case should include success and / or failure criteria. If=
it does not, send it back to the Developer.
- Update the associated JIRA tickets:=20
- Briefly include the results of your testing
- Be sure to leave feedback for the developer if you need them to take ac=
tion
- If something went wrong or the Tester still has questions, choose Review Rejected, assign it back to the Developer, and wait=
for a reply or the ticket to comeback to you
- If all test cases pass according to the standards set in the ticket and=
its comments, choose Review Accepted and assign=
it back to the Developer
- After all tickets in the associated JIRA tickets are marked as =
Resolved, merge the Pull Request into the master branch on the nds-org GitHub=20
- Merging any PRs to upstream master will automatic=
ally trigger a build of "latest" on DockerHub (see below)
New Unst=
able "Latest" Release
Prerequisites
- Any related PRs have been merged to master
- Ensure that smoke test passes
- Ensure documentation in Confluence is up-to-date
- Checkout your master branch=20
- Sync with upstream=20
- git pull upstream master
- git push origin master
- Update ALL documentation in GitHub
Process (Automatic)
- New "latest" Docker images are automatically built from the upstream master branch on GitHub.
- All new changes that make it into master on GitHub will automatically t=
rigger a build on DockerHub=20
Official=
Tagged Version Release (Stable)
Prerequisites
- Ensure that all tests pass
- Ensure documentation in Confluence is up-to-date
- Checkout your master branch=20
- Sync with upstream=20
- git pull upstream master
- git push origin master
- Update ALL documentation in GitHub
Process (Semi-automat=
ic)
- For each repo: branch off of
develop
to create a release b=
ranch=20
- ndslabs and workbench-helm-chart
OR
-
- kubeadm-bootstrap and kubeadm-terraform
- Roll versions forward=20
- ndslabs=20
- gui/swagger.yaml: NDS Labs swagger API spec version number
- apiserver/build.sh: NDS Labs API Server Docker image version tag
- apiserver/cmd/clientVersion.go: NDS Labs CLI version number / build dat=
e
- gui/Dockerfile: NDS Labs UI / webserver Docker image version tag
- gui/package.json: NDS Labs UI / webserver NPM package version num=
ber
- gui/ConfigModule.js: NDS Labs UI Angular app build version number
- workbench-helm-chart=20
- values.yaml: NDS Labs API Server + UI Docker image version tags
- kubeadm-bootstrap=20
- install-kubeadm.bash: Kubernetes / Docker version numbers
- init-master.bash: Helm version numbers
- kubeadm-terraform=20
- kubeadm-bootstrap git release/tag: assets/bootstrap.sh
- PR from release branch to master=20
- Thorough testing, then more testing, then merge to master
- Tag master with new version number (freshly tested and stable)=20
- Any new merges and tags will trigger new Docker images to be built
- Wait for newly-tagged resources to automatically finish building and pu=
shing Docker images=20
- Run a quick smoke test with newly-tagged resources
- Fix any last-minute errors directly on master and recreate release
- Backport any missing changes from release-x.x.x into develop=20
- This should include, at the very least, a commit from the release branc=
h that rolls forward to new version numbers
- git checkout develop && git pull origin master && git p=
ush origin develop? Why does this not work with a PR...
Legacy Process
- Regenerate Swagger API / Client from spec (this can be skipped if the s=
pec has not changed)=20
apiserver/???: generated Go swagger server
- gui/js/app/shared/api.js: generated AngularJS swagger client
Roll forward version numbers in ndslabs-deploy-tools and ensure that all values match the version number you are about=
to create:
roles/cluster-backup/defaults/main.yml
roles/ndslabs-api-gui/defaults/main.yml
roles/k8s-nagios-nrpe/defaults/main.yml
roles/k8-glfs-server-pods/defaults/main.yml
roles/k8-glfs-client-set/defaults/main.yml
- Create a new tag from master in GitHub for the new version (i.e. 1.0.0,=
1.0.1, etc):=20
- Repositories should be tagged in the following order when possible:
- ndslabs (API server / REST API / CLI / UI)
ndslabs-specs (service specs - starting with 1.2.0, this can be =
versioned separately from Workbench, but it should be noted upon a new rele=
ase which version of Workbench the specs release will target)
- workbench-helm-chart (Helm deployment to Kubernetes)
gluster (global file system - deprecated, no longer used)
cluster-backup (cron job for backing up glfs / etcd / kubectl du=
mp - starting with 1.2.0, this can be versioned separately from Workbench)<=
/li>
ndslabs-nrpe (nagios monitoring - these can now be versione=
d separately from the rest of Labs Workbench)
ndslabs-startup (dev-cluster startup - deprecated, no longer use=
d)
ndslabs-deploy-tools (ansible scripts - deprecated, no longer us=
ed)
- kubeadm-bootstrap (kubernetes deployment scripts - versioned separately=
, but it should be noted upon a new release which version of Workbench the =
release will target)
- kubeadm-terraform (terraform deployment procedure - versioned separatel=
y, but it should be noted upon a new release which version of Workbench the=
release will target)
ndslabs-devenvs (developer environments - these can now be =
versioned separately from the rest of Labs Workbench)=20
- NOTE: ndslabs-devenvs contains a large number of casca=
ding images that will quickly fill up the build queue, that's why we do it =
last
- New versioned Docker images are automatically built from the upstream tags created on GitHub.
- All new tags that are created will trigger a build=20
- Roll forward version numbers in source and ensure that all values match=
on upstream master on Git=
Hub:=20
- Swagger API
apis/swagger-spec/ndslabs.yaml: NDS Labs swagger API spec version nu=
mber (deprecated - use the file below instead)
- gui/swagger.yaml: NDS Labs swagger API spec version number
- API Server:=20
- apiserver/build.sh: NDS Labs API Server Docker image version tag
apiserver/version.go: NDS Labs API / Server version number =
file no longer exists
- CLI Client:
apictl/build.sh: NDS Labs CLI version number file no longer exis=
ts
apictl/cmd/clientVersion.go: NDS Labs CLI / API version number&n=
bsp;file no longer exists
- UI Client:=20
- gui/Dockerfile: NDS Labs UI / webserver Docker image version tag
- gui/package.json: NDS Labs UI / webserver NPM package version num=
ber
gui/bower.json: NDS Labs UI Angular app Bower package version number=
file no longer exists
- gui/ConfigModule.js: NDS Labs UI Angular app build version number
------=_Part_309_1691427123.1711701107256--