Page tree
Skip to end of metadata
Go to start of metadata
Target releaseTBD
Epic(s)Beta Release Tasks
Document status
Document owner
Lead Developer/Architect
Lead Tester


  • Initial open release of Brown Dog
    • Highlight what can be done with a Data Transformation Service through an initial suite of transformation tools, a number of sample clients, and sample data allowing a user to quickly try things out
    • Email batches of users from signups thus far.
    • Email should contain a link to a feedback survey.

Test criteria

  • All tools in beta suite pass bi-hourly tests when run in parallel submission mode

Background and strategic fit

The beta release will:

  • Provide the community with a first glimpse of Brown Dog
  • Allow the Brown Dog team to increase exposure through presentations and hands on tutorials
  • Allow the team to identify bugs through real world usage and harden the service towards a 1.0 release


  • Many users will want to try out the system without too much effort
  • Will need to support very novice users (easy to use interfaces/samples)
  • Some users will begin to regularly use the system, throwing many requests at the service


#TitleUser StoryEffortStatusNotesAssignment
1AccessThe user will be able to signup on the Brown Dog web page and get a quota in terms of alotted requests per month as well as requests per hour.  The Brown Dog account will give them access to BD Fiddle, each of the clients, the Tools Catalog, and the ability to generate tokens to make calls from within the applications they create.LowSeems be nearly there.
2Beta Tools & Tools Catalog

The user will be able to try out one of a number of tools spanning a range of scientific domains and general data usage tasks.  Upon signing up for an account the user will be able to explore the tools catalog (utilizing the same account) to see what transformation tools are available as well as what each is capable of doing.  These tools will be deployed under the elasticity module so that a user can try the tool via one of a number of possible client interfaces as well as tested every other hour to ensure each tools available.  The user should come away with a sense of the breadth of data currently and potentially supported over time, the flexibility of the system, the potential for contributing tools of their own, and the types of tools that would fit as part of the service.

HighWork in progress
3Fence API v0Users who are developers will have access to a consistent, compact, and straight forward API providing them with extraction, conversion capabilities to be called within their own applications.MediumDefining the API is straightforward, however, will need to fix all of the clients 
4Production updates

The user will have access to the latest features through the production instances of Clowder, Polyglot, Fence, and BDMon.

Production Updates - Detail

LowMost auto-deployed at this point
  • Risk Assessments - the most work needs done on the Elasticity Module - triage of remaining tasks will need completed to determine which ones should be in Beta
5bd-swarmThe user will experience relatively consistent response times when submitting requests, whether there are many requests being made to the system at that instant or only a few.LowNeed a few additional scripts in bd-swarm to scale and monitor things  

The user will explore Brown Dog for the first time through BDFiddle.  As a user friendly web interface the user will be able to upload data, try out transformations, and discover alternative ways to utilize Brown Dog within other programming languages.  Specifically, the user will be able to:

  • upload a file or point to file URL
  • select an output OR a specific tool for a transformation
  • get the output of the transformation
  • get a code snippet that can be pasted and executed in Matlab, R, and Python
  • launch a Jupyter notebook to try the python code
  • download the needed libraries for Matlab, R, and Python
  • trace executions via a datawolf workflow link presented as part of the results

BD-1416 - Getting issue details... STATUS

Epic Link Status
In Progress In Review Done To Do T:
BDFiddle 2 1 20 15 38
Total Unique Issues: 2 1 20 15 38
Showing 1 of 1 statistics.
View in JIRA


Nearly Complete, needs polish, fix to Jupyter, fix to DataWolf, bug testing

7Test filesBDFiddle should have a tab containing a sample file for each of the the beta tools allowing the user to quickly try out any of the capabilities provided.LowWork in progress

A quick and user friendly interface allowing users to try Brown Dog on arbitrary web pages.  From BDFiddle the user will be able to drag the Brown Dog graph in the corner to their bookmarks toolbar.  While navigating the web the user will be able to click on this bookmark to activate the bookmarklet which will:

  • add a menu to each link/image on the page allowing the user to download the corresponding file as a different format
  • popup a search box when the user presses ctrl+F allowing the user to search across links/images on the page based on extracted contents from the corresponding files (e.g. extracted text)

Sample pages should be provided to the user so that they can quickly try this interface out.  The bookmarklet should work with each of the sample files on the BDFiddle sample files page as well.


Some bug fixes with regards to how data can be referenced on the web






9Command Line InterfaceThe user should be able to install on Ubuntu via "apt-get browndog" the Brown Dog command line interface and then utilize it to convert local files (e.g. bd -output jpg foo.tif) and index/search local folders (e.g. bd -v /foo).  The user will be able to handle large files by passing in the --bigdata flag which moves needed tools locally vs send the data to the remote servers.  The user will be able to get help by passing in the --help flag.  The user will be able to tell what data types are supported, available tools, output formats, and tools that can be moved locally via appropriate flags.LowNeed to figure out the apt-get part, do some cleanup, documentation, and testing 

Windows Client

The user will be able to download a setup.exe from the Brown Dog webpage to install the Windows client.  The client will modify the Windows file manager such that when a user can convert local files by simply right clicking on the file and selected the desired resulting format and index/search local folders by similarly right clicking on a folder to index then search its contents based on extracted metadata.

BD-33 - Getting issue details... STATUS

Epic Link Status
In Review Done T:
File Manager 1 5 6
Total Unique Issues: 1 5 6
Showing 1 of 1 statistics.
View in JIRA


Some cleanup, documentation, and testing still needed


11R LibraryThe user will be able to run R snippets obtained from BDFiddle by copying and pasting them in their code and installing the bd.R library via cran (e.g. install.packages("browndog")).LowNeed to figure out the cran part and do some testing
12Python LibraryThe user will be able to run Python snippets obtained from BDFiddle by copying and pasting them in their code and installing the library via pip (e.g. pip install browndog).LowNeed to figure out the pip part and do some testing
13Matlab LibraryThe user will be able to run Matlab snippets obtained from BDFiddle by copying and pasting them in their code and installing the bd.m library downloaded from the Brown Dog web page.LowNeeds some code cleanup and testing

The user will be able to find documentation and examples to help them along.  Specifically up to date:

  • Swagger documentation of the API
  • Wiki walk-throughs
  • Man page for bdcli
  • Help button for windows client
  • files with each library
HighWill need to recruit effort from the entire team
15User SupportUsers will have user support towards quickly fixing service issues and answering questions.  The user will be able to get support from a support email and confluence questions.MediumWill need a plan to support with dev staff for the time being and transition over to a new dev ops branch of the group over time
16Beta User Email & SurveyUsers that have signed up to Brown Dog will receive an email notifying them of the beta release, some quick instructions (with graphics), link to BDFiddle, and link to feedback survey.LowCan use signup email and XSEDE survey as a starting point. 

User interaction and design


Below is a list of questions to be addressed as a result of this requirements document: