Signing Up

1. (Done) User/Developer signs up on BrownDog webpage
   • http://browndog.ncsa.illinois.edu/
2. (Done) User receives email with more information and link to survey
3. (Automatic email) User fills out survey
4. (Done) User is contacted by Brown Dog developer and account is activated
5. (Not for April 4) Tools catalog support for single sign-in

Creating and Adding a New Tool

1. (Gregory Jansen, Rui Liu) Developer downloads BrownDog development environment in docker container (in dockerhub, link on BD webpage) BD-956 - Getting issue details... STATUS BD-957 - Getting issue details... STATUS BD-958 - Getting issue details... STATUS BD-959 - Getting issue details... STATUS BD-960 - Getting issue details... STATUS BD-1009 - Getting issue details... STATUS BD-1017 - Getting issue details... STATUS BD-1007 - Getting issue details... STATUS BD-1016 - Getting issue details... STATUS
   • Contains running Clowder instance, Polyglot instance, basic development tools, mabye even an IDE configured and ready to go
   • Contains a bare-bones extractor ready to be modified for a new tool (with comments suggesting where to edit, in various languages e.g. python, C++, Java, ..., and docker sample file)
   • Contains a bare-bones converter ready to be modified for a new tool (with comments suggesting where to edit, in various languages e.g. bash, bat, ahk, python, ..., and docker sample file)
   • (Done) Polyglot changes from redirecting to SS to downloading and serving converted files to users' "/file/" requests, to support 1 Pol + multi SS docker containers on the same VM.
   • (Done - Rui provided the files/images.) docker-compose.yml file, docker images for Polyglot server + 2 different SS to work together on the same VM.
2. (Gregory Jansen, Smruti Padhy) Developer installs tool, creates BrownDog interface script(s)
   • Adds sample input file(s)
   • Example extractor: imagemagick edge detector
   • Example converter: imagemagick jpg to ico converter
3. (Gregory Jansen, Smruti Padhy)Developer creates dockerfile
   • For extractors this should contain only the tool, its dependencies, and the extractor
   • For converters this should contain the basic Polyglot setup, ideally only the Software Server, the tool, its dependencies, and the converter interface script
   • BD-983 - Getting issue details... STATUS
4. (Rui Liu, Inna Zharnitsky) Developer submits tool to Tools Catalog (using BrownDog account) BD-961 - Getting issue details... STATUS BD-962 - Getting issue details... STATUS
   
   • (Done) BrownDog interface script(s), dockerfile, sample input, sample output
   • TODO: Documentation on how to add tool to Tools Catalog
   • (Done) Through web interface
   • Through script in development environment (probably the ideal way) BD-962 - Getting issue details... STATUS
5. (Inna Zharnitsky, Marcus Slavenas) Admin reviews and tests tool by pressing button which deploys tool in a separate Clowder/Polyglot sandbox, passes in the given input file, and checks the obtained output file against what is provided
   1. Manual review source code and docker file
   2. If ok, reviewer submits tool for testing from tool catalog ui  BD-968 - Getting issue details... STATUS BD-969 - Getting issue details... STATUS BD-1055 - Getting issue details... STATUS
      1. Build docker image using local parameters
      2. If user provides sample output run script to test inputs/outputs
         1. if not, manual inspections is required
      3. Scripts show pass/fail in tool script
   • Admin click accept or reject
6. (Rui Liu, Inna Zharnitsky) Admin deploys tool by selecting instances to deploy interface on  BD-916 - Getting issue details... STATUS BD-954 - Getting issue details... STATUS BD-733 - Getting issue details... STATUS
   • e.g. dts-dev or dts
   • selects minimum number of instances to provision (e.g. 5 instances)
   • presses button to deploy
   • Elasticity backend:  deploy docker images BD-890 - Getting issue details... STATUS , deploy VM images BD-930 - Getting issue details... STATUS , stop BD-941 - Getting issue details... STATUS .
   • Tool Catalog UI: BD-733 (deploy), BD-954 (stop).
   • Task: integration testing after the web UI part is done. BD-975 - Getting issue details... STATUS
7. (Sandeep Puthanveetil Satheesan) Admin receives status page showing number of tools deployed and where, load on each tool, etc...
   • No support for this at this time.
   • How to pull this from MongoDB

Using a Tool

• (Luigi Marini) API Gateway with keys and tokens (uses Crowd credentials)
  • (Not for April 4) Forwarding user credentials to Clowder
  • Clients
    • (Marcus Slavenas) Matlab  BD-620 - Getting issue details... STATUS
      • Working and includes test script: DAP Outputs, DAP Convert, DTS Extract, and Download File
      • Not yet translated from bd.py: index, find, descriptor distance
    • (Smruti Padhy) R – available
    • (Luigi Marini) bash script using curl  BD-984 - Getting issue details... STATUS
    • (Luigi Marini) Python – available bd-cli
    • (Smruti Padhy) Windows – Dap is there, for DTS not sure  BD-1004 - Getting issue details... STATUS BD-1005 - Getting issue details... STATUS
    • (Smruti Padhy) Bookmarklet - 
    • (Smruti Padhy) Chrome Extension
    • Add keys and tokens to all clients
      • BD-986 - Getting issue details... STATUS BD-987 - Getting issue details... STATUS BD-988 - Getting issue details... STATUS BD-988 - Getting issue details... STATUS BD-989 - Getting issue details... STATUS BD-990 - Getting issue details... STATUS BD-991 - Getting issue details... STATUS
• (Jong Lee, Eugene Roeder) Simple web interface (requires BrownDog account) BD-967 - Getting issue details... STATUS BD-970 - Getting issue details... STATUS BD-976 - Getting issue details... STATUS
  
  • Web form with field box for file URL or upload
  • Tool selection dropdown box (means of selecting a specific tool)
  • Output select dropdown box (means of allowing system to select tool)
    • Clowder will run all relevant extractors (obtained by selecting "json" as the output)
    • Polyglot will use I/O-graph
  • Suggestions pane will update below
    • REST API performing this same operation
    • Python code in Jupyter lab notebook
    • R code
    • Matlab code
    • curl commands
    • php