Adding Tools

TODO (use lots of pictures!):

• XSEDE Tutorial content
• BD-tmux
• Create a converter
• Create an extractor
• Dockerization
• Tools catalog

Deploying an Extractor from a Single Call Method

Here is described the entire process for taking a working piece of code and deploying it as a BrownDog Extractor.  It is assumed that the method can be invoked from a single call.  In this example, we are using the python extractor wrapper and will invoke a python function.  In a very similar fashion, a method developed in a language other than python can be invoked using subprocess.

The main steps:

1. Dockerize the extractor
2. Deploy the extractor
3. Add the extractor to the tool catalog

Creating an extractor from working python code:

In this example, the extractor "killed_photos" will be used.  Please, use the appropriate names for your extractor.

1. Install pyClowder

   pip install git+https://opensource.ncsa.illinois.edu/stash/scm/cats/pyclowder.git

2. Get your code together

   1. Clone the extractor template and rename the directory to an appropriate name

      git clone ssh://git@opensource.ncsa.illinois.edu:7999/bd/extractors-template.git
      mv extractors-template/ killed_photos

   2. Bring in working python code

      1. Create a source directory within the extractor template

         cd killed_photos/
         mkdir src
         cd src

      2. Put the existing code into the src directory.

         1. It is here assumed that the entry into the working code (the callable function) will be at the top level of the src directory
         2. Your directory structure should look like this.  The source code will probably be a lot more complicated, but the key is that there is an entry point - in this case main.py

adding to and Deploying from the Tools Catalog

Navigate to http://browndog.ncsa.illinois.edu:9000/ and Login using the button at the top right.

NOTE:  During the Beta Release the Logins for Brown Dog and the Tools Catalog use different authentication systems so you will need a Username and Password for both.

Either Log in with your SecureSocial password or Sign Up via the link provided.

When you create an account you will receive an email with a link to create the account.  Fill out the form and then use your new username and password to log into the Tool Catalog.

Adding or Contributing a Tool to the Tools Catalog

To begin the process of submitting your dockerized tool click "Contribute" at the top of the page.

You will be taken to your individual home page which will show the tools you have already contributed and a button to begin creating a tool "Create Tool".

You will be taken to an entry screen:

Enter:

• Title of the tool - note the size restrictions - no other restrictions as to format currently - *Required
• URL - location of the dockerized tool - *Required
• At a glance - a short description of the tools - Max of 140 characters - *Required
• Description - a full description of the tool
• Citation - instructions on how someone would cite the tool if they used it in their work
• YouTube Video URL - link to an instructional video if available
• Tool Type - Indicate if the tool is a Converter or Extractor

and click "Submit".

When the system returns the tool has been created and the information is displayed.

Review the information and edit if necessary.

Click "Add Brown Dog Script" to proceed.

Enter:

• Title - this is the title of the script and not the tool - there are no naming conventions currently - *Required
• Description - this is the description of what the script does - there are no length standards - *Required

Click "Next".

Additional fields will display to allow files to be submitted for the script.

Click "Choose File" to submit the following files:

• Brown Dog Script File
• Docker File
• Example Input file - for testing and illustration purposes
• Example Output - for testing and illustration putposes
• Queue Name - can be left blank  - advanced users may know the specific queue that will be used in the Elacticity Module
• Docker Image Name - can be left blank
• VM Image Name - can be left blank

Click "Submit".

Review the script information and click "Author Approve" to submit the script.  "Edit" is available to change any details.

If an incorrect file was attached, click "Delete" and re-submit the script.

At this point a Brown Dog Admin is notified that a new Tool has been submitted and they will review and test the script and approve it for use.

The tool will then be viewable in the "Browse" list of all tools in the current installation of the Tools Catalog.