TODO:

• walkthrough with pictures - done
• the examples page - what is this?
• walkthrough of trying each of the code snippets (lots of pictures!) - Shannon does not have the tech to do this
  
  

Table of Contents

Introduction

Like other "fiddle" type environments such as JSfiddle, ScalaFiddle and Python Fiddle, BDFiddle provides an easy way for non-programmers to generate and play around with metadata and code "snippets" needed to generate metadata or convert/extract data from files they submit to Brown Dog. At its core, Brown Dog provides two kinds of transformation services, namely extraction and conversion. Extraction refers to the process of automatically generating specific metadata from an file. For example, detecting the number of faces in a photograph. On the other hand, conversion services deal with changing a file from one type to another (e.g. converting an MP4 video file to AVI format). There are multiple modes in which you can use BDFiddle. For example, for a regular user there is the "Automatic" (or basic) mode in which all available transformations (conversions or extractions) are applied to the submitted input file. For a more experienced user, there is the "Manual" (or advanced) mode in which the user can pick and choose specific transformations that they want to apply to the input file. BDFiddle is mainly meant to be a tool for exploring Brown Dog by playing around with its transformation services and hence currently we do not support uploading files that are larger than 5 MB through BDFiddle. If you need to test with larger files, there is a "Big Data" option in BDFiddle that will generate code snippets in multiple languages (currently shell script, native Python, and Brown Dog Python library are supported), which you can run at your end for local processing of files and uses Docker containers to move compute to data. You can also generate workflows using BDFiddle.

Navigation

Navigate to the Brown Dog website at https://browndog.ncsa.illinois.edu/ and sign up.

Click on the "BD Fiddle" button on the left.

or you can navigate directly using the URL https://browndog.ncsa.illinois.edu/bdfiddle/.

Login with your Brown Dog username and password.

You will be taken to the BDFiddle page itself.
This is what you will see. The whole page is setup to assist the user in:

1. Selecting a file
2. Selecting a transformation
3. Receiving the output
4. Accessing the code snippets

1. Select a File

You can choose from a file on your computer via the "Browse" button or select URL to choose a remote file.
Click on the "Get Transformations" button.

2. Select Transformation

Based on the type of the input file, a list of available conversions and extractions for that file are displayed.

Select the type of transformation needed from either of the scroll lists.

Press "Submit" button to initiate the transformation.

If the output needs to be emailed, then select the "Email Output" checkbox and provide the desired email address.

3. Receive Output

If a conversion was selected - the Output box will provide a location where the saved file of the desired format is located.

If an extraction was selected - the Output box will provide the desired extracted information. 

The below example displays metadata in the Output box generated from a .jpg.

4. Access Code Snippets

BDFiddle also generates code snippets which can be used to recreate the same Transformation in a local environment if the user needs to repeat the process from within code.
For each language, there are four templates: convert a file from a URL, convert a local file upload, extract from a file from a URL, and extract from a local file upload.
The values for the server, security token, input file, and output format are populated in appropriate template. 

Current snippets generated include:

• Terminal - code needed to run from the command line

• Python - rerun the transformation via bd.py - Brown Dog python code

• Native Python - rerun the transformation via native python language

• JavaScript - Instructions on how to re-run the process using Java Script

• Matlab - code to utilize in a Matlab program to re-run the transformation

• R - code necessary to re-run the transformation in "R"

Jupyter

In response to the popularity of Jupyter notebooks and the ease of combining run-able code and documentation in one interface that Jupyter provides, Jupyter functionalities have been added to BDFiddle.  In addition to copying the code snippet, a user can download a Jupyter notebook that contains the run-able code that is ready to implement locally (jupyter must be installed) or the user can simply open the notebook as an anonymous user on an NCSA multi-user service that utilizes Tmpnb.

Jupyter templates are JSON files that contain both the raw code (runnable) and mark down (documentation). 

Where Jupyter is available two additional buttons appear below the code snippet.

.

The buttons enable a user to

• Download the Jupyter file - to use in a local installation of Jupyter - The downloaded file can be immediately opened in a Jupyter notebook by typing “jupyter notebook” in the directory where the file is located.
• Open Jupyter Notebook - the input file and the Jupyter notebook file are posted to the remote NCSA server. 

The server uses Tmpnb (Temporary Notebook) and employs docker containers.   There are two control docker containers and as many notebook containers as needed. 

When the files are posted to the server, a notebook container is allotted to the new user in which they can edit and run the Jupyter notebook.

See Jupyter documentation for specific usage of Jupyter notebooks.

https://jupyter.readthedocs.io/en/latest/

Report Problem

Log in and click "Report Problem" in the Navigation bar then you will see the form below:

Then fill in the form with your name and detail of the problem. Screenshot or any file can be uploaded in the attachment. 

If the report is sent successfully, you will see the message as follows: