This page is under construction.

Background

Tools Catalog is the main web application that the tool contributors of Brown Dog will interact with. Current version of Tools Catalog (0.4.0) needs to be simplified and made more user friendly, in order for that to happen. Tools Catalog 0.4.0 is written in Scala using the Play framework, which is less popular than languages like Python, JavaScript, etc. There are also many features in Tools Catalog 0.4.0, which are either not needed or not clearly understood by the users. This will be an opportunity to re-design and re-implement Tools Catalog with only the required set of functionalities, which are user friendly as well.

Name

We have this ongoing task  and will tentatively use the name "Brown Dog Transformations" (previously suggested by Ben) or "BD Transformations" in short as we describe Tools Catalog "2.0" on this page. For the remaining part of this document we will use BD Transformations in place of Tools Catalog 2.0. This will help clarify thoughts related to Tool vs Software.

Definitions

This section will define the main concepts that will need to be clearly understood for easily interacting with BD Transformations. An example can be Software vs Transformation or any new terminology that will be used in BD Transformations.

Converter: Program that does data format conversion.

Extractor: Program that process and extract metadata from data.

Transformation: Any converter or extractor or extractor program.

Software: Software tool / library that is used by an extractor or converter to do data transformation.

converters_info.json: A JSON file that describes a converter. 

extractors_info.json: A JSON file that describes an extractor.

Minimum Viable Product (MVP): Version of BD Transformations with only a few basic features. This will be developed through some kind of a hackathon.

User Roles

The user roles in BD Transformations will be:

  1. Administrator
    1. Update / Delete transformations created by contributors
    2. Update / Delete software added by contributors?
    3. All Viewer permissions
  2. Contributor
    1. Create new transformation and link it to one or more existing software.
    2. Add a new software to BD Transformations
    3. Update created transformations [adding the word "created" to make it clear that a contributor can only manage the transformations they contributed]
    4. Delete created transformations
    5. View created transformations
    6. All Viewer permissions
  3. Viewer (can be any user who does not login)
    1. View all transformations
    2. View all software

Features

Wireframes

  1. Home (Yan)

  2. View Transformation (Sandeep)

  3. Login (Bing)

  4. POST Endpoint

  5. Administrator

  6. Contributor (Low priority for now)

Technologies 

Tasks

JSON Document Example

Mapping Table

Note: Changing to camel case notation for field names as MongoDB documents follow JSON.

Sl. No.Tools Catalog FieldTransformations Catalog FieldTypeRemarks
1.
status
N/AInteger
2.
creationDate
dateCreated
Date
3.
updateDate
dateUpdated
Date
4.
downloads
N/AInteger
5.
license
N/AString
6.
whatsnew
description
String
7.
compatibility
N/AString
8.
url
url
StringURL
9.
sample_input
N/AStringURL
10.
sample_output
N/AStringURL
11.
interfaceLevel
N/AInteger
12.
dockerimageName
dockerImageName
String
13.
vmImageName
N/AString
14.
dockerfile
N/AStringEmail ID
15.
author
author
String
16.
dependencies
dependencies
ArrayArray of Strings
17.
transformation_type
transformationType
StringOriginally this field type was an integrer with values 1 for converter and 2 for extractor
18.
transformation_id
transformationId
StringExtractor name / converter name
19.
title
title
String

Transformations Catalog Example

{
    "_id" : ObjectId("53864aee7400009900ec8d22"),
    "dateCreated" : ISODate("2014-05-28T20:45:34.412Z"),
    "dateUpdated" : ISODate("2016-11-21T04:09:47.836Z"),
    "description" : "Identify tumors in microscopy images.",
    "url" : "https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler/cellprofiler-tumor?raw",
    "dockerImageName" : "",
    "author" : "ruiliu@illinois.edu",
    "dependencies" : [
           "Cell Profiler"
    ],
    "externalServices" : [
    ],
    "transformationType" : "extractor",
    "transformationId" : "",
    "title" : "Tumors",
    "status" : "submitted"
}

Extractor Info Example

{
    "@context": "http://clowder.ncsa.illinois.edu/contexts/extractors.jsonld",
    "name": "ncsa.csvheaders",
    "version": "1.0",
    "description": "Extracts the headers of CSV file. Only looks at the first row.",
    "author": "Yan Zhao <yanzhao3@illinois.edu>",
    "contributors": ["Luigi Marini <lmarini@illinois.edu>"],
    "contexts": [
        {
            "csv_headers": "http://clowder.ncsa.illinois.edu/metadata/ncsa.csvheaders#csv_headers"
        }
    ],
    "repository": [
        {
            "repType": "git",
            "repUrl": "https://opensource.ncsa.illinois.edu/bitbucket/scm/iarp/extractors.git"
        },
        {
            "repType": "docker",
            "repUrl": "clowder/extractors-csvheaders"
        }
    ],
    "process": {
        "file": [
          "text/csv"
        ]
    },
    "external_services":[],
    "dependencies": [],
    "bibtex": []
}

Converter Info Example

{
    "name": "ImageMagick",
    "version": "1.0",
    "description": "Converts images from one format to another",
    "author": "Kenton McHenry <mchenry@illinois.edu>",
    "contributors": ["Bing Zhang <bing@illinois.edu>"],
    "repository": [
        {
            "repType": "git",
            "repUrl": "opensource.ncsa.illinois.edu/bitbucket/scm/pol/converters-imagemagick.git"
        },
        {
            "repType": "docker",
            "repUrl": "ncsapolyglot/converters-imagemagick"
        }
    ],
    "input_formats": [
          "png", "bmp"
        ],
    "output_formats": [
          "jpeg", "jpg", "pdf"
        ],
    "external_services":[],
    "dependencies": ["ImageMagick"],
    "bibtex": []
}

Current Tools Catalog Example

{
    "_id" : ObjectId("53864aee7400009900ec8d22"),
    "status" : 4,
    "creationDate" : ISODate("2014-05-28T20:45:34.412Z"),
    "updateDate" : ISODate("2016-11-21T04:09:47.836Z"),
    "downloads" : 2,
    "license" : "UIUC/NCSA Open Source License.\r\nhttp://otm.illinois.edu/uiuc_openSource",
    "whatsnew" : "Identify tumors in microscopy images.",
    "compatibility" : "",
    "url" : "https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler/cellprofiler-tumor?raw",
    "sample_input" : "",
    "sample_output" : "",
    "interfaceLevel" : 2,
    "dockerimageName" : "",
    "vmImageName" : "",
    "dockerfile" : "",
    "author" : "ruiliu@illinois.edu",
    "dependencies" : [
           "Cell Profiler"
    ],
    "transformation_type" : "extractor",
    "transformation_id" : "",
    "title" : "Tumors"
}