Versions Compared

Old Version 13

changes.mady.by.user Sandeep Puthanveetil Satheesan

Saved on

compared with

New Version Current

changes.mady.by.user Sandeep Puthanveetil Satheesan

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • All users can view all transformations that are linked to a software.
  • All users can view a single transformation and the software that it uses.
  • Contributors will be able to add a transformation through the BD Transformations interface.
    • This will be made available through a form that the contributor can fill and submit.
    • Ability to include icon
    • Ability to include link to software web page (e.g. ImageJ)
    • Ability to add a URL to the transformation  - source code / docker image name / website
  • Contributors will also be able to import a new transformation into the catalog by uploading an extractors_info.json or converters_info.json file.
    • Any missing piece of information can be filled in by the contributor.
  • Contributors will be able to link their transformation with one more existing software in BD Transformations. They will be able to add a new software if it doesn't already exist in BD Transformations.
  • There won't be any formal approval process for the submitted transformations. All transformations that get submitted will be available to the public.
  • Administrators can update or delete any transformation as they feel appropriate.

Wireframes

Home

Wireframe
initialResourceID2278E287-509B-183B-1098-2EC38DDDB7D8
platformArchiveID137433183
AlignmentLeft
viewsinglegrid
platformArchiveNameBalsamiqProject_137432356
initialBranchIDMaster

  1. Home (Yan)

  2. View Transformation (Sandeep)

  3. Login (Bing)

  4. POST Endpoint

  5. Administrator

  6. Contributor

...

Login

...

  1. (Low priority for now)

Technologies 

  • Flask Templates
  • Bootstrap
  • JQuery
  • Flask Services
  • MongoDB
  • Python LDAP Library
  • Python MongoDB Library

Tasks

  • Script for exporting data from Tools Catalog to BD Transformations
  • Code to talk to NCSA LDAP for authentication
  • Clusterman endpoint to read from BD Transformations database
  • Flask Services
    • POST <extractors/converters>_info.json to MongoDB
    • GET <extractors/converters>_info.json for a specific transformation
    • GET all transformations
    • GET all transformations that uses a specific software

JSON Document Example

Mapping Table

Note: Changing to camel case notation for field names as MongoDB

...

  • Code to talk to NCSA LDAP for authentication

documents follow JSON.

Sl. No.Tools Catalog FieldTransformations Catalog FieldTypeRemarks
1.
status
N/AInteger
2.
creationDate
dateCreated
Date
3.
updateDate
dateUpdated
Date
4.
downloads
N/AInteger
5.
license
N/AString
6.
whatsnew
description
String
7.
compatibility
N/AString
8.
url
url
StringURL
9.
sample_input
N/AStringURL
10.
sample_output
N/AStringURL
11.
interfaceLevel
N/AInteger
12.
dockerimageName
dockerImageName
String
13.
vmImageName
N/AString
14.
dockerfile
N/AStringEmail ID
15.
author
author
String
16.
dependencies
dependencies
ArrayArray of Strings
17.
transformation_type
transformationType
StringOriginally this field type was an integrer with values 1 for converter and 2 for extractor
18.
transformation_id
transformationId
StringExtractor name / converter name
19.
title
title
String

Transformations Catalog Example

Code Block
languagejs
linenumberstrue
{
    "_id" : ObjectId("53864aee7400009900ec8d22"),
    "dateCreated" : ISODate("2014-05-28T20:45:34.412Z"),
    "dateUpdated" : ISODate("2016-11-21T04:09:47.836Z"),
    "description" : "Identify tumors in microscopy images.",
    "url" : "https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler/cellprofiler-tumor?raw",
    "dockerImageName" : "",
    "author" : "ruiliu@illinois.edu",
    "dependencies" : [
           "Cell Profiler"
    ],
    "externalServices" : [
    ],
    "transformationType" : "extractor",
    "transformationId" : "",
    "title" : "Tumors",
    "status" : "submitted"
}

Extractor Info Example

Code Block
languagejs
linenumberstrue
{
    "@context": "http://clowder.ncsa.illinois.edu/contexts/extractors.jsonld",
    "name": "ncsa.csvheaders",
    "version": "1.0",
    "description": "Extracts the headers of CSV file. Only looks at the first row.",
    "author": "Yan Zhao <yanzhao3@illinois.edu>",
    "contributors": ["Luigi Marini <lmarini@illinois.edu>"],
    "contexts": [
        {
            "csv_headers": "http://clowder.ncsa.illinois.edu/metadata/ncsa.csvheaders#csv_headers"
        }
    ],
    "repository": [
        {
            "repType": "git",
            "repUrl": "https://opensource.ncsa.illinois.edu/bitbucket/scm/iarp/extractors.git"
        },
        {
            "repType": "docker",
            "repUrl": "clowder/extractors-csvheaders"
        }
    ],
    "process": {
        "file": [
          "text/csv"
        ]
    },
    "external_services":[],
    "dependencies": [],
    "bibtex": []
}

Converter Info Example

Code Block
languagejs
linenumberstrue
{
    "name": "ImageMagick",
    "version": "1.0",
    "description": "Converts images from one format to another",
    "author": "Kenton McHenry <mchenry@illinois.edu>",
    "contributors": ["Bing Zhang <bing@illinois.edu>"],
    "repository": [
        {
            "repType": "git",
            "repUrl": "opensource.ncsa.illinois.edu/bitbucket/scm/pol/converters-imagemagick.git"
        },
        {
            "repType": "docker",
            "repUrl": "ncsapolyglot/converters-imagemagick"
        }
    ],
    "input_formats": [
          "png", "bmp"
        ],
    "output_formats": [
          "jpeg", "jpg", "pdf"
        ],
    "external_services":[],
    "dependencies": ["ImageMagick"],
    "bibtex": []
}

Current Tools Catalog Example

Code Block
languagejs
linenumberstrue
{
    "_id" : ObjectId("53864aee7400009900ec8d22"),
    "status" : 4,
    "creationDate" : ISODate("2014-05-28T20:45:34.412Z"),
    "updateDate" : ISODate("2016-11-21T04:09:47.836Z"),
    "downloads" : 2,
    "license" : "UIUC/NCSA Open Source License.\r\nhttp://otm.illinois.edu/uiuc_openSource",
    "whatsnew" : "Identify tumors in microscopy images.",
    "compatibility" : "",
    "url" : "https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler/cellprofiler-tumor?raw",
    "sample_input" : "",
    "sample_output" : "",
    "interfaceLevel" : 2,
    "dockerimageName" : "",
    "vmImageName" : "",
    "dockerfile" : "",
    "author" : "ruiliu@illinois.edu",
    "dependencies" : [
           "Cell Profiler"
    ],
    "transformation_type" : "extractor",
    "transformation_id" : "",
    "title" : "Tumors"
}

...