Design Topics

This page includes a list of topics to be discussed with the team to find good solutions. Feel free to add/remove from it.

• Design/Implement preferred queues for projects like TERRA
  • Can leverage of https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/scripts/browse/extractor-filter
  • Currently running on ncsa.file.digest, ncsa.image.preview and ncsa.image.metadata
  • Currently running on production
  • Configured using filters.json (might need to be extended)
• Registration page post to LDAP or redirect to LDAP form?
• Google Maps/Street View

• Users provide their own Google API Keys (need to figure out details)
• how do we review the pull request? We have a API key for internal testing
• Currently temporarily replacing key in extractor for student routes
• Trying to see what we can do without API key
• Box integration
  • Box Skills (beta)
    • Register Skill (DES Galaxy Recognizer)
    • Sends a JSON body to clusterman
      • fileid (unique identifier), read token (download file), write token (upload metadata to box)
    • bdbox.ndslabs.org runs clusterman/rabbitmq/mongo/extractors
      • clusterman creates a variation of the Clowder extraction message
      • extractor is a branch of pyclowder that reads/writes to box and calls Mattias ML code

• Propagation of Fence credentials to Clowder

• If an extractor creates derived file/dataset they cannot access them
• Depends on Clowder 2.0 and Silhouette implementation
• Working on propagating User API Keys to extractors
• Need to figure out how to match fence users to clowder users and keys

• User authentication for getting API access token
  • Should getting a token require anything besides API key?
  • Sandeep Puthanveetil Satheesan looking at how other services do it
• Adoption of Datasets/Spaces in the API to leverage extraction by space
• Full docker stack deployment / writing new extractors/converters
  • Ben has in NDS workbench (Benjamin Galewsky where is this code?, can you paste the link here)
• Defining parameters for extractors in extractor-info.json
  • so that clients can provide interface/support for which parameters can be set
  • required/available
• Using binder repoToDocker as alternative to docker files for creating extractors
• Tool Catalog user experience
  • Discuss internally ways to improve UX
  • Can we do an informal user study? Provide tasks and see where users get lost