Versions Compared

Old Version 48

changes.mady.by.user Kenton McHenry

Saved on

compared with

New Version 49

changes.mady.by.user Craig Willis

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • leverage existing Labs/Kubernetes API for authentication and container orchestration / access across remote sites
    • etcd.go / kube.go can likely take care of talking to the necessary APIs for us, maybe needing some slight modification
    • possibly extend Labs apiserver to include the functionality of delegating jobs to tmpnb and/or ToolManager agents?
    • this leaves an open questions: single geodistributed kubernetes cluster? or one kubernetes cluster per site, federated across all sites ("ubernetes")

Notes from 10/28 meeting

Present: Mike, David, Kenton, Kandace, Kacper, Craig

  • Kacper explained more of the Whole Tale design:
    • There will be a website where the user can enter the DOI, DOI will resolve to remote repository (e.g., DataOne). Ingest will only happen at that point (on-demand)
    • When the data can't be moved, if compute near the data it will be used
    • Need to support composing multiple datasets – e.g., DarkSky + some smaller dataset. In this case, the smaller data will be moved to site with the large dataset.
  • Might look into Agave project for capabilities API (long-term)
  • Specific comments about SC16 demo:
    • folderId requirement in volman can be removed – just hardcode the mountpoint. So can the userId requirement.
    • tmpnb can be used to simply create a temporary notebook – not tied to a user/folder
    • The folderId is useful when the user want's access to a subset/subdirectory
    • tmpnb is nothing new, so alone this isn't much of a demo.
    • DarkSky data is NFS mountable read
    • Regarding Norman dataset, Girder does support Swift for ingest, but need to test it.
    • Girder supports Oauth, if useful
  • There is now a presentation on November 16th
  • Next steps – for the demo, we will used the "Federated" model above, but long term there's still much to discuss
    • Data transfer (MHD)
    • Write the registry API and UI for proof-of-concept
    • Swift problem: Ingest into Girder directly or find out how best to mount Swift into container
    • Create VM near data at SDSC with Girder stack
    • Example notebooks for 3 datasets

Storyboard for Demo Presentation

...