2016-02-08

Discussion of workflow:

  • A ticket is owned by Mike (implementation + documentation)
  • Create sub-task for unit testing, assign to Craig
  • Once testing complete, marked as resolved, pull request is issued
  • Create sub-task for pull-request
  • Once pull-request merged, close tickets.

Prioritization of high-level tasks (will add tickets to JIRA with more detail, then provide estimates)

  1. Kubernetes cluster
    1. On Openstack
      1. NDS-98 - Getting issue details... STATUS : Deployment of Kubernetes cluster on multiple VMs on OpenStack
        1. New Task: Testing (due 2/?)
        2. New Task: Update labs portal documentation (assigned to Mike, 2/?)
    2. Development stack– NDS Dev (David + Mike ~?) (single node)
      1. NDS-82 - Getting issue details... STATUS : Allows anyone to bring up Kubernetes
        1. New tasks: Testing (due 2/9- Craig), documentation (done - Mike)
  2. NDS Ops services (the thing that let's you deploy #3)
    1. NDS-92 - Getting issue details... STATUS : Set of containers (logging, monitor, alert) = ELK
      1. New task: Testing (needs to be done as of 2/8), documentation (done)
    2. Ability to scale it up and down (David)
      1. New task: Allows anyone (NDS Labs user) to scale up their cluster if they need to – adding more OpenStack resources. (Not for NDS5)
  3. Demo #1 (TERRA) 
    1. NDS-105 - Getting issue details... STATUS : Elasticsearch + Jupyter + Terratoolsrv
    2. Scale (elasticity, sharding mongo, sharding rabbit) see how it fits
    3. Script/documentation of what the demo is (done by Kenton in diagrams, but we need to review)
  4. Demo #2 (Dataverse)
    1. NDS-103 - Getting issue details... STATUS : Adding DataVerse support
    2. Script/documentation of what the demo is (review Kenton's diagrams)
    3. Open Question: How can we make this more compelling?
  5. Backend APIs
    1. Administrative API (provision resources, new namespace)
    2. Project user API
    3. NDS-108 - Getting issue details... STATUS : API/CLI prototype
  6. Deployment GUIs
    1. NDS-93 - Getting issue details... STATUS 1 day prototype/proof of concept (Mike)
    2. Actual GUI (est TBD)
  7. Do the TERRA demo validation (end-to-end walkthrough, Dev + OpenStack envs)
  8. Do the DataVerse demo validation (end-to-end walkthrough)
  9. Save it and make it a container, add it to the services (projects?)
  10. Workshop preparation (presentations)

 

 

2016-01-21 

Notes from discussion about tasks for upcoming NDS5 workshop.

Prioritization of high-level tasks (will add tickets to JIRA with more detail, then provide estimates):

  1. Kubernetes cluster
    1. on Openstack (99%) (David ~4hrs)  (multi node? need to confirm)
      1. Need to clean up existing tickets.  
      2. New Task: Easy deployment of cluster on openstack. Allows anyone who wants to bring up a project to boot up a Kubernetes cluster on OpenStack (nebula only) (due 1/29)
      3. New Task: Testing (due 2/5)
      4. New Task: Update labs portal documentation (assigned to Mike, due 2/5)
    2. Development stack– NDS Dev (David + Mike ~?) (single node? need to confirm)
      1. NDS-82, 80%: Allows anyone to bring up Kubernetes
      2. Two parts:  
        1. getting our devstack sorted out, how people will use ndsdev – 1/29
        2. Clowder under Kubernetes (95% done with the configuration) – will be done by 1/29. Outcome: kub config (yaml) that brings up an instance of vanilla Clowder that works end-to-end.
      3. New tasks: Testing (due 2/5- Craig), documentation (due 2/5 - Mike)
  2. NDS Ops services (the thing that let's you deploy #3)
    1. NDS-92: Set of containers (logging, monitor, alert) = ELK? (David + Craig) - (1/29)
      1. New task: Testing, documentation
    2. Ability to scale it up and down (David)
      1. New task: Allows anyone (NDS Labs user) to scale up their cluster if they need to – adding more OpenStack resources. (Not for NDS5)
  3. Demo #1 (TERRA)
    1. Write additional pod specifications 
      1. identify which ones
    2. Put those all together + glue
      1. More containerizing for Clowder
      2. Scale (elasticity, sharding mongo, sharding rabbit) see how it fits
  4. Deployment GUI
    1. 1 day prototype/proof of concept (Mike)
    2. Actual GUI (est TBD)
  5. Do the TERRA demo validation (end-to-end walkthrough, Dev + OpenStack envs)
  6. Save it and make it a container, add it to the services (projects?)
  7. Demo #2 (MDF)
    1. MDF
      1. Workout with Globus
    2. Metasearch
      1. Flesh this out (Craig)
      2. Write the pod specifications
      3. Put those all together + glue
  8. Workshop preparation (presentations)

 

There was a discussion of different services. 

Question was asked whether the services list contains high-level component (e.g., clowder) or everything (e.g., clowder, rabbit, mongo, elastic). Answer: All.

Discussion of Kubernetes -- two steps: 1 deploy, 2 scale/replication, 

Things to discuss later/

  • Implementation that spans Nebula and Amazon? Once you get to #4 – all you need is the account. Idea: Terra is running on Nebula, migrates to Google Cloud?
  • Is storage more important?

 

  • No labels