2015-05-18 Stand Up Meeting notes

Continue work on the extractors for Geodashboard.

Meet with SeaGrant to discuss the design plans for the upcoming work.

Test ingestion of data for GLTG

• Met with SeaGrant and met Brandon, who will be doing Design and assessing the project.
• Stopping work on the Geodashboard extractor-manager at Luigi's request.
• Lots of helpful meetings with Jason and Project Leads to discuss direction of our projects.
• Lots of JIRA cleanup and task reprioritization.

Edgar F. Black

• PEcan
  • meet with Rob to set objectives for the month
  • start working on the objectives
• MSC
  • keep exploring data from the Chicago team 
  • study the possibility of "decision trees" to discover interesting relationship in the data

• PEcAn
  • Account on Taub operative
  • PEcAn in local machine operative
  • local runs of Sipnet model accomplished
  • Installation of Sipnet model in taub cluster under way.

• Generate tasks for items that were discussed last week at the all-hands and regular meetings
  
  • Cleaning up the view mode elements in the various resource lists
  • Unify behavior of resource lists
  • Update user lists in the dialog to add/remove users from a space
  • Filter all datasets and collections correctly so that creator can always view, and otherwise view is governed by assignment to space.
• Work on the "cleaning up the view mode elements..." task
• Estimate and begin work on the "Filter all datasets and collections..." task

• Tasks generated for outstanding items
• Work done for "cleaning up the view mode elements"
• Pull request generated, comments and discussion followed with a decision to do some further cleanup and standardize behavior with respect to maintaining the parameter
• Updates made with respect to the decisions
• Estimates done for tasks and elements moved over to the sprint

• PEcAn
  • Get Edgar up and running
  • Finish proposal
• SEAD
  • Filtering datasets/collections by space
• Clowder
  • Cleanup interface
  • Finish CS492 branch
  • Merge develop to master and deploy
• Might take some time off

• PEcAn
  • Edgar has PEcAn up and running and ready to do remote runs
  • Send in changes, need to check up on final version
• SEAD
  • Create pull request for filtering datasets/collections by space
• Clowder
  • Little UI cleanups (move search, change login/admin icons)

• CyberSEES
  • Working on defining the possible tasks by Sep demo
• NIST-CORE
  • Administrations such as budgets and proj management
  • Supporting collecting User Story
  • Attending task meetings to provide advice/comments regarding Ergo platform
  • Helping new hire in terms of technology
• KISTI
  • Revising Joint Lab agreements
  • Preparing August meeting at NCSA
• BrownDog
  • Arranging a meeting with Praveen's group before Smruti Padhy's leave

• CyberSEES
  • Working on defining the possible tasks by Sep demo -> It needs to be continued on the week of 5/26
• NIST-CORE
  • Administrations such as budgets and proj management -> It needs to be continued on the week of 5/26
  • Supporting collecting User Story -> It needs to be continued on the week of 5/26
  • Attending task meetings to provide advice/comments regarding Ergo platform -> It needs to be continued on the week of 5/26
  • Helping new hire in terms of technology -> It needs to be continued on the week of 5/26
• KISTI
  • Revising Joint Lab agreements -> It needs to be continued on the week of 5/26
  • Preparing August meeting at NCSA -> It needs to be continued on the week of 5/26
• BrownDog
  • Arranging a meeting with Praveen's group before Smruti Padhy's leave: Smruti Padhy sent out an email; need to follow up on the week of 5/26

• Earthcube: semantic annotation; prepare for next week's trip and demo.
• BD: Continue working on SC15 poster.

• Earthcube:
  • Semantic annotation feature done (dev + testing).
    Tested on both localhost and a remote machine (ecgs-dev). Checked in the code in both Clowder and GSIS. Created a pull request for GSIS. In addition to suggesting CSDMS standard names, it now automatically pulls fields for the chosen name and adds it into the user metadata. The pulled-in fields are configurable, not just hardcoded.
  • Rob and I increased the Earthcube VM's memory for next week's demo.
    ecgs-dev had 1-CPU, 1 GB memory, insufficient as a server running many applications, slow in testing the autocompletion feature. Asked Rob to help increase the resource (ISDA-155), which he promptly helped. Now more responsive.
  • Mostafa requested to write and run scripts to add 2500+ CSDMS standard name wiki pages into the Wiki (ECGS-14).
    Did that, twice. Each time: 2573 wiki pages, 7.5 hours. The second time added a "CSDMS:" prefix for each page, to avoid future name collisions. Tuned parameters to make it faster.
  • Other prep work, such as turning nginx on, mysql and geodashboard off, and make it persistent.
  • At Luigi's suggestion, added several JIRA tasks and filled in most of them to document the work done (ECGS-10, ECGS-12, ECGS-13, ECGS-14).
• BD:
  • Did TOI with Smruti before next week's travel and the vacation afterwards. Showed her:
    • how to use OpenStack CLI or Web UI to monitor the ISL cloud, start/delete/take a snapshot of/ssh into a VM.
    • how to update a VM image and use it with the elasticity code;
    • start/stop the elasticity code, for both DTS and DAP;
    • how to generate high request rate to test elasticity, gave her the scripts and input files;
    • how to get data and generate graphs for the SC15 poster, gave her the scripts to retrieve the queue data from MongoDB, to generate Gnuplot scripts, and to generate graphs for the poster.
  • REDSC15 poster, didn't work on this.

• Out of the office

• Earthcube: improvements to metadata widgets and testing of geosemantics service
• Clowder documentation
• Clowder jira/stash/wiki/mailing lists/etc.
• IML-CZO release plan
• Medici merges
• medici-play master release
• Brown Dog API cleanup

• YELLOW Budgets
• YELLOW Earthcube: improvements to metadata widgets and testing of geosemantics service. All hand preparations
• RED Clowder documentation
• RED Clowder jira/stash/wiki/mailing lists/etc.
• YELLOW IML-CZO release plan
• GREEN Medici merges
• GREEN medici-play master release
• RED Brown Dog API cleanup

• Browndog - finish DW tool editor
• NIST - assist with new hire, provide sample data/fragility for Bruce, open issues against Ergo for version 1, look into tornado matlab code
• MSR - NFIE meeting
• CyberSEES - assist Indira with any rhessys-js questions, talk with Jong to see next steps on architecture discussion

• Browndog - cleaned up web editor code for editing DW tools, found another small bug that I need to fix when creating workflows with the new tools, should finish this today or early next week
• NIST - helped Yong Wook with workspace issues, had discussion via email on tornado algorithm and documented on wiki, installed/ran matlab code from researcher for Tornado paths, test/opened Ergo issues to fix for version 1, sent wiki links to John et al covering Ergo fragility curve data format, developer meeting
• MSR - attended NFIE meeting, a workshop is coming up, if there are scaling issues running DW on Azure I may need to modify VM size.
• CyberSEES - bi-weekly meeting, waiting for Sammy to setup meeting to discuss IUHM model which will be ran using DataWolf.

1. Continue with the Management Tool, MWRD-114, MWRD-116, work on functionality - workflow management and return fields, errors and text
2. Add storm backflow column to the Storm table (MWRD-149)
3. Continue with Geoserver and OpenLayers code (queries)

1. Some progress, not much
2. DONE in the db, it still needs the Python Storm code update in order to populate the storm table.
3. Changed time t parameter from CDT to UTC in Geoserver and js in two layers (surprise, it should have been UTC)

• eScience Poster Submission
• IEEE Big Data Paper - DTS Section, Tools Catalogue
• VM Elasticity hands on with help from Rui
• Friday off

• Poster Submitted to eScience 2015.
• Started assembling the sections for IEEE Big Data Paper: Reading old reports and focussing on DTS section right now
• Received VM Elasticity module hands on from Rui Liu
• Sent an email to Praveen's group for arranging meeting to decide on plan for summer
• Modified and tested the program provided by Praveen's student for converting into a DTS. The program is not running and showing error. It seems the arcpy script imported from model builder in ArcGIS has not correctly converted it to the script. It needs further investigation to make the program working. Sent an email to Jong Lee about the status.
• Friday off
  
  

• Work on running extractors on VAT Gateway machine
• Continue with work on person tracking visualization

• Tried fixing issues with extractors on Gateway machine. Not completed.
• Programmatically obtained video height, width, and frame rate and added them to person tracking metadata through extractor
• Tested the above feature

• Work on importing and setting up IntelliJ IDE
• Resolve build issue that I am having with 2 Thumbnail endpoint routes
  • GET /api/thumbnail/list
  • DELETE /api/thumbnail/:id
• Complete thumbnail cleanup test cases

• Completed setup on IntelliJ environment
• Resolved build issue with optional parameters not Mapping with stringify
• Wrote test cases to test 2 new Thumbnail end point routes
• Thumbnail cleanup of file by API Test Suite is complete (feature/MMDB-1845)

• Keep working on DownloadAs button, use existing libraries to make a post request to Polyglot.
• If time left - get back to Segmentor code, try to use exisitng libraries for it as well, instead of kgm/Utilities package.

• Able to make a multipart post request to Polyglot and get back the url of the converted file. Check is the file exists at the url - still work in progress.

• Vacation May 21-22
• Continue testing/improving running of GLTG data parsers on gltg-dev

• GLTG:
  • Cron on gltg-dev now works for USGS - fixed multiple bugs
• BrownDog
  • Studying Theano Python library for convolution networks applied to computer vision
  • Meeting with GI students - using Clowder
• Farm Security Images
  • Looked at Library of congress API and started Python script for getting images
  • Phone meeting with Paul Rodriguez
  • Phone meeting with Jeff, Alan, and Paul
    • Discussed first priorities, resources, and workplan

SEAD

• Adding Permissions to new roles.

CyberSees

• Learning Backbone.js
• Use backbone to read the datasets from Datawolf in rhessys.js

SEAD

• Added Permissions to new roles

CyberSees

• Backbone Tutorials.

• Kick off next 8 weeks for Brown Dog
• Book MWRDGC monthly meeting
• GLTG JIRA transition from 1.0 to 1.1
• Onboarding for Yong Wook
• More NIST project planning

• Started 3.1 Brown Dog milestone
• Several MWRD meetings to discuss possible additional work
• GLTG tasks re-prioritized, followed up with external team members on some dependancies
• Yong Wook good to go
• Added NIST-V1 and NIST-V2 versioning to ERGO for NIST-CORE issue tracking
• Lots of wiki updates across all projects

• Work on MMDB-1859.

• Resolved MMDB-1859 (limit display of Datasets to only the latest 10 on Space page), and issued a pull request.

• Finish watching all the ISDA tutorial videos
• ERGO Tutorial
• Finish onboarding process

• Finished onboarding process
• Finished watching ISDA tutorail videos
• Set up working environment for ERGO and start checking the code
• ERGO Stash SSH had a problem with my account and should be fixed
• Started ERGO tutorial (had MAEVIZ materials, should be switched to ERGO material?)