2020-02-14 Meeting notes

• Pull Request for Elastic Search 7.5 is open - we were using 2.3
  
  • Difference in how meta data is searched
• PyClowder "3" PR is available now - client based model to make API calls
• Intern Sam - working on script to upload a folder as a dataset into clowder
• New extractor for antivirus scanner - ClamAV

• Attending permafrost meetings - NSF - using clowder to streamline sharing of data related to arctic permafrost
• Have remote sensing tiff files - many large (TB images) - multi band - want to analyze on XSEDE resources
• Clowder - takes input file and makes available as well as taking output files (tiff or shape files with class)
• Dimension of launching execution of code from Clowder on XSEDE
• All data is in a center in Alaska - have to move TB of data every time - how to map and keep track of provenance - stage ahead of time and map to local version?
• Visualization - need input of ideas
• Sandeep: Is it similar to VAT?  related to ECSS - Clowder won't run on ECSS resources - ...
• Used Google Earth Engine- run geospatial code on top of data - they have large reference datasets - like Jupyter Notebook with
• Once a month there is a CSSI PI meeting - one of the things that came up was a comment regarding flexibility and customization for domains.  having the ability to have plugins that are not in the core that you can deploy outside the core - issue with versioning and plugins in the wild

writing tests using Postman to test API - will be doing a lightning talk next week

not free as a collaboration tool

Migration

ability to migrate from one cluster to another

Migration - machines on SD3 in AWS

Can't upload 4gb files through - sd3 doesn't even start

ls in temp folder - bytes show different - transfer stopping before all file gets there? firewall timeout?