• S3 Buckets
    • spend some time next week discussing
    • Max will create an issue

Discussion items:

  • Released 1.18 yesterday.  This needs to be advertised.
    Waiting for PR's to be completed.
  • Chen completed a PR and fixed a bug and is working on the CSSI Clowder project.
  • Rob, Max and Mike discovered some issues on SIMPL. See notes under Rob below.
  • Will do a presentation at SD All Staff meeting about All Paws Conference
  • Working on Permafrost and looking at different resources and may use FuncX to solve some issues.  I hope to have two outcomes from this. 1)Submitting to HPC from extractors 2) PARSL and FuncX used and will hide PARSL and make it work seamlessly with Clowder.
  • Discussion about Clowder 2.0
    Discussion about contributing and what's coming in the next year.  Because API's have changed almost everywhere, there is code that needs to be re-written.  Rewrite will take a long time. Some of the libraries are deep and difficult to find.  See link above.
    • Chen suggests that we work on the backend first, then the frontend.  I like using React which can talk to frontend and backend.
    • Rob is working on this workbench. 
    • Without changing Mongo, and keeping as much as possible. 
    • Perhaps as we migrate, we make these changes.  It will be very time consuming, but it's a one time job.
    • Max is excited to do a complete re-write, speaking from doing the patchwork in the past.
    • Luigi suggests team articulate the answers in the wiki and develop a collective solution
Maxnothing to add
Mike L.




nothing to add

Mark Fabsent
Mike B.

absent ;saw Hoang Nguyen mention of clowder2 idea3, w/py &maybe vue front-end, &wanted to share link to our vue front end again

  • SIMPL has 80 K to100K files. If I delete the index, it runs fine, but takes a good deal of time with the index in place.  The solution would be do move the ID.  We ran a replicated database, it crashed the second machine.  Removing the index put zero load on the second machine.  Make a DEV server to see what happens and how it impacts the system.
  • Tried limiting cache memory in Mongo and seems to have solved some of the issues.  Rob will talk to Mike and Michael about moving ID's which might fix this issues.  Max will help with this as well.
  • Fixed a relatively big bug before the new release.  Extractors that were disabled were ignored.  Looked at global level only.
Michael J

Looking at docker generator in Python.  Will make the repo public.

ChenSee notes under Luigi
  • All Paws videos are uploaded in sections and as a whole.  Email was sent to all who registered with links to the videos.  We would like to put this on the clowder framework website as well.
    Katie suggests we create an Announce page and create a new mailing list for newsletters only.  Discussion of webtools functions.
  • Will be working on the annual report and would like feedback and examples.  Get information from papers published, and check with Klara Narhrstedt.  Ask co-PI's for updates.

Nothing to add

Action Items/To Dos: