Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Agenda:

Version 2.0

Discussion:

Elizabeth Yanello Open meeting to everyone - advertise

Elizabeth Yanello move notes and agenda to GitHub. Request agenda items for Clowder Dev early in the week.

We will advertise as a developer meeting and own the agenda; but give people an opportunity to show what they are using Clowder for, perhaps in the last half hour.

For the time being, the dev meetings will focus on 2.0

Luigi was able to fix the current template.

Please offer suggestions and feedback for our current webpage.  Let's post more often.

We hope we can bring in new hires to take the place of people who are coming off of Clowder.

Open 2.0 repositories to public.  There are projects that are interested in using Clowder.

Go to Clowder frameowrk and login othewise you won't show up.

Mike L. shared the following link: https://github.com/orgs/clowder-framework/people
Hackathons have been productive.  Let's continue to use these to work out the continued development of 2.0.

Review of tickets opened in issues.

Discussion of funding. Will discuss with Kenton.

Luigi will go through list of bugs and prioritize them.

Abstract submitted by Chen; will start to work on poster for upcoming conference.

Looking for best practices for libraries.




WhoNotes
Luigi
Maxno updates
Mike L.no updates
Todd

Pull requests for extractors-geo - can someone check this? It's causing errors for some projects.

Pull request for 2.0 - upload multiple files.

Bingno updates
Mike B.

no updates

RobBug (question) in clowder where if you double click the create button it makes a two datasets (and maybe others)
Kastan
  •  PDG colab: They need data pipelines for ML & ML training pipelines (see post-meeting email for details). My concern: Codeflare satisfies only a small part of their needs, they primarily need data ops (like Airflow and Spotify's Luigi). Ray Workflows resolve this conflict.
  •  PDG colab: Recieved data (Commerical via XSEDE, public via Ingmar's Github). 
  •  IN-CORE colab:
They have data, need customized ML. I can help them use Codeflare to get better ML performance.
  •  Ensure Dask worker.distributed for XGBoost works on HAL (currently, workers getting killed by garbage collector)
  •  Talk to HAL admins about Ray support (for IBM Codeflare). Currently unsupported on ppcle.
SandeepChen
  • Recieved labeled data and a plan for creating new data.
  •  Resolve Ray asyncio threading conflict.
  •  PDG: Investigating PDG's current data ingest pipeline to recreate in Ray Workflows. Next deliverable is simple data-loading & pre-processing example.
  •  IN-CORE: Build prototype Ray workflow for satelite image pre-processing.
  •  Add Apple Silicon M1 support for Clowder and Clowder-monitor Docker images via Github Actions flag.
    Image Added
Sandeep
Chen
  • ACSW poster abstract submitted
  • create milestone alpha 2.0 and clean up / create some of the tasks
  • will start to work on error message modal this afternoon
Katie
Lisa

Action Items/To Dos:

...