-
Task
-
Resolution: Fixed
-
Normal
-
None
-
None
-
None
-
None
-
IMLCZO 2019 April, IMLCZO 2019 May
This task may be removed depending upon conversations
Expected Workflow (from notes 05.06.2019):
- Hourly files come from LoggerNet on Linux
- Cron checks for new files every hour
- As parsing each of these files, could then concatenate the new ones with all the existing for the year to a new file EVERY HOUR
- Upload that growing file to Clowder to a RAW folder in a new Dataset EVERY DAY
- Annually, place the full year file in to the PROCESSED folder in the same Dataset EVERY YEAR, and then delete the RAW files for that year if desired
- (Can do this year from start point if from start of year is not feasible)