Skip to end of metadata
Go to start of metadata




There are several JIRA issues in the queue between Great Lakes to Gulf and IMLCZO and we need consensus on how to implement them consistently between the projects.

Group Discussion Items

Standardizing how we store QAQC

IMLCZO-75 - Getting issue details... STATUS  Comments in this thread summarize some of the issues.

  1. Should we reformat datapoint properties field to have more structure (each field will have a parameter name, numeric value, qa/qc value) or use streams to divide datapoints in streams based on the qa/qc. 
    1. The first case makes it easier to work with the datapoints on the client side (visualization)
    2. The second case makes it easier to clearly keep the two types separate. See next question.
  2. What happens when we bin?
  3. How to handle different enumeration for qa/qc depending on source
  4. Should we have an API endpoint with metadata about parameters? Centralize definitions by parameters of things like units, qa/qc options
    1. Download 1 csv with raw data and 1 csv file with metadata (similar to census)
  5. In case of Ice/Below Dection Limit, should we threat it as QA/QC or some other entry?
  6. I've created an issue for this discussion:  GEOD-692 - Getting issue details... STATUS


Sensor Coordinate Issues

IMLCZO-82 - Getting issue details... STATUS  this issue would provide support for "anonymizing" a sensor's location. In this case, the client has asked if we can show the sensor as a broader area, without showing exactly where it is located.

  1. Do they want the exact location in the DB stored and anonymize it on the fly until one logs in and downloads the raw data? Or do they want to anonymize the raw data as well?
  2. Remove certain number of significant digits? then show a circle around that point? Maybe put a radius in the config file?
  3. As an alternative to circle, can we let the user specify a polygon and then anonymize 
    1. People complain about seeing a marker when it represents a polygon. Aways show the polygon and not a marker in that case.

IMLCZO-51 - Getting issue details... STATUS  This issue addresses how to handle when a specific sensor is deployed to a different location.

  1. If a sensor is moved to a new location, it should be a new sensor entry

GEOD-474 - Getting issue details... STATUS  This issue was raised because the sensor that was going to be deployed would have created a polygon, not a point.

  1. Geometry on streams can be used to track this

After the Group Discussion

Probably IMLCZO-specific

IMLCZO-67 - Getting issue details... STATUS  setting up Samba on dev and prod

Rob Kooper will install samba client

Luigi Marini find credentials

IMLCZO-55 - Getting issue details... STATUS  deploying extractors to dev and prod

Rob Kooper will setup a clowder vhost and a VM for clowder-extractors

IMLCZO-72 - Getting issue details... STATUS  setting up cron jobs on dev and prod

Brock Angelo Look into selenium headless firefox. For example:

IMLCZO-7 - Getting issue details... STATUS  hearing from Rob about his experience ingesting Ameriflux data

Rob Kooper and Brock Angelo will meet about this.

IMLCZO-60 - Getting issue details... STATUS  Review the remaining formats for Shapefiles

Luigi Marini and Brock Angelo will review this after meeting

IMLCZO-29 - Getting issue details... STATUS  I was able to view this data, but it is 606GB. I don't have that much space on my machine, so might still need a Samba share.

Rob Kooper fixed permissions. Brock Angelo will test it when on wired network.