This document shows how to generate list of sensors using the Geostreaming API. Included are links to code and the output csv, as well as, a data dictionary.
To create a list of sensors, use the following code: https://opensource.ncsa.illinois.edu/bitbucket/projects/GEOD/repos/gltg-parsers-py/browse/scripts/sensors_to_csv
Download the output CSV file (https://uofi.app.box.com/file/593176055855)
...
The output CSV is generated by parsing the JSON response from API. Example of JSON follows.
Code Block | ||||
---|---|---|---|---|
| ||||
{‘created'created': u'2018-03-23T15:48:32Z', u'geoType': u'Feature', u'geometry': {u'coordinates': [-84.44799549, 38.9203417, 0], u'type': u'Point'}, u'id': 1445, u'max_end_time': u'2019-12-18T21:30:00Z', u'min_start_time': u'2007-10-01T06:00:00Z', u'name': u'03254520', u'parameters': [u'discharge-ft3s', u 'discharge-ft3s-qc', u 'dissolved-oxygen-mgl', u 'dissolved-oxygen-mgl-qc', u 'nitrate-nitrite-as-n-mgl', u 'nitrate-nitrite-as-n-mgl-qc', u 'pH', u 'pH-qc', u 'specific-conductance-uScm', u 'specific-conductance-uScm-qc', u 'turbidity-fnu', u 'turbidity-fnu-qc', u 'water-temperature-c', u 'water-temperature-c-qc'], u'properties': {u'huc': {u'huc2': {u'code': u'05'}, u 'huc4': {u'code': u'0510'}, u 'huc6': {u'code': u'051001'}, u 'huc8': {u'code': u'05100101'}, u 'huc_name': u'Licking'}, u 'location': u'LICKING RIVER AT HWY 536 NEAR ALEXANDRIA, KY', u 'name': u'03254520', u 'online_status': u'online', u 'popupContent': u'03254520', u 'region': u'0510', u 'type': {u'id': u'usgs', u 'network': u'NWIS', u 'title': u'United States Geological Survey'}}} |
...
|
Column Definitions
column name | definition |
---|---|
source | Name of Institution that provides the data. |
name | Name of sensor provided by source. If no name is provided by the source, naming convention applied in parser. |
location | Description of the location of the sensor provided by source. |
max_end_time | Timestamp of most recent datapoint. |
min_start_time | Timestamp of first chronological datapoint. |
parameters | List of measurements collected by the sensor e.g., nitrate, phosphate, discharge. |
huc8 | Hydrological Unit Code at subbasin level. |
huc_name | Name of HUC. |
online_status | Typical available for continuous data. If the sensor data has been updated within the near past (~5 days), the sensor is designated 'online'. If there is no recent data, the sensor will be designated 'offline'. This is viewable in the UI. |
Example Columns and Row
source | name | location | longitude | latitude | max_end_time | min_start_time | parameters | huc8 | huc_name | online_status |
---|---|---|---|---|---|---|---|---|---|---|
United States Geological Survey | 3254520 | LICKING RIVER AT HWY 536 NEAR ALEXANDRIA, KY | -84.447995 | 38.9203417 | 2019-12-18T21:30:00Z | 2007-10-01T06:00:00Z | discharge-ft3s, dissolved-oxygen-mgl, nitrate-nitrite-as-n-mgl, pH, specific-conductance-uScm, turbidity-fnu, water-temperature-c, | 5100101 | Licking | online |
Issues with Data
Source | Issue | Pertinence? |
---|---|---|
USGS | A few locations show id instead of text | |
Water Quality Portal | many missing start and end time (update?) many missing parameters | |
several sources | missing location | |
several sources | missing huc_name | |
several sources | missing online status |
...