This document shows how to generate list of sensors using the Geostreaming API. Included are links to code and the output csv, as well as, a data dictionary.
To create a list of sensors, use the following code: https://opensource.ncsa.illinois.edu/bitbucket/projects/GEOD/repos/gltg-parsers-py/browse/scripts/sensors_to_csv
Download the output CSV file (https://uofi.app.box.com/file/593176055855)
...
The output CSV is generated by parsing the JSON response from API. Example of JSON follows.
Code Block | ||||
---|---|---|---|---|
| ||||
{‘created': '2018-03-23T15:48:32Z',
'geoType': 'Feature',
'geometry': {'coordinates': [-84.44799549, 38.9203417, 0],
'type': 'Point'},
'id': 1445,
'max_end_time': '2019-12-18T21:30:00Z',
'min_start_time': '2007-10-01T06:00:00Z',
'name': '03254520',
'parameters': ['discharge-ft3s',
'discharge-ft3s-qc',
'dissolved-oxygen-mgl',
'dissolved-oxygen-mgl-qc',
'nitrate-nitrite-as-n-mgl',
'nitrate-nitrite-as-n-mgl-qc',
'pH',
'pH-qc',
'specific-conductance-uScm',
'specific-conductance-uScm-qc',
'turbidity-fnu',
'turbidity-fnu-qc',
'water-temperature-c',
'water-temperature-c-qc'],
'properties': {'huc': {'huc2': {'code': '05'},
'huc4': {'code': '0510'},
'huc6': {'code': '051001'},
'huc8': {'code': '05100101'},
'huc_name': 'Licking'},
'location': 'LICKING RIVER AT HWY 536 NEAR ALEXANDRIA, KY',
'name': '03254520',
'online_status': 'online',
'popupContent': '03254520',
'region': '0510',
'type': {'id': 'usgs',
'network': 'NWIS',
'title': 'United States Geological Survey'}}}
|
Column Definitions
column name | definition |
---|---|
source | Name of Institution that provides the data. |
name | Name of sensor provided by source. If no name is provided by the source, naming convention applied in parser. |
location | Description of the location of the sensor provided by source. |
max_end_time | Timestamp of most recent datapoint. |
min_start_time | Timestamp of first chronological datapoint. |
parameters | List of measurements collected by the sensor e.g., nitrate, phosphate, discharge. |
huc8 | Hydrological Unit Code at subbasin level. |
huc_name | Name of HUC. |
online_status | Typical available for continuous data. If the sensor data has been updated within the near past (~5 days), the sensor is designated 'online'. If there is no recent data, the sensor will be designated 'offline'. This is viewable in the UI. |
Example Columns and Row
source | name | location | longitude | latitude | max_end_time | min_start_time | parameters | huc8 | huc_name | online_status |
---|---|---|---|---|---|---|---|---|---|---|
United States Geological Survey | 3254520 | LICKING RIVER AT HWY 536 NEAR ALEXANDRIA, KY | -84.447995 | 38.9203417 | 2019-12-18T21:30:00Z | 2007-10-01T06:00:00Z | discharge-ft3s, dissolved-oxygen-mgl, nitrate-nitrite-as-n-mgl, pH, specific-conductance-uScm, turbidity-fnu, water-temperature-c, | 5100101 | Licking | online |
Issues with Data
Source | Issue | Pertinence? |
---|---|---|
USGS | A few locations show id instead of text | |
Water Quality Portal | many missing start and end time (update?) many missing parameters | |
several sources | missing location | |
several sources | missing huc_name | |
several sources | missing online status |