Uploaded image for project: 'Clowder'
  1. Clowder
  2. CATS-42

The Create Datapoint endpoint does not properly parse 12 Noon

XMLWordPrintableJSON

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocking Blocking
    • Old Issues
    • None
    • API, GeoStreams
    • None

      If you POST a datapoint that has a start_time or end_time of 12 noon (regardless of its time zone offset) Clowder treats it as midnight.

      If found this while working on IMLCZO-72 where I need to ensure that I'm not re-ingesting duplicate datapoints at particular times.

      Here is an example of the JSON I'm POSTing. Note that the start_time and end_time are between 12:00:00 PM and 12:59:59 PM:

      {
        "geometry": {
          "type": "Point",
          "coordinates": [
            -88.4449935793094,
            40.1712060862677,
            0
          ]
        },
        "start_time": "2015-05-21T12:44:00-05:00",
        "stream_id": "54",
        "end_time": "2015-05-21T12:44:00-05:00",
        "type": "Feature",
        "properties": {
          "relative-humidity": 0.4926948283435387,
          "leaf-wetness-minutes": 0,
          "wind-speed-mps": 6.036,
          "solar-radiation-wm2": 994.2626953125,
          "precipitation-mm": 0,
          "wind-direction": 304,
          "counts": 441,
          "temperature-c": 17.1,
          "gusting-wind-speed-mps": 6.036
        }
      }

      I've tested with different time-zone offsets, and each one treats this as midnight instead of noon. 12:00:00 PM -05:00 should be 17:00:00Z, but becomes 05:00:00. 12:00:00-09:00 should be 21:00:00 but becomes 09:00:00.

      You can verify this by looking at one of our datapoint streams and then searching for 17:00:00. It does not exist in the link below, but there are double instances of 05:00:00. These aren't duplicate datapoints, as you'll see they have different values for each parameter. One was for 12 noon and one was for 12 midnight.

      http://gltg-dev.ncsa.illinois.edu/clowder/api/geostreams/datapoints?stream_id=2

      The timestamp stored in the Postgres database is incorrect. If you look at the attached screenshot of my Postgres table, you'll see that the last row where I changed the time to "11:44:00-05:00" is correct, and the rows above it (at varying time zone offsets) are incorrect.

      So it seems like it would be related to this line in PostgresPlugin.scala:
      https://opensource.ncsa.illinois.edu/stash/projects/CATS/repos/clowder/browse/app/services/PostgresPlugin.scala#57

      It looks like we cast the "start" parameter as java.util.Date, then call java.sql.Timestamp.setTimestamp() on the date object. I'm not sure if I've narrowed it down to the right spot, but it seems like we could confirm that by parsing to a java.util.Date object using the format we pass in "2015-05-21T12:44:00-05:00" and then see what timestamp we get.

              kooper Rob Kooper
              brock Brock Angelo (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: