Page History

Versions Compared

Key

This line was added.
This line was removed.
Formatting was changed.

...

gltg-dev
- resources
  - nebula, proxy 2cpu 4ram
  - nebula, postgres (4 CPU, 8G RAM)
- times by source
  - greon .5h
  - iwqis 1h
  - usgs 1h 24m failed presumably do to http timeout
gltg
- resources
  - sd stack, proxy 4cpu 6ram
  - nebula, postgres (4 CPU, 8G RAM)
- times by source
  - greon 6m
  - iwqis 40m
  - usgs 3h 10m
- however, geostreams not responding for long time after parsers ended
ilnlrs-dev
- setup
  - 10m sleep between greon and iwqis, 20m sleep between iwqis and usgs
  - binning
    - 2 workers (not 4)
    - USGS uses 60s sleep after a bin finishes
- times
  - greon 25m (can be improved
  - iwqis 40m
  - usgs 54m
- however, geostreams not responding for long time after parsers ended
- major problem - authentication of cache client timing out
- added a try with 10minute wait if timed out and try again, then another try ans wait 20minutes - still timed out and crashed the parser
- this was with the usgs parser only

GLM Parsing Time

GLM Zooplankton/Phytoplankton ingestion timing for Production server (141.142.211.239):

...