Creating the Database

PyFluxPro initially reads meteorological and flux data from a two page excel sheet. It converts the excel sheet into a netCDF file, which is one of the more common file formats used in flux data processing, and the format that PyFluxPro uses.

  1. Create an excel file placed and titled to match its reference in the corresponding L1 control file, roughly titled “site_year_L1_today’s date[yyyymmdd]”. The L1 control file will call up this excel sheet, so it must be titled properly.
    1. Create two separate sheets, ‘Met_data_30’ (meteorological data), and ‘full_output’ (Flux output from EddyPro).
      1. ‘full_output’ uses the ‘full_output’ excel sheet created by EddyPro.
      2. ‘Met_Data_30’ uses meteorological data from the weather stations. This database was created and properly formatted during the preparatory steps toward EddyPro flux processing.
    2. Copy the Master Meteorological spreadsheet into ‘Met_Data_30’. It will be pre-formatted if all steps were followed earlier.
    3. The ‘full_output’ generated by EddyPro requires a small amount of formatting for PyFluxPro
      1. The actual data in ‘Met_data_30’ starts on Row 5, after 4 rows of column headings. ‘full_output’ needs to match, so add a new blank row of cells after Row 3 such that the actual data in ‘full_output’ begin on Row 5 as well
      2. Create a new date/time column at the very front of the sheet titled ‘TIMESTAMP’ in Row 2 with units ‘yyyy/mm/dd HH:MM’ in Row 3. In the new column add date and time together, “=[date yyyy-mm-dd]+[time HH:MM]”, to create a single timestamp appearing as ‘yyyy/mm/dd HH:MM’
      3. Convert sonic_temperature from Kelvin to Celsius “=[value]-273.15”. Name new column ‘sonic_temperature_C’, and state units as ‘[C]’
      4. Convert ‘air_pressure’ from Pa to kPa “=[value]/1000”. Name new column ‘air_pressure_kPa’, and state units as ‘[kPa]’
      5. Time points will probably be missing, so we need to perform time-series integrity checks. For a non-leap-year years there should be 17524 lines of 30 minute data, and for leap years there should be 17572 lines.
      6. First, make sure the initial time point is “yyyy-01-01 00:00”, and the last one is “yyyy-12-31 23:30”. If not, add a blank data row with that time point in the ‘TIMESTAMP’
        1. To locate missing time periods,
          1. Make a new column with the formula “=1440*([current date]-[previous date])”. If the measurement duration is every 30 minutes, the formula should return “30”
            1. Graph, or filter the column. If anything but 30 appears, then there are problems with the time series.

As before, use ‘Insert Dates’ from ‘ofTools’ to fill missing dates. Select all the TIMESTAMP values, set 30 step and click ok, dates should be inserted.  Then re-compute the 30 minute formula for all rows. Repeat if more missing dates remain

  • No labels