1
0
-1

I'd like to download single-epoch images (reduced i.e. what goes into the coadds or meds files) from the deep SN fields C3 and X3. I assume I can get a list of file locations using a database query but not quite sure where to start. I'd like a list of all images from Y1 to Y3, and if possible would also like to apply some image-quality flags such as seeing < 1.2". Thanks.

    CommentAdd your comment...

    1 answer

    1.  
      1
      0
      -1

      As a coincidence I just wrote a simple routine that finds all images within a tag.  It was meant as the start of a utility for the wide-field difference imaging effort that persons are going to be trying in the Y5 season.  I have held off adding some bits that would add collection of QA information... and possibly switching to use bind variables (so that it can query lots of positions at once).  There is a telecon in a couple of hours that will likely inform how I update that utility.

      The current versions is available through SVN (I just checked it in).  It is qatoolkit/trunk/bin/find_template_images.py  (and makes use of qatoolkit/trunk/python/qatoolkit/template.py  

      On the other hand if you want a query and you  are confining yourself to the SN fields the query is not too bad.


      select fai.path, i.filename, fai.compression, q.psf_fwhm
      from y3a2_proctag t, y3a2_image i, y3a2_exposure e, y3a2_file_archive_info fai, y3a2_qa_summary q
      where t.tag='Y3A1_FINALCUT'
      	and t.pfw_attempt_id=i.pfw_attempt_id
      	and i.filetype='red_immask'
      	and i.expnum=e.expnum
      	and e.program='supernova'
      	and e.field='SN-C3'
      	and i.pfw_attempt_id=q.pfw_attempt_id
      	and i.filename=fai.filename;
      
      

      That is going to give you a lot of files (>130,000).  So I would guess you want to add constraints like:

      • and q.psf_fwhm<1.2
      • and i.band='r'
      • and i.ccdnum=56

      I think the above will only pick up Y1-Y3 images because early DES observing did not populate the program and field columns in a regimented way.... but it might be safe to add a constraint like:  'and e.nite>20130601'

      You can also join to the y3a2_zeropoint table (i.filename=z.imagename and z.source='FGCM' and z.version='v2.0' and z.flag<16) and obtain the calibration that was the same as that used when making the COADDs.

      The location of the actual files (from the above) is:

      /archive_data/desarchive/{fai.path}/{i.filename}{fai.compression}

      or for http transfer:  

      https://desar2.cosmology.illinois.edu/DESFiles/desarchive/{fai.path}/{i.filename}{fai.compression} 

      In SQL you could form that string within the query as follows:

      'https://desar2.cosmology.illinois.edu/DESFiles/desarchive/'||fai.path||'/'||i.filename||fai.compression


      1. Niall MacCrann

        Fantastic, thanks Robert. Will all these images have undergone Y3 processing and e.g. include brighter-fatter correction, Gary's improved astrometry etc.?

      2. Gary Bernstein

        Niall, new astrometry has not been run on the bulk of the data yet, only on test regions.  When it does it will create aux files that give new solutions for the exposures.

        I wasn't planning on running on the SN exposures (although I have run on all C3-z exposures since that was one of my test sets).  If we put together bright-star catalogs for them all, I can run the codes, so we can talk about this if you want.

      3. Alex Drlica-Wagner

        Just a note for posterity the "actual file location" that Robert mentions:

        /archive_data/desarchive/{fai.path}/{i.filename}{fai.compression}

        is a path that can be accessed through the deslogin machine(s).

      CommentAdd your comment...