Skip to end of metadata
Go to start of metadata

Overview

Details on the production of the SVA1 shear catalogs, including the raw measurements, object selection, and many tests of the accuracy of the measured shear estimates, can be found in Jarvis et al. (2015).  Users should see that paper for any questions about the content of the catalogs. Further questions can be addressed to Mike Jarvis, the corresponding author of that paper, whose email address is given therein.  On this page, we merely describe the structure of the files being distributed, including brief descriptions of each of the columns included in the files, often pointing to the relevant sections of Jarvis et al. (2015) for more details.

Table of Contents


Download


Im3Shape Catalog

This catalog includes the shear estimates made using the Im3Shape algorithm, described in Section 7.3 of Jarvis et al, 2015.  

Catalog Description

Column NameData Type (Bytes)Description
COADD_OBJECTS_IDINT (8)Unique object identifier.

E_1
E_2

 

FLOAT (8)
FLOAT (8)
The raw (biased) shear estimator.
NBC_M
NBC_C1
NBC_C2 
FLOAT (8)
FLOAT (8)
FLOAT (8)

The multiplicative bias correction.
The additive bias corrections for E_1 and E_2.

See Section 9.2 of Jarvis et al, 2015 for instructions on how the bias corrections should be applied to an ensemble of galaxies.

Aside: NBC stands for "noise bias correction", although in fact the correction also accounts for model and selection biases.

WFLOAT (8)

The recommended weight to use for each galaxy.

See Section 7.3.4 in Jarvis et al, 2015 for the method used to construct the weights. 

ERROR_FLAG
INFO_FLAG 

INT (4)
INT (4)

Two flags describing potential problems with the shear estimate.

See the following tables for descriptions of what the different bits in the flags mean. Objects with non-zero ERROR_FLAG should not be used for any scientific analysis. Objects with non-zero INFO_FLAG may be acceptable for some use cases, but the conservative selection is to only use objects with INFO_FLAG == 0.

SNR_W
SNR_R 

FLOAT (4)
FLOAT (4)

The standard optimal weight S/N estimate.
The "roundified" S/N estimate.

See Section 7.1 of Jarvis et al, 2015 for an explanation of the differences between these two signal-to-noise estimates. In particular, it is recommended that any selection involving S/N beyond the cuts we have already applied should use SNR_W, not SNR_R. (This is different for ngmix!)

FLUX_RFLOAT (4)

An estimate of the r-band flux (in ADU) that comes out of the same fitting process as was used for the shapes.

This should only be used as a diagnostic. For accurate, calibrated fluxes, users should use the flux values in the Gold catalogs.

RADIUSFLOAT (4)

An estimate of the radius of the galaxy in arcsec.

IS_BULGEINT (4)

1 if the model used was a bulge model, 0 if it was a disc model.

MEAN_RGPP_RP

FLOAT (4)

The mean value of the radius of the observed object divided by the radius of the PSF among the different observations of the galaxy.

The is referred to as R_gp / R_p in Jarvis et al, 2015. It is first introduced in Section 7.3.2.

MEAN_PSF_E1
MEAN_PSF_E2
MEAN_PSF_FWHM 
FLOAT (8)
FLOAT (8)
FLOAT (4)

The mean measured ellipticity and size of the PSF for the different exposures that went into the shear estimates for this galaxy. 

RA_SHIFT
DEC_SHIFT 

FLOAT (4)
FLOAT (4)
The shift in the nominal centroid position of the galaxy model relative to the RA, DEC position from the SVA1 Gold catalog. (arcsec)
CHI2
LIKELIHOOD 
FLOAT (4)
FLOAT (4)
The chi^2 per pixel of the fitted galaxy model.
The likelihood of the fitted galaxy model. 
STAMP_SIZEINT (4)The size (in pixels) of the postage stamp images used for this galaxy.
N_EXPOSUREINT (4)The number of exposures used for constraining the model of the galaxy.

 

Meaning of ERROR_FLAG in Im3Shape Catalog

PositionValueDescription
01Im3Shape failed completely.

1

2Minimizer failed to converge.
24Tiny ellipticity e < 10^-4; Im3Shape fit failed.
38e1 or e2 outside of (-1,1).
416Radius > 20 arcsec.
532Rgp/Rp > 6.
664Negative or nan Rgp/Rp.
7128S/N < 1.
8256chi^2 per effective pixel > 3.
9512Normed residuals < -20 somewhere.
101024Normed residuals > 20 somewhere.
112048RA_SHIFT > 10 arcsec.
124096DEC_SHIFT > 10 arcsec.
138192Failed to measure FWHM of PSF or galaxy.
1416384r-band SExtractor flag has 0x4 or higher.
301073741824 No attempt at a fit was made due to cuts prior to running im3shape.

 

Meaning of INFO_FLAG in Im3Shape Catalog

The order of the flags are such that the larger values are more egregious problems.  This potentially allows a user to cut with something like INFO_FLAG < 512 for instance, to select galaxies that may have some bits at positions 0-8 set, but nothing higher than that.  This is why some flag items are redundant with other items at higher bit positions.

PositionValueDescription
01r-band SExtractor flag 0x1, indicating bright neighbors.

1

2r-band SExtractor flag 0x2, indicating blending.
24Mask fraction > 0.5.
38Model image < -0.01 somewhere.
416Rgp/Rp < 1.15.
532Radius > 5 arcsec.
664Centroid moved more than 0.6 arcsec from nominal.
7128chi^2 per effective pixel > 1.25.
8256Rgp/Rp > 3.5.
9512Normed residuals < -2 somewhere.
101024Normed residuals > 2 somewhere.
112048Declination < 61 degrees S, the limit where we have good photometry.
124096(S/N)_w > 10^5.
138192Radius > 10 arcsec.
1416384(S/N)_w < 10.
15

32768

Model image < -0.05 somewhere.
1665536chi^2 per effective pixel < 0.8.
17131072More than 70% of fitted flux is in masked region.
18262144Model is completely negative.
19524288chi^2 per effective pixel > 2.
201048576PSF FWHM > 10 arcsec
212097152Negative PSF FWHM.
224194304Rgp/Rp < 1.1.
238388608Centroid more than 1 arcsec from nominal.
2416777216Mask fraction > 0.75.
2533554432One or more error flags is set.

 


NGMix Catalog

This catalog includes the shear estimates made using the NGMix algorithm, described in Section 7.4 of Jarvis et al, 2015.  

Catalog Description

Column NameData Type (Bytes)Description
COADD_OBJECTS_IDINT (8)Unique object identifier.

E_1
E_2

 

FLOAT (8)
FLOAT (8)
The raw (biased) shear estimator.
SENS_AVGFLOAT (8)

The sensitivity estimate.

See Section 9.2 of Jarvis et al, 2015 for instructions on how the sensitivity should be applied to an ensemble of galaxies. 

WFLOAT (8)

The recommended weight to use for each galaxy.

See Eqn 7.12 in Jarvis et al, 2015 for the exact definition of this choice.

E_COV_1_1
E_COV_1_2
E_COV_2_1
E_COV_2_2 

FLOAT (8)
FLOAT (8)
FLOAT (8)
FLOAT (8)
The estimated covariance matrix of (E_1, E_2).
ERROR_FLAG

INT (4)

A flag describing potential problems with the shear estimate.

See the following table for descriptions of what the different bits in the flags mean. Only objects with ERROR_FLAG == 0 should be used for any scientific analysis.

SNR_W
SNR_R 
FLOAT (4)
FLOAT (4)

The standard optimal weight S/N estimate.
The "roundified" S/N estimate.

See Section 7.1 of Jarvis et al, 2015 for an explanation of the differences between these two signal-to-noise estimates. In particular, it is recommended that any selection involving S/N beyond the cuts we have already applied should use SNR_R, not SNR_W. (This is different for im3shape!)

FLUX_I
MAG_I 
FLOAT (4)
FLOAT (4)

An estimate of the i-band flux (and corresponding magnitude) that comes out
of the same fitting process as was used for the shapes.

These should only be used as a diagnostic. For accurate, calibrated fluxes, users should use the flux values in the Gold catalogs. 

T
T_ERR
T_R

FLOAT (4)
FLOAT (4)
FLOAT (4)

An estimate of the size, using the T = Ixx + Iyy definition of size, in arcsec^2.
The estimated uncertainty in the estimate of T.
An estimate of the size that the object would have had if it were round.

See Section 8.3.2 in Jarvis et al, 2015 for a description of the difference between these two size estimates. In particular, the formula for T_R is Eqn. 8.4. It is recommended that any selection involving size beyond the cuts we have already applied should use T_R, not T. 

SNR_T
SNR_T_R 
FLOAT (4)
FLOAT (4)

The estimated S/N of T and T_R.

LOG10_SB_IFLOAT (4)

The base 10 logarithm of an estimate of the surface brightness of the object in units of (ADU / arcsec^2). This is a derived value, calculated as log10(abs(FLUX_I / T)).

MEAN_PSF_E1
MEAN_PSF_E2
MEAN_PSF_T 

FLOAT (8)
FLOAT (8)
FLOAT (4)

The mean measured ellipticity and size of the PSF for the different exposures that went into the shear estimates for this galaxy.

SENS_1
SENS_2 
FLOAT (4)
FLOAT (4)

The raw sensitivity estimates for E_1 and E_2.

We recommend using SENS_AVG, which is the average of these two quantities, for both E_1 and E_2, rather than these individual estimates.

ARATEFLOAT (4)Acceptance rate. Galaxies with 0.4 < ARATE < 0.6 are considered good.
STAMP_SIZEINT (4)The size (in pixels) of the postage stamp images used for this galaxy.
MASK_FRACFLOAT (4)The fraction of the pixels in all the postage stamps used for this galaxy that were masked for some reason (neighboring objects, dead pixels, bad columns, cosmic rays, etc.) 

 

Meaning of ERROR_FLAG in NGMix Catalog

PositionValueDescription
01There were no postage stamps for this object.

1

2PSF fitting failed for all epochs.
38Galaxy fitting failed.
416Stamp size was larger than 2048. Galaxy skipped.
664The S/N of the PSF flux was < 4 in all bands.
7128Utter failure of the fitting. For this release, this flag was set when no valid guesses for the fitters could be generated.
9512No attempt was made for round measures because of prior failure.
101024

Round model could not be evaluated within the allowed region of parameter space.

112048Fitting failed when trying to estimate SNR_T_R.
30

1073741824 

No attempt at a fit was made due to other error flags or cuts prior to running ngmix.

 


WL Info Catalog

This catalog is merely for convenience, containing only information derived from other SVA1 catalogs.  Internally, within DES WL, we found it convenient to have the more commonly needed columns from some other catalogs collated here so most weak lensing scientific analyses would only need this file plus one or both shear catalogs.  We provide it here in the same thought that it might be useful for some users.

Catalog Description

Column NameData Type (Bytes)Description
COADD_OBJECTS_IDINT (8)Unique object identifier.

RA
DEC 

FLOAT (8)
FLOAT (8)
The position of the object in the sky. (Both given in degrees.)
MAG_AUTO_G
MAG_AUTO_R
MAG_AUTO_I
MAG_AUTO_Z 
FLOAT (4)
FLOAT (4)
FLOAT (4)
FLOAT (4)

Photometric magnitudes in g,r,i,z, bands. (mag = 99.0 indicates no detection in that band.)

PHOTOZ_BININT (4)

The cosmological photometric redshift bin (0, 1, or 2), as described in DES Collaboration, 2015. (-99 if unable to place in any bin due to other flags.)

MEAN_PHOTOZ

FLOAT (4)A point estimator of the photometric redshift for this object from the SkyNet photo-z catalog. (-99.0 if no point estimate is available.)
SVA1_FLAG

INT (4)

A flag describing potential problems with this object prior to the shear estimation process.

See the following table for descriptions of what the different bits in the flags mean. Only objects with SVA1_FLAG == 0 should be used for any scientific analysis.

IM3SHAPE_FLAGINT (4)

0 if this object has a good im3shape shear estimate. 1 if not.

This flag includes all cuts recommended for the im3shape catalog by Jarvis et al, 2015. cf. Section 9.1 and Appendix B. Using columns from the Im3Shape Catalog, this column is the following derived quantity:

IM3SHAPE_FLAG = ~((ERROR_FLAG == 0) & (INFO_FLAG == 0) & (SNR_W > 15) & (MEAN_RGPP_RP > 1.2))

NGMIX_FLAG INT (4)

0 if this object has a good ngmix shear estimate. 1 if not.

This flag includes all cuts recommended for the ngmix catalog by Jarvis et al, 2015. cf. Section 9.1 and Appendix B. Using columns from the NGMix Catalog, this column is the following derived quantity:

NGMIX_FLAG = ~((ERROR_FLAG == 0) & (0.4 < ARATE < 0.6) & (SENS_AVG > 0) & (SNR_R > 15) & (T_R / T_PSF > 0.15))

 

 

Meaning of SVA1_FLAG in WL Info Catalog

PositionValueDescription
01i-band SExtractor flag has 0x1 set, indicating bright neighbors

1

2i-band SExtractor flag has 0x2 set, indicating blending
24

Modest classification calls this object a star. See Section 2.2 in Jarvis et al, 2015. This corresponds to (bright_test or locus_test) described there.

38Modest classification calls this object junk. See Section 2.2 in Jarvis et al, 2015. This corresponds to faint_psf_test described there.
416In region with a high density of objects with "crazy colors".

“Crazy colors” mean any of the following: g−r < −1, g−r > 4, i−z < −1, or i−z > 4, as defined in Jarvis et al, 2015, Section 2.1

532In region with a high density of objects with large centroid shifts between bandpasses. See Jarvis et al, 2015, Section 2.1 for details.
664Near a 2MASS star with J_M < 12. The mask radius is flux dependent, up to 120 arcmin for the brightest stars. See Jarvis et al, Section 2.1.
7128Large offset in g and i-band windowed positions.
8

256

Object was not measured by NGMix.
9512Likely star according to NGMix T + T_ERR < 0.02 arcsec^2.
101024Very low surface brightness according to NGMix.
112048Object does not satisfy good measurement flags in NGMix.
124096Object does not have a valid magnitude in all g,r,i,z bands. (That is, at least one of them is invalid.) Dropout galaxies tend to produce poor photometric redshift estimates, so we excluded them.

 

Revision History

  • sva1_gold_r1.0_im3shape.fits.gz 

    This version of the im3shape catalog incorrectly had all non-zero INFO_FLAG values set to 33554432 and all non-zero ERROR_FLAG values set to 1073741824.  This was fixed in r1.1.

 


The DES Data Management system is supported by the National Science Foundation under Grant Number (1138766).

             

  • No labels