DECADE public release
List of data products
First, we list the different products made available so far. Each is detailed further below in its own subsection. If you do not see a product of interest in this list, please contact me! Our goal is to have made all useful products available right away.
1. Shear Catalog: The shear catalog is available at Noirlab, and via Globus. If you want to query small subsets, then the Noirlab portal is the best place to go. If you want to use the full catalog (e.g., for correlation studies) the Globus portal is easier. This portal also contains a single HDF5 file (shear_catalog_sparse.hdf) which is a curated subset of all objects relevant for the cosmology analysis alone. This is a significantly smaller subset of the full catalog. The globus portal also contains the redshift distributions for the four bins. The necessary calibrations are found in the Table below, can also be obtained from the paper release here. You will need to apply the shear response yourself.
2. Cosmology Likelihood: The full cosmology likelihood is available in the cosmosis-standard-library. The ini file for the decam13k likelihood is here and is the recommended starting point. This file can run a combination of DECADE NGC, DECADE SGC, and DES Y3. All the necessary data for the likelihood runs are already on cosmosis.
3. Cosmic Shear Datavectors: The associated cosmic shear datavectors, ensemble redshift distributions, and covariance matrices are available in this public folder, and also already in the cosmosis-standard-library. See the ini file mentioned above to see the relative filepath within the CSL library. They are also found in the datavectors and n_of_z folders within the Globus endpoint.
4. DELVE DR3: The superset DELVE DR3 catalog, which contains all detected objects, is available at Noirlab via a queryable database. See also this page for more details on the DELVE data release. The raw coadd images, including a cutout service, are made available via Noirlab.
5. Cosmology Posteriors: The chains from the various results shown in Anbajagane, Chang et. al (2025e) are all available as MCMC chains provided here. See the subsection below for details on what chains are available and what the filenames map to.
6. Balrog (SSI) Catalog: The Balrog catalog is available via Globus. This is used to compute the transfer function, i.e., the probability of selecting an object in our sample given some object properties. The main file of interest is the balrog/matched_catalog.hdf5 file, which contains the injections matched to their noisy, “widefield” realizations.
Shear catalog
Both Noirlab and Globus have the entire base shear catalog, i.e. any and all measurements of shape made for the DECADE coadd detections. This is a significantly larger superset of the fiducial cosmology catalog. The latter catalog is also provided as a single HDF5 file for ease of use.
Shear selection
The cosmology selections can be applied, starting from the sparse catalog, as
with h5py.File('shear_catalog_sparse.hdf5', 'r') as f:
mask = f['MCAL_SEL_NOSHEAR'][:] == bin_ind
where bin_ind = (1, 2, 3, 4). We denote unselected objects with bin_ind = 0.
Shear responses
The compute_response.py file in the scripts directory of the Globus endpoint lists the shear response calculation required. The quick calibration parameters are found below. If you are not using our fiducial selection, these numbers will be different. If you subselect objects based on area, it is mostly fine to use the numbers below. If you subselect on magnitudes and colors, then that is no longer true. We may be able to provide you with recalibrated numbers for simpler selection choices, please reach out!
| Bin | \(m\) [\(10^{-2}\)] | \(\sigma(\langle z\rangle)\) [\(10^{-2}\)] | \(R = (R_{11, \rm tot} + R_{22, \rm tot})/2\) |
|---|---|---|---|
| NGC Bin 1 | \(-0.92 \pm 0.296\) | \(1.63\) | \(0.836\) |
| NGC Bin 2 | \(-1.90 \pm 0.421\) | \(1.39\) | \(0.771\) |
| NGC Bin 3 | \(-4.00 \pm 0.428\) | \(1.01\) | \(0.741\) |
| NGC Bin 4 | \(-3.73 \pm 0.462\) | \(1.17\) | \(0.621\) |
| SGC Bin 1 | \(-1.33 \pm 0.472\) | \(1.61\) | \(0.844\) |
| SGC Bin 2 | \(-2.26 \pm 0.657\) | \(1.40\) | \(0.778\) |
| SGC Bin 3 | \(-3.67 \pm 0.697\) | \(1.00\) | \(0.748\) |
| SGC Bin 4 | \(-5.72 \pm 0.804\) | \(1.16\) | \(0.626\) |
All of the above will compute the average response for a given ensemble. If you want them for individual galaxies, you can follow the script mentioned above, and do
with h5py.File('shear_catalog_sparse.hdf5', 'r') as f:
R11 = (f['MCAL_G_1_1P'][:] - f['MCAL_G_1_1M'][:])/0.02
R22 = (f['MCAL_G_2_2P'][:] - f['MCAL_G_2_2M'][:])/0.02
Note that the above is the response \(R_\gamma\) pertaining to an individual galaxy. The ensemble-averaged response of a sample is \(R_{\rm tot} = \langle R_\gamma \rangle + R_{\rm sel}\). This is the number listed in the table above, and computed in the script. Please see Section 3.1 in the shear catalog paper for more details.
Column descriptions
Here is a description of all the columns in the catalog:
| Column | Description |
|---|---|
| COADD_OBJECT_ID | Unique identifier for the coadded objects |
| DEC | Declination, in degrees |
| DNF_D1 | DNF photo-z photometric directional distance to nearest neighbor. |
| DNF_DE1 | DNF photo-z photometric Euclidean distance to nearest neighbor. |
| DNF_ID1 | DNF photo-z COADD_OBJECT_ID corresponding to the nearest neighbor in the training sample. |
| DNF_Z | DNF photo-z primary point estimate obtained by fitting to the directional neighborhood. |
| DNF_ZN | DNF photo-z nearest neighbor spec-z. Useful for estimating N(z) for a given sample. |
| DNF_ZSIGMA | DNF photo-z uncertainty estimate from photometric uncertainties and residuals from the neighborhood fit |
| DNF_ZERR_FIT | DNF photo-z error term derived from the residuals of the regression fit |
| DNF_ZERR_PARAM | DNF photo-z error term due to magnitude/flux uncertainties propagation |
| DNF_NNEIGHBORS | DNF photo-z number of neighbors used for the DNF_Z estimate |
| FLAGS_FOOTPRINT | Use FLAGS_FOOTPRINT == 1 to select objects inside the standard “Gold” footprint. |
| FLAGS_FOREGROUND | Flag describing whether the object is contained in a region with a foreground astrophysical object; recommend FLAGS_FOREGROUND = 0 for general extragalactic studies. |
| MCAL_FLAGS | use MCAL_FLAGS == 0 for the science sample |
| MCAL_FLUX_(RIZ)_(NOSHEAR, 1P, 1M, 2P, 2M) | Metacal fluxes in r, i, z bands |
| MCAL_FLUX_(RIZ)_(NOSHEAR, 1P, 1M, 2P, 2M)_DERED_SFD98 | extinction-corrected (SFD98) version of metacal fluxes |
| MCAL_FLUX_(RIZ)_ERR_(NOSHEAR, 1P, 1M, 2P, 2M) | The 1sigma error on the metacal fluxes |
| MCAL_FLUX_(RIZ)_ERR_(NOSHEAR, 1P, 1M, 2P, 2M)_DERED_SFD98 | The 1sigma error on the extinction-corrected metacal fluxes |
| MCAL_G_1_(NOSHEAR, 1P, 1M, 2P, 2M) | First component of object ellipticity |
| MCAL_G_2_(NOSHEAR, 1P, 1M, 2P, 2M) | Second component of object ellipticity |
| MCAL_G_COV_0_0_(NOSHEAR, 1P, 1M, 2P, 2M) | Covariance of the G_1 estimate |
| MCAL_G_COV_0_1_(NOSHEAR, 1P, 1M, 2P, 2M) | Cross covariance of G_1 and G2 |
| MCAL_G_COV_1_0_(NOSHEAR, 1P, 1M, 2P, 2M) | Cross covariance of G_2 and G_1 (same as COV_0_1) |
| MCAL_G_COV_1_1_(NOSHEAR, 1P, 1M, 2P, 2M) | Covariance of the G_2 estimate |
| MCAL_W_(NOSHEAR, 1P, 1M, 2P, 2M) | Shear weights used in the cosmology analysis |
| MCAL_PSF_G_1_NOSHEAR | First component of PSF ellipticity |
| MCAL_PSF_G_2_NOSHEAR | Second component of PSF ellipticity |
| MCAL_PSF_T_NOSHEAR | Size of the PSF |
| MCAL_S2N_(NOSHEAR, 1P, 1M, 2P, 2M) | Signal-to-noise of the object |
| MCAL_SEL_(NOSHEAR, 1P, 1M, 2P, 2M) | Object selection for the cosmology sample. If “mcal_sel == 0” the object is not selected. Objects are assigned values of [1, 4] depending on which tomographic redshift bin they are assigned to. |
| MCAL_T_(NOSHEAR, 1P, 1M, 2P, 2M) | Size of the object (after PSF deconvolution) |
| MCAL_T_RATIO_(NOSHEAR, 1P, 1M, 2P, 2M) | Ratio of object and PSF sizes |
| RA | Right ascension |
Note that the catalogs contain redshift point estimates from the “Directional Neighborhood Fitting” (DNF) algorithm. These were not used in our fiducial cosmology analysis, but are provided here as they are useful for cluster lensing and other downstream uses of the public catalog. This redshift was determined using the SOF (single object fitting) fluxes provided in the DELVE DR3 processing.
The metacal-based quantities have five variants. Noshear is the fiducial result, while 1p, 1m, 2p, 2m are variants where the galaxy image was sheared in a specific direction. The latter are used to estimate the shear response. Metacal was only run on riz bands.
Cosmology Posteriors
This table describes the chains provided here. In all analyses, we do not consider the shear ratio measurements from DES.
Table of filenames
| Filename | Description |
|---|---|
| decam13k.txt | The fiducial analysis combining NGC, SGC, and DES Y3. |
| decam10k.txt | Combination of just NGC and DES Y3 |
| decade-ngc.txt | DECADE NGC-only results |
| decade-sgc.txt | DECADE SGC-only results |
| des.txt | Results from DES Y3 (note we never use shear ratio in any of our analyses) |
| decam13k-ext.txt | Combination of DECam 13k with DES Y5 SNe and DESI DR2 BAO |
| decam13k-ext-w0wa.txt | Decam13k + Ext constraints on the w0-wa dynamical dark energy model |
| decam13k-ext-wphi.txt | Decam13k + Ext constraints on the wphi model of Shajib & Frieman 2025 |
| decam13k-bacco.txt | DECam13k analysis using all scales and modelling baryons with the public Bacco emulator |
| decam13k-bcemu.txt | DECam13k analysis using all scales and modelling baryons with the public Bcemu emulator |
| decam13k-Tagn.txt | DECam13k analysis using all scales and modelling baryons with HMCode 2020 |
| decam13k-bfg-bacco.txt | DECam13k analysis using all scales and modelling baryons with BaryonForge-based halo model of Bacco |
| decam13k-bfg-bcemu.txt | DECam13k analysis using all scales and modelling baryons with BaryonForge-based halo model of BCemu |
| decam13k-bfg-bacco-wide.txt | The “BFG-Bacco Wide” constraints from the paper |
| decam13k-Tagn-wide.txt | The “Tagn Wide” constraints from the paper |
Reading out the chains
Here are some utils for reading the chains into a dataframe.
def _get_names(file):
with open(file, 'r') as f:
params = f.readline()[1:-1].split('\t')
p_type = [x.split('--')[0] if '--' in x else 'None' for x in params]
p_name = [x.split('--')[1] if '--' in x else x for x in params]
p_name = [n + '_delve' if 'delve' in t.lower() else n for n, t in zip(p_name, p_type)]
p_name = [n + '_sgc' if 'sgc' in t.lower() else n for n, t in zip(p_name, p_type)]
p_name = [n + '_des' if 'des' in t.lower() else n for n, t in zip(p_name, p_type)]
p_name = [n + '_kids' if 'kids' in t.lower() else n for n, t in zip(p_name, p_type)]
p_name = [n + '_' + t if 'halo_model_parameters' in t else n for n, t in zip(p_name, p_type)]
p_type = ['None' if p.upper() == p else p for p in p_type]
p_name = [n + '_lens' if 'lens' in t else n for t, n in zip(p_type, p_name)]
return p_type, p_name
def load_chain_info(file):
assert os.path.isfile(file), f"The path {file} does not exist"
CHAINS = pd.read_csv(file, comment = '#', names = _get_names(file)[1], sep = r'\t', engine = 'python')
if 'log_weight' in CHAINS.keys(): CHAINS['weight'] = np.exp(CHAINS['log_weight'])
if 'OMEGA_M' in CHAINS.keys(): CHAINS['omega_m'] = CHAINS['OMEGA_M']
CHAINS = CHAINS[np.isfinite(CHAINS['post'])]
CHAINS['S8'] = CHAINS['SIGMA_8'] * (CHAINS['omega_m'] / 0.3)**0.5
return CHAINS
def setup_MCsamples(C, P):
"""
C is the list of chains (as dataframes) and P is the list of param names you want.
If a given param name is not in a given chain, we will drop the param from the call for
that chain
"""
F = []
for c in C:
P_tmp = [p for p in P if p in c.keys()]
F.append(MCSamples(samples = c[P_tmp].to_numpy(), weights = c['weight'].values, names = c[P_tmp].keys()))
for f in F:
for param in f.paramNames.names:
if param.name in labels: param.label = labels[param.name]
return F
Balrog catalog
Balrog is a synthetic source injection catalog, designed to estimate the transfer function of DES and DECADE. It is produced by injecting simulated light profiles into the real images and reprocessing them the same way we do the data. This provides a mapping between input photometry and the output photometry/measurements. The exact method is detailed in Paper II.
Reading out: The catalog can be found in the balrog/matched_catalog.hdf5 dataset. This contains two groups, NGC and SGC. In both cases, we simulated around 1000 coadd tiles, which is about 500 sq. deg, or 10% of the survey. Each group contains a variety of columns, noted below. The sample used for our main calibrations is selected using mask = good_inj. Any injections that are not in good_inj are discarded in our analysis. Selected objects can be found using the selection flag. This includes all area masks as well.
Balrog in DECADE (following DES Y6) uses a new injection scheme, where some objects are injected many more times than others. This lets us boost the statistics of objects more likely to pass our selection cuts and make it into the final selection. All redshift estimation uses the probability, \(p({\rm sel}) = N_{\rm sel} / N_{\rm inj}\), and is therefore agnostic to the differences in \(N_{\rm inj}\) between two objects A and B. Any other uses of Balrog should ensure this preferential injection scheme is accounted for. Please reach out to Dhayaa if you have questions.
Column descriptions
Here is a description of all the columns in the catalog:
| Column | Description |
|---|---|
| d_arcsec | Angular separation between the injected object position and the nearest detected object, in arcseconds. |
| d_contam_arcsec | Angular distance to the second nearest neighbor object, in arcseconds. If this is < 1.5 arcsec, we consider the injection contaminated. |
| dec | Declination of the detected object, in degrees. |
| good_inj | Flag indicating whether this specific injection is good for the cosmology analysis. Injections are removed if they come from deepfield objects that we don’t use (e.g., they only have 4-band and not 8-band flux measurements), or fail the d_contam_arcsec < 1.5 cut. |
| id_y3_deepfield | Unique identifier of the injection associated with the DES Y3 deep field catalog. |
| injection_class | Index identifying which injection scheme the object was placed under. Follows, 0 - uniform/random sampling, 1 - preferentially sample mag_i < 23.5 with sigmoid, 2 - Same as (1) but only inject deepfield objects with redshift estimates. |
| mcal_T_noshear | Size of the object (after PSF deconvolution) |
| mcal_T_ratio_noshear | Ratio of object and PSF sizes |
| mcal_flux_noshear_dered_sfd98 | Flux (in riz) corrected for Galactic extinction using the SFD98. Convert to mag as \(m = 30 - 2.5\\log_{10}f\) |
| mcal_flux_err_noshear_dered_sfd98 | Flux (in riz) corrected for Galactic extinction using the SFD98. |
| mcal_g_noshear | Object ellipticity components |
| mcal_psf_T_noshear | Size of the PSF in arcsec^2 |
| mcal_psf_g_noshear | PSF ellipticity components |
| mcal_s2n_noshear | Signal-to-noise ratio of the object. |
| ra | Right ascension of the detected object, in degrees. |
| selection | Flag indicating if the object passes the cosmology sample selection criteria |
| tomobin | Tomographic redshift bin assignment used for cosmological analysis. Some objecs that don’t pass the selection criteria still have assignments, but note these objects are not actually used in anny calibrations. |
| true_bdf_flux_i | True (i)-band flux in the object’s BDF fit model from DES Y3 deepfields |
| true_bdf_flux_r | True (r)-band flux in the object’s BDF fit model from DES Y3 deepfields |
| true_bdf_flux_z | True (z)-band flux in the object’s BDF fit model from DES Y3 deepfields |
| true_dec | Declination of the the chosen injection location |
| true_ra | Right ascension of the the chosen injection location |
| z_weights | The weighting used in redshift estimation, \(w_z = w_\gamma \times R_\gamma\), which includes both the shear weight and the response. See equation (5) in Paper II |
