arXiv subjects

Boris Leistedt

Publications and source records attributed to Boris Leistedt.

At least 37 records · Page 2Linked to original sources

arXiv2022

Spurious correlations between galaxies and multi-epoch image stacks in the DESI Legacy Surveys

A non-negligible source of systematic bias in cosmological analyses of galaxy surveys is the on-sky modulation caused by foregrounds and variable image characteristics such as observing conditions. Standard mitigation techniques perform a regression between the observed galaxy density field and sky maps of the potential contaminants. Such maps are ad-hoc, lossy summaries of the heterogeneous sets of co-added exposures that contribute to the survey. We present a methodology to address this limitation, and extract the spurious correlations between the observed distribution of galaxies and arbitrary stacks of single-epoch exposures. We study four types of galaxies (LRGs, ELGs, QSOs, LBGs) in the three regions of the DESI Legacy Surveys (North, South, DES), which results in twelve samples with varying levels and type of contamination. We find that the new technique outperforms the traditional ones in all cases, and is able to remove higher levels of contamination. This paves the way for new methods that extract more information from multi-epoch galaxy survey data and mitigate large-scale biases more effectively.

astro-ph.CO↗

arXiv2022

Mapping Interstellar Dust with Gaussian Processes

Interstellar dust corrupts nearly every stellar observation, and accounting for it is crucial to measuring physical properties of stars. We model the dust distribution as a spatially varying latent field with a Gaussian process (GP) and develop a likelihood model and inference method that scales to millions of astronomical observations. Modeling interstellar dust is complicated by two factors. The first is integrated observations. The data come from a vantage point on Earth and each observation is an integral of the unobserved function along our line of sight, resulting in a complex likelihood and a more difficult inference problem than in classical GP inference. The second complication is scale; stellar catalogs have millions of observations. To address these challenges we develop ziggy, a scalable approach to GP inference with integrated observations based on stochastic variational inference. We study ziggy on synthetic data and the Ananke dataset, a high-fidelity mechanistic model of the Milky Way with millions of stars. ziggy reliably infers the spatial dust map with well-calibrated posterior uncertainties.

astro-ph.GA↗

arXiv2021

Mapping dark matter on the celestial sphere with weak gravitational lensing

Convergence maps of the integrated matter distribution are a key science result from weak gravitational lensing surveys. To date, recovering convergence maps has been performed using a planar approximation of the celestial sphere. However, with the increasing area of sky covered by dark energy experiments, such as Euclid, the Large Synoptic Survey Telescope (LSST), and the Wide Field Infrared Survey Telescope (WFIRST), this assumption will no longer be valid. We recover convergence fields on the celestial sphere using an extension of the Kaiser-Squires estimator to the spherical setting. Through simulations we study the error introduced by planar approximations. Moreover, we examine how best to recover convergence maps in the planar setting, considering a variety of different projections and defining the local rotations that are required when projecting spin fields such as cosmic shear. For the sky coverages typical of future surveys, errors introduced by projection effects can be of order tens of percent, exceeding 50% in some cases. The stereographic projection, which is conformal and so preserves local angles, is the most effective planar projection. In any case, these errors can be avoided entirely by recovering convergence fields directly on the celestial sphere. We apply the spherical Kaiser-Squires mass-mapping method presented to the public Dark Energy Survey (DES) science verification data to recover convergence maps directly on the celestial sphere.

astro-ph.CO↗

arXiv2020

Data Analysis Recipes: Products of multivariate Gaussians in Bayesian inferences

A product of two Gaussians (or normal distributions) is another Gaussian. That's a valuable and useful fact! Here we use it to derive a refactoring of a common product of multivariate Gaussians: The product of a Gaussian likelihood times a Gaussian prior, where some or all of those parameters enter the likelihood only in the mean and only linearly. That is, a linear, Gaussian, Bayesian model. This product of a likelihood times a prior pdf can be refactored into a product of a marginalized likelihood (or a Bayesian evidence) times a posterior pdf, where (in this case) both of these are also Gaussian. The means and variance tensors of the refactored Gaussians are straightforward to obtain as closed-form expressions; here we deliver these expressions, with discussion. The closed-form expressions can be used to speed up and improve the precision of inferences that contain linear parameters with Gaussian priors. We connect these methods to inferences that arise frequently in physics and astronomy. If all you want is the answer, the question is posed and answered at the beginning of Section 3. We show two toy examples, in the form of worked exercises, in Section 4. The solutions, discussion, and exercises in this Note are aimed at someone who is already familiar with the basic ideas of Bayesian inference and probability.

stat.CO↗

arXiv2020

SPECULATOR: Emulating stellar population synthesis for fast and accurate galaxy spectra and photometry

We present SPECULATOR - a fast, accurate, and flexible framework for emulating stellar population synthesis (SPS) models for predicting galaxy spectra and photometry. For emulating spectra, we use principal component analysis to construct a set of basis functions, and neural networks to learn the basis coefficients as a function of the SPS model parameters. For photometry, we parameterize the magnitudes (for the filters of interest) as a function of SPS parameters by a neural network. The resulting emulators are able to predict spectra and photometry under both simple and complicated SPS model parameterizations to percent-level accuracy, giving a factor of $10^3$-$10^4$ speed up over direct SPS computation. They have readily-computable derivatives, making them amenable to gradient-based inference and optimization methods. The emulators are also straightforward to call from a GPU, giving an additional order-of-magnitude speed-up. Rapid SPS computations delivered by emulation offers a massive reduction in the computational resources required to infer the physical properties of galaxies from observed spectra or photometry and simulate galaxy populations under SPS models, whilst maintaining the accuracy required for a range of applications.

astro-ph.IM↗

arXiv2020

Target Neutrino Mass Precision for Determining the Neutrino Hierarchy

Recent works combining neutrino oscillation and cosmological data to determine the neutrino hierarchy found a range of odds in favour of the normal hierarchy. These results arise from differing approaches to incorporating prior knowledge about neutrinos. We develop a hierarchy-agnostic prior and show that the hierarchy cannot be conclusively determined with current data. The determination of the hierarchy is limited by the neutrino mass scale $Σ_ν$ measurement. We obtain a target precision of $σ(Σ_ν) = 0.014$ eV, necessary for conclusively establishing the normal hierarchy with future data.

astro-ph.CO↗

arXiv2018

Hierarchical modeling and statistical calibration for photometric redshifts

The cosmological exploitation of modern photometric galaxy surveys requires both accurate (unbiased) and precise (narrow) redshift probability distributions derived from broadband photometry. Existing methodologies do not meet those requirements. Standard template fitting delivers interpretable models and errors, but lacks flexibility to learn inaccuracies in the observed photometry or the spectral templates. Machine learning addresses those issues, but requires representative training data, and the resulting models and uncertainties cannot be interpreted in the context of a physical model or outside of the training data. We present a hierarchical modeling approach simultaneously addressing the issues of flexibility, interpretability, and generalization. It combines template fitting with flexible (machine learning-like) models to correct the spectral templates, model their redshift distributions, and recalibrate the photometric observations. By optimizing the full posterior distribution of the model and solving for its (thousands of) parameters, one can perform a global statistical calibration of the data and the SED model. We apply this approach to the public Dark Energy Survey Science Verification data, and show that it provides more accurate and compact redshift posterior distributions than existing methods, as well as insights into residual photometric and SED systematics. The model is causal, makes predictions for future data (e.g., additional photometric bandpasses), and its internal parameters and components are interpretable. This approach does not formally require the training data to be complete or representative; in principle it can even work in regimes in which few or no spectroscopic redshifts are available.

astro-ph.CO↗

arXiv2018

Correlations in the three-dimensional Lyman-alpha forest contaminated by high column density absorbers

Correlations measured in three dimensions in the Lyman-alpha forest are contaminated by the presence of the damping wings of high column density (HCD) absorbing systems of neutral hydrogen (HI; having column densities $N(\mathrm{HI}) > 1.6\times10^{17}\,\mathrm{atoms}\,\mathrm{cm}^{-2}$), which extend significantly beyond the redshift-space location of the absorber. We measure this effect as a function of the column density of the HCD absorbers and redshift by measuring 3D flux power spectra in cosmological hydrodynamical simulations from the Illustris project. Survey pipelines exclude regions containing the largest damping wings. We find that, even after this procedure, there is a scale-dependent correction to the 3D Lyman-alpha forest flux power spectrum from residual contamination. We model this residual using a simple physical model of the HCD absorbers as linearly biased tracers of the matter density distribution, convolved with their Voigt profiles and integrated over the column density distribution function. We recommend the use of this model over existing models used in data analysis, which approximate the damping wings as top-hats and so miss shape information in the extended wings. The simple 'linear Voigt model' is statistically consistent with our simulation results for a mock residual contamination up to small scales ($|k| < 1\,h\,\mathrm{Mpc}^{-1}$). It does not account for the effect of the highest column density absorbers on the smallest scales (e.g., $|k| > 0.4\,h\,\mathrm{Mpc}^{-1}$ for small damped Lyman-alpha absorbers; HCD absorbers with $N(\mathrm{HI}) \sim 10^{21}\,\mathrm{atoms}\,\mathrm{cm}^{-2}$). However, these systems are in any case preferentially removed from survey data. Our model is appropriate for an accurate analysis of the baryon acoustic oscillations feature. It is additionally essential for reconstructing the full shape of the 3D flux power spectrum.

astro-ph.CO↗

arXiv2018

Inferring binary and trinary stellar populations in photometric and astrometric surveys

Multiple stellar systems are ubiquitous in the Milky Way, but are often unresolved and seen as single objects in spectroscopic, photometric, and astrometric surveys. Yet, modeling them is essential for developing a full understanding of large surveys such as Gaia, and connecting them to stellar and Galactic models. In this paper we address this problem by jointly fitting the Gaia and 2MASS photometric and astrometric data using a data-driven Bayesian hierarchical model that includes populations of binary and trinary systems. This allows us to classify observations into singles, binaries, and trinaries, in a robust and efficient manner, without resorting to external models. We are able to identify multiple systems and, in some cases, make strong predictions for the properties of its unresolved stars. We will be able to compare such predictions with Gaia Data Release 4, which will contain astrometric identification and analysis of binary systems.

astro-ph.SR↗

arXiv2018

Simulating the effect of high column density absorbers on the one-dimensional Lyman-alpha forest flux power spectrum

We measure the effect of high column density absorbing systems of neutral hydrogen (HI) on the one-dimensional (1D) Lyman-alpha forest flux power spectrum using cosmological hydrodynamical simulations from the Illustris project. High column density absorbers (which we define to be those with HI column densities $N(\mathrm{HI}) > 1.6 \times 10^{17}\,\mathrm{atoms}\,\mathrm{cm}^{-2}$) cause broadened absorption lines with characteristic damping wings. These damping wings bias the 1D Lyman-alpha forest flux power spectrum by causing absorption in quasar spectra away from the location of the absorber itself. We investigate the effect of high column density absorbers on the Lyman-alpha forest using hydrodynamical simulations for the first time. We provide templates as a function of column density and redshift, allowing the flexibility to accurately model residual contamination, i.e., if an analysis selectively clips out the largest damping wings. This flexibility will improve cosmological parameter estimation, e.g., allowing more accurate measurement of the shape of the power spectrum, with implications for cosmological models containing massive neutrinos or a running of the spectral index. We provide fitting functions to reproduce these results so that they can be incorporated straightforwardly into a data analysis pipeline.

astro-ph.CO↗

arXiv2017

Red clump stars and Gaia: Calibration of the standard candle using a hierarchical probabilistic model

Distances to individual stars in our own Galaxy are critical in order to piece together the nature of its velocity and spatial structure. Core helium burning red clump (RC) stars have similar luminosities, are abundant throughout the Galaxy, and thus constitute good standard candles. We build a hierarchical probabilistic model to quantify the quality of RC stars as standard candles using parallax measurements from the first Gaia data release. A unique aspect of our methodology is to fully account for (and marginalize over) parallax, photometry, and dust corrections uncertainties, which leads to more robust results than standard approaches. We determine the absolute magnitude and intrinsic dispersion of the RC in 2MASS bands J, H, Ks, Gaia G band, and WISE bands W1, W2, W3, and W4. We find that the absolute magnitude of the RC is $-1.61 \pm$ 0.01 (in Ks), $+0.44 \pm$ 0.01 (in G) , $-0.93 \pm$ 0.01 (in J), $-1.46 \pm$ 0.01 (in H), $-1.68 \pm$ 0.02 (in W1), $-1.69\pm$ 0.02 (in W2), $-1.67 \pm$ 0.02 (in W3), $1.76 \pm$ 0.01 mag (in W4). The mean intrinsic dispersion is $\sim 0.17 \pm$ 0.03 mag across all bands (yielding a typical distance precision of $\sim$ 8%). Thus RC stars are reliable and precise standard candles. In addition, we have also re-calibrated the zero point of the absolute magnitude of the RC in each band, which provide a benchmark for future studies to estimate distances to RC stars. Finally, the parallax error shrinkage in the hierarchical model outlined in this work can be used to obtain more precise parallaxes than Gaia for the most distant RC stars across the Galaxy.

astro-ph.GA↗

arXiv2017

Improving \textsl{Gaia} parallax precision with a data-driven model of stars

Converting a noisy parallax measurement into a posterior belief over distance requires inference with a prior. Usually this prior represents beliefs about the stellar density distribution of the Milky Way. However, multi-band photometry exists for a large fraction of the \textsl{\small{Gaia}} \textsl{\small{TGAS}} Catalog and is incredibly informative about stellar distances. Here we use \textsl{\small{2MASS}} colors for 1.4 million \textsl{\small{TGAS}} stars to build a noise-deconvolved empirical prior distribution for stars in color--magnitude space. This model contains no knowledge of stellar astrophysics or the Milky Way, but is precise because it accurately generates a large number of noisy parallax measurements under an assumption of stationarity; that is, it is capable of combining the information from many stars. We use the Extreme Deconvolution (\textsl{\small{XD}}) algorithm---an Empirical Bayes approximation to a full hierarchical model of the true parallax and photometry of every star---to construct this prior. The prior is combined with a \textsl{\small{TGAS}} likelihood to infer a precise photometric parallax estimate and uncertainty (and full posterior) for every star. Our parallax estimates are more precise than the \textsl{\small{TGAS}} catalog entries by a median factor of 1.2 (14% are more precise by a factor >2) and are more precise than previous Bayesian distance estimates that use spatial priors. We validate our parallax inferences using members of the Milky Way star cluster M67, which is not visible as a cluster in the \textsl{\small{TGAS}} parallax estimates, but appears as a cluster in our posterior parallax estimates. Our results, including a parallax posterior pdf for each of 1.4 million \textsl{\small{TGAS}} stars, are available in companion electronic tables.

astro-ph.GA↗

arXiv2017

Directional spin wavelets on the sphere

We construct a directional spin wavelet framework on the sphere by generalising the scalar scale-discretised wavelet transform to signals of arbitrary spin. The resulting framework is the only wavelet framework defined natively on the sphere that is able to probe the directional intensity of spin signals. Furthermore, directional spin scale-discretised wavelets support the exact synthesis of a signal on the sphere from its wavelet coefficients and satisfy excellent localisation and uncorrelation properties. Consequently, directional spin scale-discretised wavelets are likely to be of use in a wide range of applications and in particular for the analysis of the polarisation of the cosmic microwave background (CMB). We develop new algorithms to compute (scalar and spin) forward and inverse wavelet transforms exactly and efficiently for very large data-sets containing tens of millions of samples on the sphere. By leveraging a novel sampling theorem on the rotation group developed in a companion article, only half as many wavelet coefficients as alternative approaches need be computed, while still capturing the full information content of the signal under analysis. Our implementation of these algorithms is made publicly available.

cs.IT↗

arXiv2017

Hierarchical probabilistic inference of the color-magnitude diagram and shrinkage of stellar distance uncertainties

We present a hierarchical probabilistic model for improving geometric stellar distance estimates using color--magnitude information. This is achieved with a data driven model of the color--magnitude diagram, not relying on stellar models but instead on the relative abundances of stars in color--magnitude cells, which are inferred from very noisy magnitudes and parallaxes. While the resulting noise-deconvolved color--magnitude diagram can be useful for a range of applications, we focus on deriving improved stellar distance estimates relying on both parallax and photometric information. We demonstrate the efficiency of this approach on the 1.4 million stars of the Gaia TGAS sample that also have APASS magnitudes. Our hierarchical model has 4~million parameters in total, most of which are marginalized out numerically or analytically. We find that distance estimates are significantly improved for the noisiest parallaxes and densest regions of the color--magnitude diagram. In particular, the average distance signal-to-noise ratio and uncertainty improve by 19~percent and 36~percent, respectively, with 8~percent of the objects improving in SNR by a factor greater than 2. This computationally efficient approach fully accounts for both parallax and photometric noise, and is a first step towards a full hierarchical probabilistic model of the Gaia data.

astro-ph.SR↗

arXiv2016

Wavelet reconstruction of E and B modes for CMB polarisation and cosmic shear analyses

We present new methods for mapping the curl-free (E-mode) and divergence-free (B-mode) components of spin 2 signals using spin directional wavelets. Our methods are equally applicable to measurements of the polarisation of the cosmic microwave background (CMB) and the shear of galaxy shapes due to weak gravitational lensing. We derive pseudo and pure wavelet estimators, where E-B mixing arising due to incomplete sky coverage is suppressed in wavelet space using scale- and orientation-dependent masking and weighting schemes. In the case of the pure estimator, ambiguous modes (which have vanishing curl and divergence simultaneously on the incomplete sky) are also cancelled. On simulations, we demonstrate the improvement (i.e., reduction in leakage) provided by our wavelet space estimators over standard harmonic space approaches. Our new methods can be directly interfaced in a coherent and computationally-efficient manner with component separation or feature extraction techniques that also exploit wavelets.

astro-ph.CO↗

arXiv2016

Data-driven, interpretable photometric redshifts trained on heterogeneous and unrepresentative data

We present a new method for inferring photometric redshifts in deep galaxy and quasar surveys, based on a data driven model of latent spectral energy distributions (SEDs) and a physical model of photometric fluxes as a function of redshift. This conceptually novel approach combines the advantages of both machine-learning and template-fitting methods by building template SEDs directly from the training data. This is made computationally tractable with Gaussian Processes operating in flux--redshift space, encoding the physics of redshift and the projection of galaxy SEDs onto photometric band passes. This method alleviates the need of acquiring representative training data or constructing detailed galaxy SED models; it requires only that the photometric band passes and calibrations be known or have parameterized unknowns. The training data can consist of a combination of spectroscopic and deep many-band photometric data, which do not need to entirely spatially overlap with the target survey of interest or even involve the same photometric bands. We showcase the method on the $i$-magnitude-selected, spectroscopically-confirmed galaxies in the COSMOS field. The model is trained on the deepest bands (from SUBARU and HST) and photometric redshifts are derived using the shallower SDSS optical bands only. We demonstrate that we obtain accurate redshift point estimates and probability distributions despite the training and target sets having very different redshift distributions, noise properties, and even photometric bands. Our model can also be used to predict missing photometric fluxes, or to simulate populations of galaxies with realistic fluxes and redshifts, for example. This method opens a new era in which photometric redshifts for large photometric surveys are derived using a flexible yet physical model of the data trained on all available surveys (spectroscopic and photometric).

astro-ph.CO↗

arXiv2016

Second-Generation Curvelets on the Sphere

Curvelets are efficient to represent highly anisotropic signal content, such as a local linear and curvilinear structure. First-generation curvelets on the sphere, however, suffered from blocking artefacts. We present a new second-generation curvelet transform, where scale-discretised curvelets are constructed directly on the sphere. Scale-discretised curvelets exhibit a parabolic scaling relation, are well-localised in both spatial and harmonic domains, support the exact analysis and synthesis of both scalar and spin signals, and are free of blocking artefacts. We present fast algorithms to compute the exact curvelet transform, reducing computational complexity from $\mathcal{O}(L^5)$ to $\mathcal{O}(L^3\log_{2}{L})$ for signals band-limited at $L$. The implementation of these algorithms is made publicly available. Finally, we present an illustrative application demonstrating the effectiveness of curvelets for representing directional curve-like features in natural spherical images.

cs.IT↗

arXiv2016

Unbiased pseudo-Cl power spectrum estimation with mode projection

With the steadily improving sensitivity afforded by current and future galaxy surveys, a robust extraction of two-point correlation function measurements may become increasingly hampered by the presence of astrophysical foregrounds or observational systematics. The concept of mode projection has been introduced as a means to remove contaminants for which it is possible to construct a spatial map reflecting the expected signal contribution. Owing to its computational efficiency compared to minimum-variance methods, the sub-optimal pseudo-Cl (PCL) power spectrum estimator is a popular tool for the analysis of high-resolution data sets. Here, we integrate mode projection into the framework of PCL power spectrum estimation. In contrast to results obtained with optimal estimators, we show that the uncorrected projection of template maps leads to biased power spectra. Based on analytical calculations, we find exact closed-form expressions for the expectation value of the bias and demonstrate that they can be recast in a form that allows a numerically efficient evaluation, preserving the favorable O(l_max^3) time complexity of PCL estimator algorithms. Using simulated data sets, we assess the scaling of the bias with various analysis parameters and demonstrate that it can be reliably removed. We conclude that in combination with mode projection, PCL estimators allow for a fast and robust computation of power spectra in the presence of systematic effects -- properties in high demand for the analysis of ongoing and future large scale structure surveys.

astro-ph.CO↗