arXiv subjects

Jason D. McEwen

Publications and source records attributed to Jason D. McEwen.

At least 73 records · Page 4Linked to original sources

arXiv2019

Quantifying Uncertainty in High Dimensional Inverse Problems by Convex Optimisation

Inverse problems play a key role in modern image/signal processing methods. However, since they are generally ill-conditioned or ill-posed due to lack of observations, their solutions may have significant intrinsic uncertainty. Analysing and quantifying this uncertainty is very challenging, particularly in high-dimensional problems and problems with non-smooth objective functionals (e.g. sparsity-promoting priors). In this article, a series of strategies to visualise this uncertainty are presented, e.g. highest posterior density credible regions, and local credible intervals (cf. error bars) for individual pixels and superpixels. Our methods support non-smooth priors for inverse problems and can be scaled to high-dimensional settings. Moreover, we present strategies to automatically set regularisation parameters so that the proposed uncertainty quantification (UQ) strategies become much easier to use. Also, different kinds of dictionaries (complete and over-complete) are used to represent the image/signal and their performance in the proposed UQ methodology is investigated.

eess.SP↗

arXiv2019

Cosmic Shear: Inference from Forward Models

Density-estimation likelihood-free inference (DELFI) has recently been proposed as an efficient method for simulation-based cosmological parameter inference. Compared to the standard likelihood-based Markov Chain Monte Carlo (MCMC) approach, DELFI has several advantages: it is highly parallelizable, there is no need to assume a possibly incorrect functional form for the likelihood and complicated effects (e.g the mask and detector systematics) are easier to handle with forward models. In light of this, we present two DELFI pipelines to perform weak lensing parameter inference with lognormal realizations of the tomographic shear field -- using the C_l summary statistic. The first pipeline accounts for the non-Gaussianities of the shear field, intrinsic alignments and photometric-redshift error. We validate that it is accurate enough for Stage III experiments and estimate that O(1000) simulations are needed to perform inference on Stage IV data. By comparing the second DELFI pipeline, which makes no assumption about the functional form of the likelihood, with the standard MCMC approach, which assumes a Gaussian likelihood, we test the impact of the Gaussian likelihood approximation in the MCMC analysis. We find it has a negligible impact on Stage IV parameter constraints. Our pipeline is a step towards seamlessly propagating all data-processing, instrumental, theoretical and astrophysical systematics through to the final parameter constraints.

astro-ph.CO↗

arXiv2019

Non-parametric Cosmology with Cosmic Shear

We present a method to measure the growth of structure and the background geometry of the Universe -- with no a priori assumption about the underlying cosmological model. Using Canada-France-Hawaii Lensing Survey (CFHTLenS) shear data we simultaneously reconstruct the lensing amplitude, the linear intrinsic alignment amplitude, the redshift evolving matter power spectrum, P(k,z), and the co-moving distance, r(z). We find that lensing predominately constrains a single global power spectrum amplitude and several co-moving distance bins. Our approach can localise precise scales and redshifts where Lambda-Cold Dark Matter (LCDM) fails -- if any. We find that below z = 0.4, the measured co-moving distance r (z) is higher than that expected from the Planck LCDM cosmology by ~1.5 sigma, while at higher redshifts, our reconstruction is fully consistent. To validate our reconstruction, we compare LCDM parameter constraints from the standard cosmic shear likelihood analysis to those found by fitting to the non-parametric information and we find good agreement.

astro-ph.CO↗

arXiv2019

Radio Galaxy Detection in the Visibility Domain

We explore a new Bayesian method of detecting galaxies from radio interferometric data of the faint sky. Working in the Fourier domain, we fit a single, parameterised galaxy model to simulated visibility data of star-forming galaxies. The resulting multimodal posterior distribution is then sampled using a multimodal nested sampling algorithm such as MultiNest. For each galaxy, we construct parameter estimates for the position, flux, scale-length and ellipticities from the posterior samples. We first test our approach on simulated SKA1-MID visibility data of up to 100 galaxies in the field of view, considering a typical weak lensing survey regime (SNR $\ge 10$) where 98% of the input galaxies are detected with no spurious source detections. We then explore the low SNR regime, finding our approach reliable in galaxy detection and providing in particular high accuracy in positional estimates down to SNR $\sim 5$. The presented method does not require transformation of visibilities to the image domain, and requires no prior knowledge of the number of galaxies in the field of view, thus could become a useful tool for constructing accurate radio galaxy catalogs in the future.

astro-ph.IM↗

arXiv2019

Load balancing for distributed interferometric image reconstruction

We present a new algorithm to perform wide-field radio interferometric image reconstruction, with exact non-coplanar correction, that scales to big-data. This algorithm allows us to image 2 billion visibilities on 50 nodes of a computing cluster for a 25 by 25 degree field of view, in a little over an hour. We build on the recently developed distributed $w$-stacking $w$-projection hybrid algorithm, extending it to include a new distributed degridding algorithm that balances the computational load of the $w$-projection gridding kernels. The implementation of our algorithm is made publicly available in the PURIFY software package. Wide-field image reconstruction for data sets of this size cannot be performed effectively using the allocated computational resources without computational load balancing, demonstrating that our algorithms are critical for next-generation wide-field radio interferometers.

astro-ph.IM↗

arXiv2019

A fast and exact $w$-stacking and $w$-projection hybrid algorithm for wide-field interferometric imaging

The standard wide-field imaging technique, the $w$-projection, allows correction for wide-fields of view for non-coplanar radio interferometric arrays. However, calculating exact corrections for each measurement has not been possible due to the amount of computation required at high resolution and with the large number of visibilities from current interferometers. The required accuracy and computational cost of these corrections is one of the largest unsolved challenges facing next generation radio interferometers such as the Square Kilometre Array. We show that the same calculation can be performed with a radially symmetric $w$-projection kernel, where we use one dimensional adaptive quadrature to calculate the resulting Hankel transform, decreasing the computation required for kernel generation by several orders of magnitude, whilst preserving the accuracy. We confirm that the radial $w$-projection kernel is accurate to approximately 1% by imaging the zero-spacing with an added $w$-term. We demonstrate the potential of our radially symmetric $w$-projection kernel via sparse image reconstruction, using the software package PURIFY. We develop a distributed $w$-stacking and $w$-projection hybrid algorithm. We apply this algorithm to individually correct for non-coplanar effects in 17.5 million visibilities over a $25$ by $25$ degree field of view MWA observation for image reconstruction. Such a level of accuracy and scalability is not possible with standard $w$-projection kernel generation methods. This demonstrates that we can scale to a large number of measurements with large image sizes whilst still maintaining both speed and accuracy.

astro-ph.IM↗

arXiv2019

Efficient sampling and robust 3D diffusion magnetic resonance imaging signal reconstruction

This paper presents novel single and multi-shell sampling schemes for diffusion MRI. In diffusion MRI, it is paramount that the number of samples is as small as possible in order that scan times are practical in a clinical setting. The proposed schemes use an efficient number of measurements in that the number of samples is equal to the degrees of freedom in the orthonormal bases used for reconstruction. Novel reconstruction algorithms based on smaller subsystems of linear equations, as compared to the standard regularized least-squares method, are developed for both single and multi-shells sampling schemes. The smaller matrices used in these novel reconstruction algorithms are designed to be well-conditioned, leading to improved reconstruction accuracy. Accurate and robust reconstruction is also achieved through incorporation of regularization into the novel reconstruction algorithms and using a Rician or non-central Chi noise model. We quantitatively validate our single and multi-shell schemes against standard least-squares reconstruction methods to show that they enable more accurate reconstruction when the number of samples is equal to the degrees of freedom in the basis. Human brain data is also used to qualitatively evaluate reconstruction

eess.SP↗

arXiv2019

Covariant polarized radiative transfer on cosmological scales for investigating large-scale magnetic field structures

Polarization of radiation is a powerful tool to study cosmic magnetism and analysis of polarization can be used as a diagnostic tool for large-scale structures. In this paper, we present a solid theoretical foundation for using polarized light to investigate large-scale magnetic field structures: the cosmological polarized radiative transfer (CPRT) formulation. The CPRT formulation is fully covariant. It accounts for cosmological and relativistic effects in a self-consistent manner and explicitly treats Faraday rotation, as well as Faraday conversion, emission, and absorption processes. The formulation is derived from the first principles of conservation of phase-space volume and photon number. Without loss of generality, we consider a flat Friedmann-Robertson-Walker (FRW) space-time metric and construct the corresponding polarized radiative transfer equations. We propose an all-sky CPRT calculation algorithm, based on a ray-tracing method, which allows cosmological simulation results to be incorporated and, thereby, model templates of polarization maps to be constructed. Such maps will be crucial in our interpretation of polarized data, such as those to be collected by the Square Kilometer Array (SKA). We describe several tests which are used for verifying the code and demonstrate applications in the study of the polarization signatures in different distributions of electron number density and magnetic fields. We present a pencil-beam CPRT calculation and an all-sky calculation, using a simulated galaxy cluster or a model magnetized universe obtained from GCMHD+ simulations as the respective input structures. The implications on large-scale magnetic field studies are discussed; remarks on the standard methods using rotation measure are highlighted.

astro-ph.CO↗

arXiv2018

Optimizing the LSST Observing Strategy for Dark Energy Science: DESC Recommendations for the Wide-Fast-Deep Survey

Cosmology is one of the four science pillars of LSST, which promises to be transformative for our understanding of dark energy and dark matter. The LSST Dark Energy Science Collaboration (DESC) has been tasked with deriving constraints on cosmological parameters from LSST data. Each of the cosmological probes for LSST is heavily impacted by the choice of observing strategy. This white paper is written by the LSST DESC Observing Strategy Task Force (OSTF), which represents the entire collaboration, and aims to make recommendations on observing strategy that will benefit all cosmological analyses with LSST. It is accompanied by the DESC DDF (Deep Drilling Fields) white paper (Scolnic et al.). We use a variety of metrics to understand the effects of the observing strategy on measurements of weak lensing, large-scale structure, clusters, photometric redshifts, supernovae, strong lensing and kilonovae. In order to reduce systematic uncertainties, we conclude that the current baseline observing strategy needs to be significantly modified to result in the best possible cosmological constraints. We provide some key recommendations: moving the WFD (Wide-Fast-Deep) footprint to avoid regions of high extinction, taking visit pairs in different filters, changing the 2x15s snaps to a single exposure to improve efficiency, focusing on strategies that reduce long gaps (>15 days) between observations, and prioritizing spatial uniformity at several intervals during the 10-year survey.

astro-ph.IM↗

arXiv2018

Optimizing the LSST Observing Strategy for Dark Energy Science: DESC Recommendations for the Deep Drilling Fields and other Special Programs

We review the measurements of dark energy enabled by observations of the Deep Drilling Fields and the optimization of survey design for cosmological measurements. This white paper is the result of efforts by the LSST DESC Observing Strategy Task Force (OSTF), which represents the entire collaboration, and aims to make recommendations on observing strategy for the DDFs that will benefit all cosmological analyses with LSST. It is accompanied by the DESC-WFD white paper (Lochner et al.). We argue for altering the nominal deep drilling plan to have $>6$ month seasons, interweaving $gri$ and $zy$ observations every 3 days with 2, 4, 8, 25, 4 visits in $grizy$, respectively. These recommendations are guided by metrics optimizing constraints on dark energy and mitigation of systematic uncertainties, including specific requirements on total number of visits after Y1 and Y10 for photometric redshifts (photo-$z$) and weak lensing systematics. We specify the precise locations for the previously-chosen LSST deep fields (ELAIS-S1, XMM-LSS, CDF-S, and COSMOS) and recommend Akari Deep Field South as the planned fifth deep field in order to synergize with Euclid and WFIRST. Our recommended DDF strategy uses $6.2\%$ of the LSST survey time. We briefly discuss synergy with white papers from other collaborations, as well as additional mini-surveys and Target-of-Opportunity programs that lead to better measurements of dark energy.

astro-ph.IM↗

arXiv2018

Evaluating machine learning techniques for predicting power spectra from reionization simulations

Upcoming experiments such as the SKA will provide huge quantities of data. Fast modelling of the high-redshift 21cm signal will be crucial for efficiently comparing these data sets with theory. The most detailed theoretical predictions currently come from numerical simulations and from faster but less accurate semi-numerical simulations. Recently, machine learning techniques have been proposed to emulate the behaviour of these semi-numerical simulations with drastically reduced time and computing cost. We compare the viability of five such machine learning techniques for emulating the 21cm power spectrum of the publicly-available code SimFast21. Our best emulator is a multilayer perceptron with three hidden layers, reproducing SimFast21 power spectra $10^8$ times faster than the simulation with 4% mean squared error averaged across all redshifts and input parameters. The other techniques (interpolation, Gaussian processes regression, and support vector machine) have slower prediction times and worse prediction accuracy than the multilayer perceptron. All our emulators can make predictions at any redshift and scale, which gives more flexible predictions but results in significantly worse prediction accuracy at lower redshifts. We then present a proof-of-concept technique for mapping between two different simulations, exploiting our best emulator's fast prediction speed. We demonstrate this technique to find a mapping between SimFast21 and another publicly-available code 21cmFAST. We observe a noticeable offset between the simulations for some regions of the input space. Such techniques could potentially be used as a bridge between fast semi-numerical simulations and accurate numerical radiative transfer simulations.

astro-ph.CO↗

arXiv2018

The Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC): Data set

The Photometric LSST Astronomical Time Series Classification Challenge (PLAsTiCC) is an open data challenge to classify simulated astronomical time-series data in preparation for observations from the Large Synoptic Survey Telescope (LSST), which will achieve first light in 2019 and commence its 10-year main survey in 2022. LSST will revolutionize our understanding of the changing sky, discovering and measuring millions of time-varying objects. In this challenge, we pose the question: how well can we classify objects in the sky that vary in brightness from simulated LSST time-series data, with all its challenges of non-representativity? In this note we explain the need for a data challenge to help classify such astronomical sources and describe the PLAsTiCC data set and Kaggle data challenge, noting that while the references are provided for context, they are not needed to participate in the challenge.

astro-ph.IM↗

arXiv2018

Uncertainty quantification for radio interferometric imaging: I. proximal MCMC methods

Uncertainty quantification is a critical missing component in radio interferometric imaging that will only become increasingly important as the big-data era of radio interferometry emerges. Since radio interferometric imaging requires solving a high-dimensional, ill-posed inverse problem, uncertainty quantification is difficult but also critical to the accurate scientific interpretation of radio observations. Statistical sampling approaches to perform Bayesian inference, like Markov Chain Monte Carlo (MCMC) sampling, can in principle recover the full posterior distribution of the image, from which uncertainties can then be quantified. However, traditional high-dimensional sampling methods are generally limited to smooth (e.g. Gaussian) priors and cannot be used with sparsity-promoting priors. Sparse priors, motivated by the theory of compressive sensing, have been shown to be highly effective for radio interferometric imaging. In this article proximal MCMC methods are developed for radio interferometric imaging, leveraging proximal calculus to support non-differential priors, such as sparse priors, in a Bayesian framework. Furthermore, three strategies to quantify uncertainties using the recovered posterior distribution are developed: (i) local (pixel-wise) credible intervals to provide error bars for each individual pixel; (ii) highest posterior density credible regions; and (iii) hypothesis testing of image structure. These forms of uncertainty quantification provide rich information for analysing radio interferometric observations in a statistically robust manner.

astro-ph.IM↗

arXiv2018

Uncertainty quantification for radio interferometric imaging: II. MAP estimation

Uncertainty quantification is a critical missing component in radio interferometric imaging that will only become increasingly important as the big-data era of radio interferometry emerges. Statistical sampling approaches to perform Bayesian inference, like Markov Chain Monte Carlo (MCMC) sampling, can in principle recover the full posterior distribution of the image, from which uncertainties can then be quantified. However, for massive data sizes, like those anticipated from the Square Kilometre Array (SKA), it will be difficult if not impossible to apply any MCMC technique due to its inherent computational cost. We formulate Bayesian inference problems with sparsity-promoting priors (motivated by compressive sensing), for which we recover maximum a posteriori (MAP) point estimators of radio interferometric images by convex optimisation. Exploiting recent developments in the theory of probability concentration, we quantify uncertainties by post-processing the recovered MAP estimate. Three strategies to quantify uncertainties are developed: (i) highest posterior density credible regions; (ii) local credible intervals (cf. error bars) for individual pixels and superpixels; and (iii) hypothesis testing of image structure. These forms of uncertainty quantification provide rich information for analysing radio interferometric observations in a statistically robust manner. Our MAP-based methods are approximately $10^5$ times faster computationally than state-of-the-art MCMC methods and, in addition, support highly distributed and parallelised algorithmic structures. For the first time, our MAP-based techniques provide a means of quantifying uncertainties for radio interferometric imaging for realistic data volumes and practical use, and scale to the emerging big-data era of radio astronomy.

astro-ph.IM↗

arXiv2018

Preparing for the Cosmic Shear Data Flood: Optimal Data Extraction and Simulation Requirements for Stage IV Dark Energy Experiments

Upcoming photometric lensing surveys will considerably tighten constraints on the neutrino mass and the dark energy equation of state. Nevertheless it remains an open question of how to optimally extract the information and how well the matter power spectrum must be known to obtain unbiased cosmological parameter estimates. By performing a Principal Component Analysis (PCA), we quantify the sensitivity of 3D cosmic shear and tomography with different binning strategies to different regions of the lensing kernel and matter power spectrum, and hence the background geometry and growth of structure in the Universe. We find that a large number of equally spaced tomographic bins in redshift can extract nearly all the cosmological information without the additional computational expense of 3D cosmic shear. Meanwhile a large fraction of the information comes from small poorly understood scales in the matter power spectrum, that can lead to biases on measurements of cosmological parameters. In light of this, we define and compute a cosmology-independent measure of the bias due to imperfect knowledge of the power spectrum. For a Euclid-like survey, we find that the power spectrum must be known to an accuracy of less than 1% on scales with k = 7 h /Mpc This requirement is not absolute since the bias depends on the magnitude of modelling errors, where they occur in k-z space, and the correlation between them, all of which are specific to any particular model. We therefore compute the bias in several of the most likely modelling scenarios and introduce a general formalism and public code, RequiSim, to compute the expected bias from any non-linear model.

astro-ph.CO↗

arXiv2018

An Optimal-Dimensionality Sampling for Spin-$s$ Functions on the Sphere

For the representation of spin-$s$ band-limited functions on the sphere, we propose a sampling scheme with optimal number of samples equal to the number of degrees of freedom of the function in harmonic space. In comparison to the existing sampling designs, which require ${\sim}2L^2$ samples for the representation of spin-$s$ functions band-limited at $L$, the proposed scheme requires $N_o=L^2-s^2$ samples for the accurate computation of the spin-$s$ spherical harmonic transform~($s$-SHT). For the proposed sampling scheme, we also develop a method to compute the $s$-SHT. We place the samples in our design scheme such that the matrices involved in the computation of $s$-SHT are well-conditioned. We also present a multi-pass $s$-SHT to improve the accuracy of the transform. We also show the proposed sampling design exhibits superior geometrical properties compared to existing equiangular and Gauss-Legendre sampling schemes, and enables accurate computation of the $s$-SHT corroborated through numerical experiments.

astro-ph.IM↗

arXiv2018

Testing the Cosmic Shear Spatially-Flat Universe Approximation with Generalized Lensing and Shear Spectra

We introduce the Generalised Lensing and Shear Spectra GLaSS code which is available for download from https://github.com/astro-informatics/GLaSS It is a fast and flexible public code, written in Python, that computes generalized spherical cosmic shear spectra. The commonly used tomographic and spherical Bessel lensing spectra come as built-in run-mode options. GLaSS is integrated into the Cosmosis modular cosmological pipeline package. We outline several computational choices that accelerate the computation of cosmic shear power spectra. Using GLaSS, we test whether the assumption that using the lensing and projection kernels for a spatially-flat universe -- in a universe with a small amount of spatial curvature -- negligibly impacts the lensing spectrum. We refer to this assumption as The Spatially-Flat Universe Approximation, that has been implicitly assumed in all cosmic shear studies to date. We confirm that The Spatially-Flat Universe Approximation has a negligible impact on Stage IV cosmic shear experiments.

astro-ph.CO↗

arXiv2017

Online radio interferometric imaging: assimilating and discarding visibilities on arrival

The emerging generation of radio interferometric (RI) telescopes, such as the Square Kilometre Array (SKA), will acquire massive volumes of data and transition radio astronomy to a big-data era. The ill-posed inverse problem of imaging the raw visibilities acquired by RI telescopes will become significantly more computationally challenging, particularly in terms of data storage and computational cost. Current RI imaging methods, such as CLEAN, its variants, and compressive sensing approaches (sparse regularisation), have yielded excellent reconstruction fidelity. However, scaling these methods to big-data remains difficult if not impossible in some cases. All state-of-the-art methods in RI imaging lack the ability to process data streams as they are acquired during the data observation stage. Such approaches are referred to as online processing methods. We present an online sparse regularisation methodology for RI imaging. Image reconstruction is performed simultaneously with data acquisition, where observed visibilities are assimilated into the reconstructed image as they arrive and then discarded. Since visibilities are processed online, good reconstructions are recovered much faster than standard (offline) methods which cannot start until the data acquisition stage completes. Moreover, the online method provides additional computational savings and, most importantly, dramatically reduces data storage requirements. Theoretically, the reconstructed images are of the same fidelity as those recovered by the equivalent offline approach and, in practice, very similar reconstruction fidelity is achieved. We anticipate online imaging techniques, as proposed here, will be critical in scaling RI imaging to the emerging big-data era of radio astronomy.

astro-ph.IM↗