arXiv subjects

Shirley Ho

Publications and source records attributed to Shirley Ho.

At least 73 records · Page 4Linked to original sources

arXiv2022

HIFlow: Generating Diverse HI Maps and Inferring Cosmology while Marginalizing over Astrophysics using Normalizing Flows

A wealth of cosmological and astrophysical information is expected from many ongoing and upcoming large-scale surveys. It is crucial to prepare for these surveys now and develop tools that can efficiently extract most information. We present HIFlow: a fast generative model of the neutral hydrogen (HI) maps that is conditioned only on cosmology ($Ω_{m}$ and $σ_{8}$) and designed using a class of normalizing flow models, the Masked Autoregressive Flow (MAF). HIFlow is trained on the state-of-the-art simulations from the Cosmology and Astrophysics with MachinE Learning Simulations (CAMELS) project. HIFlow has the ability to generate realistic diverse maps without explicitly incorporating the expected 2D maps structure into the flow as an inductive bias. We find that HIFlow is able to reproduce the CAMELS average and standard deviation HI power spectrum (Pk) within a factor of $\lesssim$ 2, scoring a very high $R^{2} > 90\%$. By inverting the flow, HIFlow provides a tractable high-dimensional likelihood for efficient parameter inference. We show that the conditional HIFlow on cosmology is successfully able to marginalize over astrophysics at the field level, regardless of the stellar and AGN feedback strengths. This new tool represents a first step toward a more powerful parameter inference, maximizing the scientific return of future HI surveys, and opening a new avenue to minimize the loss of complex information due to data compression down to summary statistics.

astro-ph.CO↗

arXiv2022

TNT: Vision Transformer for Turbulence Simulations

Turbulence is notoriously difficult to model due to its multi-scale nature and sensitivity to small perturbations. Classical solvers of turbulence simulation generally operate on finer grids and are computationally inefficient. In this paper, we propose the Turbulence Neural Transformer (TNT), which is a learned simulator based on the transformer architecture, to predict turbulent dynamics on coarsened grids. TNT extends the positional embeddings of vanilla transformers to a spatiotemporal setting to learn the representation in the 3D time-series domain, and applies Temporal Mutual Self-Attention (TMSA), which captures adjacent dependencies, to extract deep and dynamic features. TNT is capable of generating comparatively long-range predictions stably and accurately, and we show that TNT outperforms the state-of-the-art U-net simulator on several metrics. We also test the model performance with different components removed and evaluate robustness to different initial conditions. Although more experiments are needed, we conclude that TNT has great potential to outperform existing solvers and generalize to additional simulation datasets.

physics.flu-dyn↗

arXiv2022

Simple lessons from complex learning: what a neural network model learns about cosmic structure formation

We train a neural network model to predict the full phase space evolution of cosmological N-body simulations. Its success implies that the neural network model is accurately approximating the Green's function expansion that relates the initial conditions of the simulations to its outcome at later times in the deeply nonlinear regime. We test the accuracy of this approximation by assessing its performance on well understood simple cases that have either known exact solutions or well understood expansions. These scenarios include spherical configurations, isolated plane waves, and two interacting plane waves: initial conditions that are very different from the Gaussian random fields used for training. We find our model generalizes well to these well understood scenarios, demonstrating that the networks have inferred general physical principles and learned the nonlinear mode couplings from the complex, random Gaussian training data. These tests also provide a useful diagnostic for finding the model's strengths and weaknesses, and identifying strategies for model improvement. We also test the model on initial conditions that contain only transverse modes, a family of modes that differ not only in their phases but also in their evolution from the longitudinal growing modes used in the training set. When the network encounters these initial conditions that are orthogonal to the training set, the model fails completely. In addition to these simple configurations, we evaluate the model's predictions for the density, displacement, and momentum power spectra with standard initial conditions for N-body simulations. We compare these summary statistics against N-body results and an approximate, fast simulation method called COLA. Our model achieves percent level accuracy at nonlinear scales of $k\sim 1\ \mathrm{Mpc}^{-1}\, h$, representing a significant improvement over COLA.

astro-ph.CO↗

arXiv2022

Field Level Neural Network Emulator for Cosmological N-body Simulations

We build a field level emulator for cosmic structure formation that is accurate in the nonlinear regime. Our emulator consists of two convolutional neural networks trained to output the nonlinear displacements and velocities of N-body simulation particles based on their linear inputs. Cosmology dependence is encoded in the form of style parameters at each layer of the neural network, enabling the emulator to effectively interpolate the outcomes of structure formation between different flat $Λ$CDM cosmologies over a wide range of background matter densities. The neural network architecture makes the model differentiable by construction, providing a powerful tool for fast field level inference. We test the accuracy of our method by considering several summary statistics, including the density power spectrum with and without redshift space distortions, the displacement power spectrum, the momentum power spectrum, the density bispectrum, halo abundances, and halo profiles with and without redshift space distortions. We compare these statistics from our emulator with the full N-body results, the COLA method, and a fiducial neural network with no cosmological dependence. We find our emulator gives accurate results down to scales of $k \sim 1\ \mathrm{Mpc}^{-1}\, h$, representing a considerable improvement over both COLA and the fiducial neural network. We also demonstrate that our emulator generalizes well to initial conditions containing primordial non-Gaussianity, without the need for any additional style parameters or retraining.

astro-ph.CO↗

arXiv2022

Predicting the Thermal Sunyaev-Zel'dovich Field using Modular and Equivariant Set-Based Neural Networks

Theoretical uncertainty limits our ability to extract cosmological information from baryonic fields such as the thermal Sunyaev-Zel'dovich (tSZ) effect. Being sourced by the electron pressure field, the tSZ effect depends on baryonic physics that is usually modeled by expensive hydrodynamic simulations. We train neural networks on the IllustrisTNG-300 cosmological simulation to predict the continuous electron pressure field in galaxy clusters from gravity-only simulations. Modeling clusters is challenging for neural networks as most of the gas pressure is concentrated in a handful of voxels and even the largest hydrodynamical simulations contain only a few hundred clusters that can be used for training. Instead of conventional convolutional neural net (CNN) architectures, we choose to employ a rotationally equivariant DeepSets architecture to operate directly on the set of dark matter particles. We argue that set-based architectures provide distinct advantages over CNNs. For example, we can enforce exact rotational and permutation equivariance, incorporate existing knowledge on the tSZ field, and work with sparse fields as are standard in cosmology. We compose our architecture with separate, physically meaningful modules, making it amenable to interpretation. For example, we can separately study the influence of local and cluster-scale environment, determine that cluster triaxiality has negligible impact, and train a module that corrects for mis-centering. Our model improves by 70 % on analytic profiles fit to the same simulation data. We argue that the electron pressure field, viewed as a function of a gravity-only simulation, has inherent stochasticity, and model this property through a conditional-VAE extension to the network. This modification yields further improvement by 7 %, it is limited by our small training set however. (abridged)

astro-ph.CO↗

arXiv2022

Cosmological Information in the Marked Power Spectrum of the Galaxy Field

Marked power spectra are two-point statistics of a marked field obtained by weighting each location with a function that depends on the local density around that point. We consider marked power spectra of the galaxy field in redshift space that up-weight low density regions, and perform a Fisher matrix analysis to assess the information content of this type of statistics using the Molino mock catalogs built upon the Quijote simulations. We identify four different ways to up-weight the galaxy field, and compare the Fisher information contained in their marked power spectra to the one of the standard galaxy power spectrum, when considering monopole and quadrupole of each statistic. Our results show that each of the four marked power spectra can tighten the standard power spectrum constraints on the cosmological parameters $Ω_{\rm m}$, $Ω_{\rm b}$, $h$, $n_s$, $M_ν$ by $15-25\%$ and on $σ_8$ by a factor of 2. The same analysis performed by combining the standard and four marked power spectra shows a substantial improvement compared to the power spectrum constraints that is equal to a factor of 6 for $σ_8$ and $2.5-3$ for the other parameters. Our constraints may be conservative, since the galaxy number density in the Molino catalogs is much lower than the ones in future galaxy surveys, which will allow them to probe lower density regions of the large-scale structure.

astro-ph.CO↗

arXiv2022

An exploration of the properties of cluster profiles for the thermal and kinetic Sunyaev-Zel'dovich effects

With the advent of high-resolution, low-noise CMB measurements, the ability to extract cosmological information from thermal Sunyaev-Zel'dovich effect and kinetic Sunyaev-Zel'dovich effect will be limited not by statistical uncertainties but rather by systematic and theoretical uncertainties. The theoretical uncertainty is driven by the lack of knowledge about the electron pressure and density. Thus we explore the electron pressure and density distributions in the IllustrisTNG hydrodynamical simulations, and we demonstrate that the cluster properties exhibit a strong dependence on the halo concentration -- providing some of the first evidence of cluster assembly bias in the electron pressure and density. Further, our work shows evidence for a broken power-law mass dependence, with lower pressure in lower mass halos than previous work and a strong evolution with mass of the radial correlations in the electron density and pressure. Both of these effects highlight the differing impact of active galactic nuclei and supernova feedback on the gas in galaxy groups compared to massive clusters. We verified that we see qualitatively similar features in the SIMBA hydro-dynamical simulations, suggesting these effects could be generic features. Finally, we provide a parametric formula for the electron pressure and density profile as a function of dark matter halo mass, halo concentration, and redshift. These fitting formulae can reproduce the distribution of density and pressure of clusters and will be useful in extracting cosmological information from upcoming CMB surveys.

astro-ph.CO↗

arXiv2022

Learned Coarse Models for Efficient Turbulence Simulation

Turbulence simulation with classical numerical solvers requires high-resolution grids to accurately resolve dynamics. Here we train learned simulators at low spatial and temporal resolutions to capture turbulent dynamics generated at high resolution. We show that our proposed model can simulate turbulent dynamics more accurately than classical numerical solvers at the comparably low resolutions across various scientifically relevant metrics. Our model is trained end-to-end from data and is capable of learning a range of challenging chaotic and turbulent dynamics at low resolution, including trajectories generated by the state-of-the-art Athena++ engine. We show that our simpler, general-purpose architecture outperforms various more specialized, turbulence-specific architectures from the learned turbulence simulation literature. In general, we see that learned simulators yield unstable trajectories; however, we show that tuning training noise and temporal downsampling solves this problem. We also find that while generalization beyond the training distribution is a challenge for learned models, training noise, added loss constraints, and dataset augmentation can help. Broadly, we conclude that our learned simulator outperforms traditional solvers run on coarser grids, and emphasize that simple design choices can offer stability and robust generalization.

physics.flu-dyn↗

arXiv2022

Super-resolving Dark Matter Halos using Generative Deep Learning

Generative deep learning methods built upon Convolutional Neural Networks (CNNs) provide a great tool for predicting non-linear structure in cosmology. In this work we predict high resolution dark matter halos from large scale, low resolution dark matter only simulations. This is achieved by mapping lower resolution to higher resolution density fields of simulations sharing the same cosmology, initial conditions and box-sizes. To resolve structure down to a factor of 8 increase in mass resolution, we use a variation of U-Net with a conditional GAN, generating output that visually and statistically matches the high resolution target extremely well. This suggests that our method can be used to create high resolution density output over Gpc/h box-sizes from low resolution simulations with negligible computational effort.

astro-ph.CO↗

arXiv2022

Wavelet Moments for Cosmological Parameter Estimation

Extracting non-Gaussian information from the non-linear regime of structure formation is key to fully exploiting the rich data from upcoming cosmological surveys probing the large-scale structure of the universe. However, due to theoretical and computational complexities, this remains one of the main challenges in analyzing observational data. We present a set of summary statistics for cosmological matter fields based on 3D wavelets to tackle this challenge. These statistics are computed as the spatial average of the complex modulus of the 3D wavelet transform raised to a power $q$ and are therefore known as invariant wavelet moments. The 3D wavelets are constructed to be radially band-limited and separable on a spherical polar grid and come in three types: isotropic, oriented, and harmonic. In the Fisher forecast framework, we evaluate the performance of these summary statistics on matter fields from the Quijote suite, where they are shown to reach state-of-the-art parameter constraints on the base $Λ$CDM parameters, as well as the sum of neutrino masses. We show that we can improve constraints by a factor 5 to 10 in all parameters with respect to the power spectrum baseline.

astro-ph.CO↗

arXiv2022

Rediscovering orbital mechanics with machine learning

We present an approach for using machine learning to automatically discover the governing equations and hidden properties of real physical systems from observations. We train a "graph neural network" to simulate the dynamics of our solar system's Sun, planets, and large moons from 30 years of trajectory data. We then use symbolic regression to discover an analytical expression for the force law implicitly learned by the neural network, which our results showed is equivalent to Newton's law of gravitation. The key assumptions that were required were translational and rotational equivariance, and Newton's second and third laws of motion. Our approach correctly discovered the form of the symbolic force law. Furthermore, our approach did not require any assumptions about the masses of planets and moons or physical constants. They, too, were accurately inferred through our methods. Though, of course, the classical law of gravitation has been known since Isaac Newton, our result serves as a validation that our method can discover unknown laws and hidden properties from observed data. More broadly this work represents a key step toward realizing the potential of machine learning for accelerating scientific discovery.

astro-ph.EP↗

arXiv2021

A New Classification Model for the ZTF Catalog of Periodic Variable Stars

Using the second data release from the Zwicky Transient Facility (ZTF, Bellm et al. 2019), Chen et al. (2020) created a ZTF Catalog of Periodic Variable Stars (ZTF CPVS) of 781, 602 periodic variables stars (PVSs) with 11 class labels. Here, we provide a new classification model of PVSs in the ZTF CPVS using a convolutional variational autoencoder and hierarchical random forest. We cross-match the sky-coordinate of PVSs in the ZTF CPVS with those presented in the SIMBAD catalog. We identify non-stellar objects that are not previously classified, including extragalactic objects such as Quasi-Stellar Objects, Active Galactic Nuclei, supernovae and planetary nebulae. We then create a new labelled training set with 13 classes in two levels. We obtain a reasonable level of completeness (> 90 %) for certain classes of PVSs, although we have poorer completeness in other classes (~ 40 % in some cases). Our new labels for the ZTF CPVS are available via Zenodo Cheung et al. (2021).

astro-ph.IM↗

arXiv2021

Searching for Anomalies in the ZTF Catalog of Periodic Variable Stars

Periodic variables illuminate the physical processes of stars throughout their lifetime. Wide-field surveys continue to increase our discovery rates of periodic variable stars. Automated approaches are essential to identify interesting periodic variable stars for multi-wavelength and spectroscopic follow-up. Here, we present a novel unsupervised machine learning approach to hunt for anomalous periodic variables using phase-folded light curves presented in the Zwicky Transient Facility Catalogue of Periodic Variable Stars by \citet{Chen_2020}. We use a convolutional variational autoencoder to learn a low dimensional latent representation, and we search for anomalies within this latent dimension via an isolation forest. We identify anomalies with irregular variability. Most of the top anomalies are likely highly variable Red Giants or Asymptotic Giant Branch stars concentrated in the Milky Way galactic disk; a fraction of the identified anomalies are more consistent with Young Stellar Objects. Detailed spectroscopic follow-up observations are encouraged to reveal the nature of these anomalies.

astro-ph.SR↗

arXiv2021

A Convolutional Autoencoder-Based Pipeline for Anomaly Detection and Classification of Periodic Variables

The periodic pulsations of stars teach us about their underlying physical process. We present a convolutional autoencoder-based pipeline as an automatic approach to search for out-of-distribution anomalous periodic variables within The Zwicky Transient Facility Catalog of Periodic Variable Stars (ZTF CPVS). We use an isolation forest to rank each periodic variable by its anomaly score. Our overall most anomalous events have a unique physical origin: they are mostly highly variable and irregular evolved stars. Multiwavelength data suggest that they are most likely Red Giant or Asymptotic Giant Branch stars concentrated in the Milky Way galactic disk. Furthermore, we show how the learned latent features can be used for the classification of periodic variables through a hierarchical random forest. This novel semi-supervised approach allows astronomers to identify the most anomalous events within a given physical class, significantly increasing the potential for scientific discovery.

astro-ph.SR↗

arXiv2021

The High Latitude Spectroscopic Survey on the Nancy Grace Roman Space Telescope

The Nancy Grace Roman Space Telescope will conduct a High Latitude Spectroscopic Survey (HLSS) over a large volume at high redshift, using the near-IR grism (1.0-1.93 $μ$m, $R=435-865$) and the 0.28 deg$^2$ wide field camera. We present a reference HLSS which maps 2000 deg$^2$ and achieves an emission line flux limit of 10$^{-16}$ erg/s/cm$^2$ at 6.5$σ$, requiring $\sim$0.6 yrs of observing time. We summarize the flowdown of the Roman science objectives to the science and technical requirements of the HLSS. We construct a mock redshift survey over the full HLSS volume by applying a semi-analytic galaxy formation model to a cosmological N-body simulation, and use this mock survey to create pixel-level simulations of 4 deg$^2$ of HLSS grism spectroscopy. We find that the reference HLSS would measure $\sim$ 10 million H$α$ galaxy redshifts that densely map large scale structure at $z=1-2$ and 2 million [OIII] galaxy redshifts that sparsely map structures at $z=2-3$. We forecast the performance of this survey for measurements of the cosmic expansion history with baryon acoustic oscillations and the growth of large scale structure with redshift space distortions. We also study possible deviations from the reference design, and find that a deep HLSS at $f_{\rm line}>7\times10^{-17}$erg/s/cm$^2$ over 4000 deg$^2$ (requiring $\sim$1.5 yrs of observing time) provides the most compelling stand-alone constraints on dark energy from Roman alone. This provides a useful reference for future optimizations. The reference survey, simulated data sets, and forecasts presented here will inform community decisions on the final scope and design of the Roman HLSS.

astro-ph.CO↗

arXiv2021

Detecting Neutrino Mass by Combining Matter Clustering, Halos, and Voids

We quantify the information content of the non-linear matter power spectrum, the halo mass function, and the void size function, using the Quijote $N$-body simulations. We find that these three statistics exhibit very different degeneracies amongst the cosmological parameters, and thus the combination of all three probes enables the breaking of degeneracies, in turn yielding remarkably tight constraints. We perform a Fisher analysis using the full covariance matrix, including all auto- and cross-correlations, finding that this increases the information content for neutrino mass compared to a correlation-free analysis. The multiplicative improvement of the constraints on the cosmological parameters obtained by combining all three probes compared to using the power spectrum alone are: 137, 5, 8, 20, 10, and 43, for $Ω_m$, $Ω_b$, $h$, $n_s$, $σ_8$, and $M_ν$, respectively. The marginalized error on the sum of the neutrino masses is $σ(M_ν)=0.018\,{\rm eV}$ for a cosmological volume of $1\,(h^{-1}{\rm Gpc})^3$, using $k_{\max}=0.5\,h{\rm Mpc}^{-1}$, and without CMB priors. We note that this error is an underestimate insomuch as we do not consider super-sample covariance, baryonic effects, and realistic survey noises and systematics. On the other hand, it is an overestimate insomuch as our cuts and binning are suboptimal due to restrictions imposed by the simulation resolution. Given upcoming galaxy surveys will observe volumes spanning $\sim 100\,(h^{-1}{\rm Gpc})^3$, this presents a promising new avenue to measure neutrino mass without being restricted by the need for accurate knowledge of the optical depth, which is required for CMB-based measurements. Furthermore, the improved constraints on other cosmological parameters, notably $Ω_m$, may also be competitive with CMB-based measurements.

astro-ph.CO↗

arXiv2021

The CAMELS project: Cosmology and Astrophysics with MachinE Learning Simulations

We present the Cosmology and Astrophysics with MachinE Learning Simulations --CAMELS-- project. CAMELS is a suite of 4,233 cosmological simulations of $(25~h^{-1}{\rm Mpc})^3$ volume each: 2,184 state-of-the-art (magneto-)hydrodynamic simulations run with the AREPO and GIZMO codes, employing the same baryonic subgrid physics as the IllustrisTNG and SIMBA simulations, and 2,049 N-body simulations. The goal of the CAMELS project is to provide theory predictions for different observables as a function of cosmology and astrophysics, and it is the largest suite of cosmological (magneto-)hydrodynamic simulations designed to train machine learning algorithms. CAMELS contains thousands of different cosmological and astrophysical models by way of varying $Ω_m$, $σ_8$, and four parameters controlling stellar and AGN feedback, following the evolution of more than 100 billion particles and fluid elements over a combined volume of $(400~h^{-1}{\rm Mpc})^3$. We describe the simulations in detail and characterize the large range of conditions represented in terms of the matter power spectrum, cosmic star formation rate density, galaxy stellar mass function, halo baryon fractions, and several galaxy scaling relations. We show that the IllustrisTNG and SIMBA suites produce roughly similar distributions of galaxy properties over the full parameter space but significantly different halo baryon fractions and baryonic effects on the matter power spectrum. This emphasizes the need for marginalizing over baryonic effects to extract the maximum amount of information from cosmological surveys. We illustrate the unique potential of CAMELS using several machine learning applications, including non-linear interpolation, parameter estimation, symbolic regression, data generation with Generative Adversarial Networks (GANs), dimensionality reduction, and anomaly detection.

astro-ph.CO↗

arXiv2021

The Quijote simulations

The Quijote simulations are a set of 44,100 full N-body simulations spanning more than 7,000 cosmological models in the $\{Ω_{\rm m}, Ω_{\rm b}, h, n_s, σ_8, M_ν, w \}$ hyperplane. At a single redshift the simulations contain more than 8.5 trillions of particles over a combined volume of 44,100 $(h^{-1}{\rm Gpc})^3$; each simulation follow the evolution of $256^3$, $512^3$ or $1024^3$ particles in a box of $1~h^{-1}{\rm Gpc}$ length. Billions of dark matter halos and cosmic voids have been identified in the simulations, whose runs required more than 35 million core hours. The Quijote simulations have been designed for two main purposes: 1) to quantify the information content on cosmological observables, and 2) to provide enough data to train machine learning algorithms. In this paper we describe the simulations and show a few of their applications. We also release the Petabyte of data generated, comprising hundreds of thousands of simulation snapshots at multiple redshifts, halo and void catalogs, together with millions of summary statistics such as power spectra, bispectra, correlation functions, marked power spectra, and estimated probability density functions.

astro-ph.CO↗