Searcharxiv⌕ Search

arXiv subjects

Yi Mao

Publications and source records attributed to Yi Mao.

At least 37 records · Page 2Linked to original sources
arXiv2025

Likelihood-free Model Selection in Cosmic Reionization with Three-dimensional Tomographic 21 cm Lightcone Images

We explore likelihood-free (aka simulation-based) Bayesian model selection to quantify model comparison analyses of reionisation scenarios. We iteratively train the 3D Convolutional Neural Network (CNN) on four toy EoR models based on 21cmFAST simulations with contrasting morphology to obtain summaries of the 21 cm lightcone. Within the pyDelfi framework, we replaced the Emcee sampler with MultiNest to integrate learnt posteriors and produce the Bayesian Evidence. We comfortably distinguish the model used to produce the mock data set in all cases. However, we struggle to produce accurate posterior distributions for outside-in reionisation models. After a variety of cross-checks and alternate analyses we discuss the flexibility of summarising models that differ from precisely the intended network training conditions as this should be more widely scrutinised before CNN can reliably analyse observed data.

astro-ph.IM↗
arXiv2024

Reionization relics in the cross-correlation between the Ly$α$ forest and 21 cm intensity mapping in the post-reionization era

The tumultuous effects of ultraviolet photons that source cosmic reionization, the subsequent compression and shock-heating of low-density regions, and the modulation of baryons in shallow potential wells induced by the passage of ionization fronts, collectively introduce perturbations to the evolution of the intergalactic medium in the post-reionization era. These enduring fluctuations persist deep into the post-reionization era, casting a challenge upon precision cosmology endeavors targeting tracers in this cosmic era. Simultaneously, these relics from reionization also present a unique opportunity to glean insights into the astrophysics that govern the epoch of reionization. In this work, we propose a first study of the cross-correlation of \lya forest and 21 cm intensity mapping, accounting for the repercussions of inhomogeneous reionization in the post-reionization era. We investigate the ability of SKA $\times$ DESI-like, SKA $\times$ MUST-like, and PUMA $\times$ MUST-like instrumental setups to achieve a high signal-to-noise ratio (SNR) in the redshift range $3.5 \leq z \leq 4$. Moreover, we assess how alterations in integration time, survey area, and reionization scenarios impact the SNR. Furthermore, we forecast the cross-correlation's potential to constrain cosmological parameters under varying assumptions: considering or disregarding reionization relics, marginalizing over reionization astrophysics, and assuming perfect knowledge of reionization. Notably, our findings underscore the remarkable capability of a futuristic PUMA $\times$ MUST-like setup, with a modest 100-hour integration time over a 100 sq. deg. survey, to constrain the ionization efficiency error to $σ_ζ= 3.42 $.

astro-ph.CO↗
arXiv2024

Reionization Parameter Inference from 3D Minkowski Functionals of the 21 cm Signals

The Minkowski Functionals (MFs), a set of topological summary statistics, have emerged as a powerful tool for extracting non-Gaussian information. We investigate the prospect of constraining the reionization parameters using the MFs of the 21 cm brightness temperature field from the epoch of reionization (EoR). Realistic effects, including thermal noise, synthesized beam, and foreground avoidance, are applied to the mock observations from the radio interferometric array experiments such as the Hydrogen Epoch of Reionization Array (HERA) and the Square Kilometre Array (SKA). We demonstrate that the MFs of the 21 cm signal measured with SKA-Low can be used to distinguish different reionization models, whereas the MF measurement with a HERA-like array cannot be made accurately enough. We further forecast the accuracies with which the MF measurements can place constraints on reionization parameters, using the standard MCMC analysis for parameter inference based on forward modeling. We find that for SKA-Low observation, MFs provide unbiased estimations of the reionization parameters with accuracies comparable to the power spectrum (PS) analysis. Furthermore, joint constraints using both MFs and PS can improve the constraint accuracies by up to $30\%$ compared to those with the PS alone. Nevertheless, the constraint accuracies can be degraded if the EoR window is shrunk with strong foreground avoidance. Our analysis demonstrates the promise of MFs as a set of summary statistics that extract complementary information from the 21 cm EoR field to the two-point statistics, which suggests a strong motivation for incorporating the MFs into the data analysis of future 21 cm observations.

astro-ph.CO↗
arXiv2024

Simulation-based Inference of Reionization Parameters from 3D Tomographic 21 cm Light-cone Images -- II: Application of Solid Harmonic Wavelet Scattering Transform

The information regarding how the intergalactic medium is reionized by astrophysical sources is contained in the tomographic three-dimensional 21 cm images from the epoch of reionization. In Zhao et al. (2022a) ("Paper I"), we demonstrated for the first time that density estimation likelihood-free inference (DELFI) can be applied efficiently to perform a Bayesian inference of the reionization parameters from the 21 cm images. Nevertheless, the 3D image data needs to be compressed into informative summaries as the input of DELFI by, e.g., a trained 3D convolutional neural network (CNN) as in Paper I (DELFI-3D CNN). Here in this paper, we introduce an alternative data compressor, the solid harmonic wavelet scattering transform (WST), which has a similar, yet fixed (i.e. no training), architecture to CNN, but we show that this approach (i.e. solid harmonic WST with DELFI) outperforms earlier analyses based on 3D 21 cm images using DELFI-3D CNN in terms of credible regions of parameters. Realistic effects, including thermal noise and residual foreground after removal, are also applied to the mock observations from the Square Kilometre Array (SKA). We show that under the same inference strategy using DELFI, the 21 cm image analysis with solid harmonic WST outperforms the 21 cm power spectrum analysis. This research serves as a proof of concept, demonstrating the potential to harness the strengths of WST and simulation-based inference to derive insights from future 21 cm light-cone image data.

astro-ph.IM↗
arXiv2024

Modeling the Nonlinear Power Spectrum in Low-redshift HI Intensity Mapping

We present a simulation-based framework to forecast the HI power spectrum on non-linear scales ($k\gtrsim 1\ {\rm Mpc^{-1}}$), as measured by interferometer arrays like MeerKAT in the low-redshift ($z\leq 1.0$) universe. Building on a galaxy-based HI mock catalog, we meticulously consider various factors, including the emission line profiles of HI discs and some observational settings, and explore their impacts on the HI power spectrum. While it is relatively insensitive to the profile shape of HI emission line at these scales, we identify a strong correlation with the profile width, that is, the Full Width at Half Maxima (FWHM, also known as $W_{\rm 50}$ in observations) in this work. By modeling the width function of $W_{50}$ as a function of $v_{\rm max}$, we assign each HI source a emission line profile and find that the resulting HI power spectrum is comparatively close to results from particles in the IllustrisTNG hydrodynamical simulation. After implementing $k$-space cuts matching the MeerKAT data, our prediction replicates the trend of the measurements obtained by MeerKAT at $z\approx 0.44$, though with a significantly lower amplitude. Utilizing a Monte Carlo Markov Chain sampling method, we constrain the parameter $A_{W_{\rm 50}}$ in the $W_{\rm 50}$ models and $Ω_{\rm HI}$ with the MeerKAT measurements and find that a strong degeneracy exists between these two parameters.

astro-ph.CO↗
arXiv2024

The long-lasting effect of X-ray preheating in the post-reionization intergalactic medium

X-ray photons can penetrate deep into the intergalactic medium (IGM), leading to preheating of the IGM prior to cosmic reionization. X-ray preheating wipes out some of the small-scale structures that would otherwise be present prior to the passage of an ionization front. Accurate modeling of the small-scale structure is vital to the post-reionization IGM since the small-scale structure is ultimately the dominant source of long-lasting relics from hydrogen reionization. However, the precise impact of X-ray preheating in the fossils from hydrogen reionization is highly uncertain. In this work, we explore and establish for the first time, the long-lasting impact of X-ray preheating in the post-reionization IGM via hydrodynamic simulations with high-mass resolution. We find that the addition of X-ray preheating astrophysics leads to an overall lesser impact of the effect of inhomogeneous reionization in the Lyman-$α$ forest -- depending on specific X-ray prescription -- at low redshifts ($z \sim 2$) with respect to a model with no X-ray preheating. However, at high redshifts ($z \sim 4$), our results indicate a strengthening of the relics of reionization in the Lyman-$α$ forest because the IGM becomes more transparent compared to the scenario with no preheating. Thus, the absence of X-ray preheating in Lyman-$α$ modeling can lead to a biased inference of cosmological parameters. Nevertheless, optimistically, the inclusion of X-ray preheating emerges as a promising novel avenue to probe the astrophysics of cosmic dawn.

astro-ph.CO↗
arXiv2023

Exploring Distributional Shifts in Large Language Models for Code Analysis

We systematically study how three large language models with code capabilities - CodeT5, Codex, and ChatGPT - generalize to out-of-domain data. We consider two fundamental applications - code summarization, and code generation. We split data into domains following its natural boundaries - by an organization, by a project, and by a module within the software project. We establish that samples from each new domain present all the models with a significant challenge of distribution shift. We study how established methods adapt models to better generalize to new domains. Our experiments show that while multitask learning alone is a reasonable baseline, combining it with few-shot finetuning on examples retrieved from training data can achieve very strong performance. Moreover, this solution can outperform direct finetuning for very low-data scenarios. Finally, we consider variations of this approach to create a more broadly applicable method to adapt to multiple domains at once. We find that for code generation, a model adapted to multiple domains simultaneously performs on par with those adapted to a single domain

cs.CL↗
arXiv2023

Reconstruction of Cosmological Initial Density Field with Observations from the Epoch of Reionization

Initial density distribution provides a basis for understanding the complete evolution of cosmological density fluctuations. While reconstruction in our local Universe exploits the observations of galaxy surveys with large volumes, observations of high-redshift galaxies are performed with a small field of view and therefore can hardly be used for reconstruction. Here we propose to reconstruct the initial density field using the H I 21 cm and CO line intensity maps from the epoch of reionization. Observations of these two intensity maps provide complementary information of the density field -- the H I 21 cm field is a proxy of matter distributions in the neutral regions, while the CO line intensity maps are sensitive to the high-density, star-forming regions that host the sources for reionization. Technically, we employ the conjugate gradient method and develop the machinery for minimizing the cost function for the intensity mapping observations. Analytical expressions for the gradient of cost function are derived explicitly. We show that the resimulated intensity maps match the input maps of mock observations using semi-numerical simulations of reionization with an rms error $\lesssim 7\%$ at all stages of reionization. This reconstruction is also robust at the same level of accuracy when a noise at the level of $\lesssim 1\%$ of the standard deviation is applied to each map. Our proof-of-concept work demonstrates the robustness of the reconstruction method, thereby providing an effective technique for reconstructing the cosmological initial density distribution from high-redshift observations.

astro-ph.CO↗
arXiv2023

Can Diffusion Model Conditionally Generate Astrophysical Images?

Generative adversarial networks (GANs) are frequently utilized in astronomy to construct an emulator of numerical simulations. Nevertheless, training GANs can prove to be a precarious task, as they are prone to instability and often lead to mode collapse problems. Conversely, the diffusion model also has the ability to generate high-quality data without adversarial training. It has shown superiority over GANs with regard to several natural image datasets. In this study, we undertake a quantitative comparison between the denoising diffusion probabilistic model (DDPM) and StyleGAN2 (one of the most robust types of GANs) via a set of robust summary statistics from scattering transform. In particular, we utilize both models to generate the images of 21 cm brightness temperature mapping, as a case study, conditionally based on astrophysical parameters that govern the process of cosmic reionization. Using our new Fréchet Scattering Distance (FSD) as the evaluation metric to quantitatively compare the sample distribution between generative models and simulations, we demonstrate that DDPM outperforms StyleGAN2 on varied sizes of training sets. Through Fisher forecasts, we demonstrate that on our datasets, StyleGAN2 exhibits mode collapses in varied ways, while DDPM yields a more robust generation. We also explore the role of classifier-free guidance in DDPM and show the preference for a non-zero guidance scale only when the training data is limited. Our findings indicate that the diffusion model presents a promising alternative to GANs in the generation of accurate images. These images can subsequently provide reliable parameter constraints, particularly in the realm of astrophysics.

astro-ph.IM↗
arXiv2023

RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation

The task of repository-level code completion is to continue writing the unfinished code based on a broader context of the repository. While for automated code completion tools, it is difficult to utilize the useful information scattered in different files. We propose RepoCoder, a simple, generic, and effective framework to address the challenge. It streamlines the repository-level code completion process by incorporating a similarity-based retriever and a pre-trained code language model in an iterative retrieval-generation pipeline. RepoCoder makes effective utilization of repository-level information for code completion and has the ability to generate code at various levels of granularity. Moreover, we propose a new benchmark RepoEval, which consists of the latest and high-quality real-world repositories covering line, API invocation, and function body completion scenarios. Experimental results indicate that RepoCoder significantly improves the In-File completion baseline by over 10% in all settings and consistently outperforms the vanilla retrieval-augmented code completion approach. Furthermore, we validate the effectiveness of RepoCoder through comprehensive analysis, providing valuable insights for future research. Our source code and benchmark are publicly available: https://github.com/microsoft/CodeT/tree/main/RepoCoder

cs.CL↗
arXiv2023

AMD-DBSCAN: An Adaptive Multi-density DBSCAN for datasets of extremely variable density

DBSCAN has been widely used in density-based clustering algorithms. However, with the increasing demand for Multi-density clustering, previous traditional DSBCAN can not have good clustering results on Multi-density datasets. In order to address this problem, an adaptive Multi-density DBSCAN algorithm (AMD-DBSCAN) is proposed in this paper. An improved parameter adaptation method is proposed in AMD-DBSCAN to search for multiple parameter pairs (i.e., Eps and MinPts), which are the key parameters to determine the clustering results and performance, therefore allowing the model to be applied to Multi-density datasets. Moreover, only one hyperparameter is required for AMD-DBSCAN to avoid the complicated repetitive initialization operations. Furthermore, the variance of the number of neighbors (VNN) is proposed to measure the difference in density between each cluster. The experimental results show that our AMD-DBSCAN reduces execution time by an average of 75% due to lower algorithm complexity compared with the traditional adaptive algorithm. In addition, AMD-DBSCAN improves accuracy by 24.7% on average over the state-of-the-art design on Multi-density datasets of extremely variable density, while having no performance loss in Single-density scenarios. Our code and datasets are available at https://github.com/AlexandreWANG915/AMD-DBSCAN.

cs.DB↗
arXiv2023

3D ScatterNet: Inference from 21 cm Light-cones

The Square Kilometre Array (SKA) will have the sensitivity to take the 3D light-cones of the 21 cm signal from the epoch of reionization. This signal, however, is highly non-Gaussian and can not be fully interpreted by the traditional statistic using power spectrum. In this work, we introduce the 3D ScatterNet that combines the normalizing flows with solid harmonic wavelet scattering transform, a 3D CNN featurizer with inductive bias, to perform implicit likelihood inference (ILI) from 21 cm light-cones. We show that 3D ScatterNet outperforms the ILI with a fine-tuned 3D CNN in the literature. It also reaches better performance than ILI with the power spectrum for varied light-cone effects and varied signal contaminations.

astro-ph.IM↗
arXiv2023

Multi-fidelity Emulator for Cosmological Large Scale 21 cm Lightcone Images: a Few-shot Transfer Learning Approach with GAN

Large-scale numerical simulations ($\gtrsim 500\rm{Mpc}$) of cosmic reionization are required to match the large survey volume of the upcoming Square Kilometre Array (SKA). We present a multi-fidelity emulation technique for generating large-scale lightcone images of cosmic reionization. We first train generative adversarial networks (GAN) on small-scale simulations and transfer that knowledge to large-scale simulations with hundreds of training images. Our method achieves high accuracy in generating lightcone images, as measured by various statistics with mostly percentage errors. This approach saves computational resources by 90% compared to conventional training methods. Our technique enables efficient and accurate emulation of large-scale images of the Universe.

astro-ph.CO↗
arXiv2023

Evaluating Summary Statistics with Mutual Information for Cosmological Inference

The ability to compress observational data and accurately estimate physical parameters relies heavily on informative summary statistics. In this paper, we introduce the use of mutual information (MI) as a means of evaluating the quality of summary statistics in inference tasks. MI can assess the sufficiency of summaries, and provide a quantitative basis for comparison. We propose to estimate MI using the Barber-Agakov lower bound and normalizing flow based variational distributions. To demonstrate the effectiveness of our method, we compare three different summary statistics (namely the power spectrum, bispectrum, and scattering transform) in the context of inferring reionization parameters from mock images of 21~cm observations with Square Kilometre Array. We find that this approach is able to correctly assess the informativeness of different summary statistics and allows us to select the optimal set of statistics for inference tasks.

astro-ph.CO↗
arXiv2023

Separating the memory of reionization from cosmology in the Ly$α$ forest power spectrum at the post-reionization era

It has been recently shown that the astrophysics of reionization can be extracted from the Ly$α$ forest power spectrum by marginalizing the memory of reionization over cosmological information. This impact of cosmic reionization on the Ly$α$ forest power spectrum can survive cosmological time scales because cosmic reionization, which is inhomogeneous, and subsequent shocks from denser regions can heat the gas in low-density regions to $\sim 3\times10^4$ K and compress it to mean-density. Current approach of marginalization over the memory of reionization, however, is not only model-dependent, based on the assumption of a specific reionization model, but also computationally expensive. Here we propose a simple analytical template for the impact of cosmic reionization, thereby treating it as a broadband systematic to be marginalized over for Bayesian inference of cosmological information from the Ly$α$ forest in a model-independent manner. This template performs remarkably well with an error of $\leq 6 \%$ at large scales $k \approx 0.19$ Mpc$^{-1}$ where the effect of the memory of reionization is important, and reproduces the broadband effect of the memory of reionization in the Ly$α$ forest correlation function, as well as the expected bias of cosmological parameters due to this systematic. The template can successfully recover the morphology of forecast errors in cosmological parameter space as expected when assuming a specific reionization model for marginalization purposes, with a slight overestimation of tens of per cent for the forecast errors on the cosmological parameters. We further propose a similar template for this systematic on the Ly$α$ forest 1D power spectrum.

astro-ph.CO↗
arXiv2023

A Semi-blind PCA-based Foreground Subtraction Method for 21 cm Intensity Mapping

The Principal Component Analysis (PCA) method and the Singular Value Decomposition (SVD) method are widely used for foreground subtraction in 21 cm intensity mapping experiments. We show their equivalence, and point out that the condition for completely clean separation of foregrounds and cosmic 21 cm signal using the PCA/SVD is unrealistic. We propose a PCA-based foreground subtraction method, dubbed "Singular Vector Projection (SVP)" method, which exploits a priori information of the left and/or right singular vectors of the foregrounds. We demonstrate with simulation tests that this new, semi-blind method can reduce the error of the recovered 21 cm signal by orders of magnitude, even if only the left and/or right singular vectors in the largest few modes are exploited. The SVP estimators provide a new, effective approach for 21 cm observations to remove foregrounds and uncover the physics in the cosmic 21 cm signal.

astro-ph.CO↗
arXiv2022

Generation-Augmented Query Expansion For Code Retrieval

Pre-trained language models have achieved promising success in code retrieval tasks, where a natural language documentation query is given to find the most relevant existing code snippet. However, existing models focus only on optimizing the documentation code pairs by embedding them into latent space, without the association of external knowledge. In this paper, we propose a generation-augmented query expansion framework. Inspired by the human retrieval process - sketching an answer before searching, in this work, we utilize the powerful code generation model to benefit the code retrieval task. Specifically, we demonstrate that rather than merely retrieving the target code snippet according to the documentation query, it would be helpful to augment the documentation query with its generation counterpart - generated code snippets from the code generation model. To the best of our knowledge, this is the first attempt that leverages the code generation model to enhance the code retrieval task. We achieve new state-of-the-art results on the CodeSearchNet benchmark and surpass the baselines significantly.

cs.SE↗
arXiv2022

HyperTuning: Toward Adapting Large Language Models without Back-propagation

Fine-tuning large language models for different tasks can be costly and inefficient, and even methods that reduce the number of tuned parameters still require full gradient-based optimization. We propose HyperTuning, a novel approach to model adaptation that uses a hypermodel to generate task-specific parameters for a fixed downstream model. We demonstrate a simple setup for hypertuning with HyperT5, a T5-based hypermodel that produces soft prefixes or LoRA parameters for a frozen T5 model from few-shot examples. We train HyperT5 in two stages: first, hyperpretraining with a modified conditional language modeling objective that trains a hypermodel to generate parameters; second, multi-task fine-tuning (MTF) on a large number of diverse language tasks. We evaluate HyperT5 on P3, MetaICL and Super-NaturalInstructions datasets, and show that it can effectively generate parameters for unseen tasks. Moreover, we show that using hypermodel-generated parameters as initializations for further parameter-efficient fine-tuning improves performance. HyperTuning can thus be a flexible and efficient way to leverage large language models for diverse downstream applications.

cs.CL↗