arXiv subjects

T. Lucas Makinen

Publications and source records attributed to T. Lucas Makinen.

14 recordsLinked to original sources

arXiv2026

Introducing sapphire: Towards Hybrid Physics-Informed, Data-Driven Modeling of Galaxy Formation

Semi-analytic models (SAMs) have been treating galaxy populations as dynamical systems for $\gtrsim50$ years, but their evolution equations remain poorly constrained. We introduce sapphire, a modular, automatically differentiable, GPU-accelerated SAM written in JAX. For the first time, we compute exact Jacobian and Hessian matrices of a galaxy formation SAM, using the Pandya et al. (2023) nonlinear differential equation system as an example. These allow efficient, interpretable local and global sensitivity analyses, which reveal that supernova energy loading is the key astrophysical parameter. We use gradient descent and Hamiltonian Monte Carlo (HMC) to perform comprehensive mock parameter recovery tests. These indicate that the $z=0$ stellar-to-halo-mass relation alone does not contain enough information to infer many astrophysical parameters. Using observations of star-forming galaxies from the MaNGA survey and the Behroozi et al. (2019) empirical model as one baseline, we derive multiple posteriors assuming different combinations of data, including $z=0$ interstellar medium gas fractions and metallicities. The inferred physical parameters suggest that galaxies self-regulate their star formation primarily through preventative rather than ejective feedback, though this remains uncertain due to the lack of satellite galaxies, black holes and multi-phase galactic atmosphere physics. Both Fisher and HMC forecasts demonstrate the potential of sapphire to enable precision inference for galaxy formation and cosmology in a hybrid physics-informed, data-driven way, but more work is needed to expand its library of models and methods. We make sapphire publicly available at https://github.com/virajpandya/sapphire.

astro-ph.GA↗

arXiv2026

The Degeneracy Distillery

When two or more parameters or labels produce similar data, they are degenerate, or hard to distinguish. Degeneracies render both label prediction and inverse problems difficult, since both machine learning algorithms and probabilistic samplers rely on the distinguishability of data and its gradients with respect to parameters. However, identifying degeneracies in physical models or real-world datasets can be elucidating about the choice of model or the underlying process that produces the data. We present the degeneracy distillery, a method that (1) detects and (2) resolves degenerate parameter combinations (a) automatically and (b) symbolically, from parameter-data (or parameter-simulation) pairs alone, through estimation and flattening of the Fisher information matrix. By exploring the information geometry of the likelihood, we characterize degeneracies as an intrinsic property of the physical model, requiring no realised data observation. We demonstrate our approach on a range of synthetic and real-world problems, discovering symbolic coordinate transformations that identify the combinations of parameters of a model which yield independent effects on the data. The resulting coordinates flatten the Fisher information in expectation globally, in contrast to posterior-based methods that flatten only at a single point, and substantially reduce the simulation budget required for downstream neural posterior estimation. In test cases we require up to $10\times$ fewer simulations for posterior estimation at matched validation calibration whilst simultaneously gaining physical insight on the system.

cs.LG↗

arXiv2026

A simulation-based inference of the Milky Way merger history

Accreted stars in the Milky Way (MW) preserve information about the progenitor galaxies where they formed in their chemical and kinematic properties. In this study, we use the chemo-dynamical signatures in the merger debris to approximate the posterior distribution of disrupted satellite properties at the time of infall. Adopting a simulation-based inference framework, we train an ensemble of normalizing flows using samples of merger debris from the Auriga suite of simulations of MW-like galaxies. Applying this methodology to a local sample of accreted stars in the MW, we infer the lookback times, stellar and halo masses, and halo mass merger ratios of several known accretion events in the Galaxy: Gaia Enceladus-Sausage (GES), Helmi streams, Heracles, I'itoi, LMS-1/Wukong, Sagittarius (Sgr), Sequoia and Thamnos. Our predictions align with the accretion time and mass estimates from the literature, and the expected relation between the progenitor stellar masses and debris metallicities across redshifts. The total stellar mass accreted from these events is predicted to be $2.2^{+1.1}_{-0.6}\times10^{9}~\rm{M_{\odot}}$, with GES and Sgr being the largest contributors. The predicted stellar mass accreted from fully disrupted progenitors in the stellar halo is $1.3^{+1.0}_{-0.5}\times10^{9}~\rm{M_{\odot}}$, which is consistent with previous mass measurements of this component. We provide a prediction for the evolution of the MW halo mass until the accretion of Sgr ($z\approx1$): specifically, we find that the mass growth of the Galaxy from the time of its first merger ($z\approx5$) to $z\approx2$ exceeds the total mass of the known progenitors accreted during that interval, suggesting the presence of unidentified substructures. Our estimate of the Galaxy halo mass after the Sgr merger, but prior to the accretion of the Magellanic Clouds, is $5.9^{+1.4}_{-1.1}\times10^{11}~\rm{M_{\odot}}$.

astro-ph.GA↗

arXiv2025

Hybrid Summary Statistics

We present a way to capture high-information posteriors from training sets that are sparsely sampled over the parameter space for robust simulation-based inference. In physical inference problems, we can often apply domain knowledge to define traditional summary statistics to capture some of the information in a dataset. We show that augmenting these statistics with neural network outputs to maximise the mutual information improves information extraction compared to neural summaries alone or their concatenation to existing summaries and makes inference robust in settings with low training data. We introduce 1) two loss formalisms to achieve this and 2) apply the technique to two different cosmological datasets to extract non-Gaussian parameter information.

stat.ML↗

arXiv2025

Optimal Neural Summarisation for Full-Field Weak Lensing Cosmological Implicit Inference

Traditionally, weak lensing cosmological surveys have been analyzed using summary statistics motivated by their analytically tractable likelihoods, or by their ability to access higher-order information, at the cost of requiring Simulation-Based Inference (SBI) approaches. While informative, these statistics are neither designed nor guaranteed to be statistically sufficient. With the rise of deep learning, it becomes possible to create summary statistics optimized to extract the full data information. We compare different neural summarization strategies proposed in the weak lensing literature, to assess which loss functions lead to theoretically optimal summary statistics to perform full-field inference. In doing so, we aim to provide guidelines and insights to the community to help guide future neural-based inference analyses. We design an experimental setup to isolate the impact of the loss function used to train neural networks. We have developed the sbi_lens JAX package, which implements an automatically differentiable lognormal wCDM LSST-Y10 weak lensing simulator. The explicit full-field posterior obtained using the Hamiltonian Monte Carlo sampler gives us a ground truth to which to compare different compression strategies. We provide theoretical insight into the loss functions used in the literature and show that some do not necessarily lead to sufficient statistics (e.g. Mean Square Error (MSE)), while those motivated by information theory (e.g. Variational Mutual Information Maximization (VMIM)) can. Our numerical experiments confirm these insights and show, in our simulated wCDM scenario, that the Figure of Merit (FoM) of an analysis using neural summaries optimized under VMIM achieves 100% of the reference Omega_c - sigma_8 full-field FoM, while an analysis using neural summaries trained under MSE achieves only 81% of the same reference FoM.

astro-ph.CO↗

arXiv2024

CHARM: Creating Halos with Auto-Regressive Multi-stage networks

To maximize the amount of information extracted from cosmological datasets, simulations that accurately represent these observations are necessary. However, traditional simulations that evolve particles under gravity by estimating particle-particle interactions (N-body simulations) are computationally expensive and prohibitive to scale to the large volumes and resolutions necessary for the upcoming datasets. Moreover, modeling the distribution of galaxies typically involves identifying virialized dark matter halos, which is also a time- and memory-consuming process for large N-body simulations, further exacerbating the computational cost. In this study, we introduce CHARM, a novel method for creating mock halo catalogs by matching the spatial, mass, and velocity statistics of halos directly from the large-scale distribution of the dark matter density field. We develop multi-stage neural spline flow-based networks to learn this mapping at redshift z=0.5 directly with computationally cheaper low-resolution particle mesh simulations instead of relying on the high-resolution N-body simulations. We show that the mock halo catalogs and painted galaxy catalogs have the same statistical properties as obtained from $N$-body simulations in both real space and redshift space. Finally, we use these mock catalogs for cosmological inference using redshift-space galaxy power spectrum, bispectrum, and wavelet-based statistics using simulation-based inference, performing the first inference with accelerated forward model simulations and finding unbiased cosmological constraints with well-calibrated posteriors. The code was developed as part of the Simons Collaboration on Learning the Universe and is publicly available at \url{https://github.com/shivampcosmo/CHARM}.

astro-ph.CO↗

arXiv2024

Hybrid summary statistics: neural weak lensing inference beyond the power spectrum

In inference problems, we often have domain knowledge which allows us to define summary statistics that capture most of the information content in a dataset. In this paper, we present a hybrid approach, where such physics-based summaries are augmented by a set of compressed neural summary statistics that are optimised to extract the extra information that is not captured by the predefined summaries. The resulting statistics are very powerful inputs to simulation-based or implicit inference of model parameters. We apply this generalisation of Information Maximising Neural Networks (IMNNs) to parameter constraints from tomographic weak gravitational lensing convergence maps to find summary statistics that are explicitly optimised to complement angular power spectrum estimates. We study several dark matter simulation resolutions in low- and high-noise regimes. We show that i) the information-update formalism extracts at least $3\times$ and up to $8\times$ as much information as the angular power spectrum in all noise regimes, ii) the network summaries are highly complementary to existing 2-point summaries, and iii) our formalism allows for networks with smaller, physically-informed architectures to match much larger regression networks with far fewer simulations needed to obtain asymptotically optimal inference.

astro-ph.CO↗

arXiv2024

LtU-ILI: An All-in-One Framework for Implicit Inference in Astrophysics and Cosmology

This paper presents the Learning the Universe Implicit Likelihood Inference (LtU-ILI) pipeline, a codebase for rapid, user-friendly, and cutting-edge machine learning (ML) inference in astrophysics and cosmology. The pipeline includes software for implementing various neural architectures, training schemata, priors, and density estimators in a manner easily adaptable to any research workflow. It includes comprehensive validation metrics to assess posterior estimate coverage, enhancing the reliability of inferred results. Additionally, the pipeline is easily parallelizable and is designed for efficient exploration of modeling hyperparameters. To demonstrate its capabilities, we present real applications across a range of astrophysics and cosmology problems, such as: estimating galaxy cluster masses from X-ray photometry; inferring cosmology from matter power spectra and halo point clouds; characterizing progenitors in gravitational wave signals; capturing physical dust parameters from galaxy colors and luminosities; and establishing properties of semi-analytic models of galaxy formation. We also include exhaustive benchmarking and comparisons of all implemented methods as well as discussions about the challenges and pitfalls of ML inference in astronomical sciences. All code and examples are made publicly available at https://github.com/maho3/ltu-ili.

astro-ph.IM↗

arXiv2024

Fishnets: Information-Optimal, Scalable Aggregation for Sets and Graphs

Set-based learning is an essential component of modern deep learning and network science. Graph Neural Networks (GNNs) and their edge-free counterparts Deepsets have proven remarkably useful on ragged and topologically challenging datasets. The key to learning informative embeddings for set members is a specified aggregation function, usually a sum, max, or mean. We propose Fishnets, an aggregation strategy for learning information-optimal embeddings for sets of data for both Bayesian inference and graph aggregation. We demonstrate that i) Fishnets neural summaries can be scaled optimally to an arbitrary number of data objects, ii) Fishnets aggregations are robust to changes in data distribution, unlike standard deepsets, iii) Fishnets saturate Bayesian information content and extend to regimes where MCMC techniques fail and iv) Fishnets can be used as a drop-in aggregation scheme within GNNs. We show that by adopting a Fishnets aggregation scheme for message passing, GNNs can achieve state-of-the-art performance versus architecture size on ogbn-protein data over existing benchmarks with a fraction of learnable parameters and faster training time.

cs.LG↗

arXiv2023

Field-level inference of cosmic shear with intrinsic alignments and baryons

We construct a field-based Bayesian Hierarchical Model for cosmic shear that includes, for the first time, the important astrophysical systematics of intrinsic alignments and baryon feedback, in addition to a gravity model. We add to the BORG-WL framework the tidal alignment and tidal torquing model (TATT) for intrinsic alignments and compare them with the non-linear alignment (NLA) model. With synthetic data, we have shown that adding intrinsic alignments and sampling the TATT parameters does not reduce the constraining power of the method and the field-based approach lifts the weak lensing degeneracy. We add baryon effects at the field level using the enthalpy gradient descent (EGD) model. This model displaces the dark matter particles without knowing whether they belong to a halo and allows for self-calibration of the model parameters, which are inferred from the data. We have also illustrated the effects of model misspecification for the baryons. The resulting model now contains the most important physical effects and is suitable for application to data.

astro-ph.CO↗

arXiv2023

Exoplanet atmosphere evolution: emulation with neural networks

Atmospheric mass-loss is known to play a leading role in sculpting the demographics of small, close-in exoplanets. Knowledge of how such planets evolve allows one to ``rewind the clock'' to infer the conditions in which they formed. Here, we explore the relationship between a planet's core mass and their atmospheric mass after protoplanetary disc dispersal by exploiting XUV photoevaporation as an evolutionary process. Historically, this style of inference problem would be computationally infeasible due to the large number of planet models required; however, we make use of a novel atmospheric evolution emulator which utilises neural networks to provide three orders of magnitude in speedup. First, we provide proof-of-concept for this emulator on a real problem, by inferring the initial atmospheric conditions to the TOI-270 multi-planet system. Using the emulator we find near-indistinguishable results when compared to original model. We then apply the emulator to the more complex inference problem, which aims to find the initial conditions for a sample of \textit{Kepler}, \textit{K2} and \textit{TESS} planets with well-constrained masses and radii. We demonstrate there is a relationship between core masses and the atmospheric mass that they retain after disc dispersal, and this trend is consistent with the `boil-off' scenario, in which close-in planets undergo dramatic atmospheric escape during disc dispersal. Thus, it appears the exoplanet population is consistent with the idea that close-in exoplanets initially acquired large massive atmospheres, the majority of which is lost during disc dispersal; before the final population is sculpted by atmospheric loss over 100~Myr to Gyr timescales.

astro-ph.EP↗

arXiv2022

The Cosmic Graph: Optimal Information Extraction from Large-Scale Structure using Catalogues

We present an implicit likelihood approach to quantifying cosmological information over discrete catalogue data, assembled as graphs. To do so, we explore cosmological parameter constraints using mock dark matter halo catalogues. We employ Information Maximising Neural Networks (IMNNs) to quantify Fisher information extraction as a function of graph representation. We a) demonstrate the high sensitivity of modular graph structure to the underlying cosmology in the noise-free limit, b) show that graph neural network summaries automatically combine mass and clustering information through comparisons to traditional statistics, c) demonstrate that networks can still extract information when catalogues are subject to noisy survey cuts, and d) illustrate how nonlinear IMNN summaries can be used as asymptotically optimal compressed statistics for Bayesian simulation-based inference. We reduce the area of joint $Ω_m, σ_8$ parameter constraints with small ($\sim$100 object) halo catalogues by a factor of 42 over the two-point correlation function, and demonstrate that the networks automatically combine mass and clustering information. This work utilises a new IMNN implementation over graph data in Jax, which can take advantage of either numerical or auto-differentiability. We also show that graph IMNNs successfully compress simulations away from the fiducial model at which the network is fitted, indicating a promising alternative to n-point statistics in catalogue simulation-based analyses.

astro-ph.CO↗

arXiv2021

Lossless, Scalable Implicit Likelihood Inference for Cosmological Fields

We present a comparison of simulation-based inference to full, field-based analytical inference in cosmological data analysis. To do so, we explore parameter inference for two cases where the information content is calculable analytically: Gaussian random fields whose covariance depends on parameters through the power spectrum; and correlated lognormal fields with cosmological power spectra. We compare two inference techniques: i) explicit field-level inference using the known likelihood and ii) implicit likelihood inference with maximally informative summary statistics compressed via Information Maximising Neural Networks (IMNNs). We find that a) summaries obtained from convolutional neural network compression do not lose information and therefore saturate the known field information content, both for the Gaussian covariance and the lognormal cases, b) simulation-based inference using these maximally informative nonlinear summaries recovers nearly losslessly the exact posteriors of field-level inference, bypassing the need to evaluate expensive likelihoods or invert covariance matrices, and c) even for this simple example, implicit, simulation-based likelihood incurs a much smaller computational cost than inference with an explicit likelihood. This work uses a new IMNNs implementation in $\texttt{Jax}$ that can take advantage of fully-differentiable simulation and inference pipeline. We also demonstrate that a single retraining of the IMNN summaries effectively achieves the theoretically maximal information, enhancing the robustness to the choice of fiducial model where the IMNN is trained.

astro-ph.CO↗

arXiv2021

deep21: a Deep Learning Method for 21cm Foreground Removal

We seek to remove foreground contaminants from 21cm intensity mapping observations. We demonstrate that a deep convolutional neural network (CNN) with a UNet architecture and three-dimensional convolutions, trained on simulated observations, can effectively separate frequency and spatial patterns of the cosmic neutral hydrogen (HI) signal from foregrounds in the presence of noise. Cleaned maps recover cosmological clustering statistics within 10% at all relevant angular scales and frequencies. This amounts to a reduction in prediction variance of over an order of magnitude on small angular scales ($\ell > 300$), and improved accuracy for small radial scales ($k_{\parallel} > 0.17\ \rm h\ Mpc^{-1})$ compared to standard Principal Component Analysis (PCA) methods. We estimate posterior confidence intervals for the network's prediction by training an ensemble of UNets. Our approach demonstrates the feasibility of analyzing 21cm intensity maps, as opposed to derived summary statistics, for upcoming radio experiments, as long as the simulated foreground model is sufficiently realistic. We provide the code used for this analysis on Github https://github.com/tlmakinen/deep21 as well as a browser-based tutorial for the experiment and UNet model via the accompanying http://bit.ly/deep21-colab Colab notebook.

astro-ph.CO↗