arXiv subjects

Sammy Sharief

Publications and source records attributed to Sammy Sharief.

3 recordsLinked to original sources

arXiv2026

MIRA: A Score for Conditional Distribution Accuracy and Model Comparison

We introduce Mira, a sample-based score for assessing the accuracy of a candidate conditional distribution using only joint samples from the true data-generating process. Relying on the principle that distributions coincide if they assign equal probability mass to all regions, we derive an analytic expression for the Mira statistic, whose average defines the Mira score. This formulation further allows us to compute theoretical reference values and uncertainty estimates when the candidate distribution matches the true one. This framework enables model comparison by quantifying the alignment between the conditional distribution of a candidate model and the true data generating process. Consequently, Mira enables Bayesian model comparison through direct posterior validation, bypassing the challenging evidence computation. We demonstrate its effectiveness across several toy problems and Bayesian inference tasks.

stat.ML↗

arXiv2025

Field-Level Comparison and Robustness Analysis of Cosmological N-body Simulations

We present the first field-level comparison of cosmological N-body simulations, considering various widely used codes: Abacus, CUBEP$^3$M, Enzo, Gadget, Gizmo, PKDGrav, and Ramses. Unlike previous comparisons focused on summary statistics, we conduct a comprehensive field-level analysis: evaluating statistical similarity, quantifying implications for cosmological parameter inference, and identifying the regimes in which simulations are consistent. We begin with a traditional comparison using the power spectrum, cross-correlation coefficient, and visual inspection of the matter field. We follow this with a statistical out-of-distribution (OOD) analysis to quantify distributional differences between simulations, revealing insights not captured by the traditional metrics. We then perform field-level simulation-based inference (SBI) using convolutional neural networks (CNNs), training on one simulation and testing on others, including a full hydrodynamic simulation for comparison. We identify several causes of OOD behavior and biased inference, finding that resolution effects, such as those arising from adaptive mesh refinement (AMR), have a significant impact. Models trained on non-AMR simulations fail catastrophically when evaluated on AMR simulations, introducing larger biases than those from hydrodynamic effects. Differences in resolution, even when using the same N-body code, likewise lead to biased inference. We attribute these failures to a CNN's sensitivity to small-scale fluctuations, particularly in voids and filaments, and demonstrate that appropriate smoothing brings the simulations into statistical agreement. Our findings motivate the need for careful data filtering and the use of field-level OOD metrics, such as PQMass, to ensure robust inference.

astro-ph.CO↗

arXiv2024

PQMass: Probabilistic Assessment of the Quality of Generative Models using Probability Mass Estimation

We propose a likelihood-free method for comparing two distributions given samples from each, with the goal of assessing the quality of generative models. The proposed approach, PQMass, provides a statistically rigorous method for assessing the performance of a single generative model or the comparison of multiple competing models. PQMass divides the sample space into non-overlapping regions and applies chi-squared tests to the number of data samples that fall within each region, giving a p-value that measures the probability that the bin counts derived from two sets of samples are drawn from the same multinomial distribution. PQMass does not depend on assumptions regarding the density of the true distribution, nor does it rely on training or fitting any auxiliary models. We evaluate PQMass on data of various modalities and dimensions, demonstrating its effectiveness in assessing the quality, novelty, and diversity of generated samples. We further show that PQMass scales well to moderately high-dimensional data and thus obviates the need for feature extraction in practical applications.

stat.ML↗