Searcharxiv⌕ Search

arXiv subjects

El Mahdi Khribch

Publications and source records attributed to El Mahdi Khribch.

3 recordsLinked to original sources

Convex Order Comparisons for Sub-Gamma Random Variables

Recent work has shown that sub-Gaussian random variables are dominated in convex order by a sharp multiple of a Gaussian. We study the analogous question for the sub-Gamma class underlying Bernstein's inequality, with the Laplace law as the majorant. Here the moment generating function is controlled by a Bernstein-type bound over a finite range of frequencies, rather than by a purely quadratic bound. We derive a variational formula for the optimal multiple, show that it is strictly larger than the natural scale $σ\vee α$, and prove that this value is sharp, being attained by an asymmetric two-point distribution. We then turn to the sub-exponential class, which has a quadratic bound as in the sub-Gaussian case, but only over a bounded range as in the sub-Gamma case. Interestingly, the sub-exponential class displays a different behavior: the optimal multiple is exactly $σ\vee α$, but is attained only when $α\leq σ$; when $α> σ$, the constant remains sharp, but equality cannot hold for any non-affine convex function. Both results rely on a sharp finite-range variant of the Kearns-Saul inequality, which is of independent interest.

math.PR↗

On importance sampling and independent Metropolis-Hastings with an unbounded weight function

Importance sampling and independent Metropolis-Hastings are among the fundamental building blocks of Monte Carlo methods. Both require a proposal distribution that globally approximates the target distribution, and pointwise evaluation of the Radon-Nikodym derivative of the target distribution relative to the proposal, also called the weight function. We study the bias of importance sampling and independent Metropolis-Hastings, without assuming that the weight function is bounded. We show that the common random numbers coupling of independent Metropolis-Hastings is maximal. Using that coupling, we derive polynomial bounds on the total variation distance of the chain to its target distribution. We further consider bias removal techniques using couplings, and provide conditions under which the resulting unbiased estimators have finite moments, and under which their efficiency is comparable to that of importance sampling. Experiments illustrate unbiased estimators of the inverse of a normalizing constant, estimators of nested expectations, and combination of importance sampling with robust mean estimation methods.

math.ST↗

Convergence of Statistical Estimators via Mutual Information Bounds

Recent advances in statistical learning theory have revealed profound connections between mutual information (MI) bounds, PAC-Bayesian theory, and Bayesian nonparametrics. This work introduces a novel mutual information bound for statistical models. The derived bound has wide-ranging applications in statistical inference. It yields improved contraction rates for fractional posteriors in Bayesian nonparametrics. It can also be used to study a wide range of estimation methods, such as variational inference or Maximum Likelihood Estimation (MLE). By bridging these diverse areas, this work advances our understanding of the fundamental limits of statistical inference and the role of information in learning from data. We hope that these results will not only clarify connections between statistical inference and information theory but also help to develop a new toolbox to study a wide range of estimators.

stat.ML↗