SearcharxivSearch

arXiv subjects

Yvik Swan

Publications and source records attributed to Yvik Swan.

At least 19 recordsLinked to original sources

Edgeworth expansion on Wiener chaos

Consider $F$ an element of the $p$-th Wiener chaos $\WW_p$, and denote by $\prob_F$ its law. For a positive integer $m$, let $\boldsymbol{\gamma}_{F,m}$ be the Radon measure with density $x \mapsto \frac{e^{-x^2/2}}{\sqrt{2\pi}} \left(1 + \sum_{k=3}^{4m-1} \frac{\E[H_k(F)]}{k!}\, H_k(x)\right)$, where $H_k$ is the $k$-th Hermite polynomial. The main goal of this article is to prove that the total variation distance between $\prob_F$ and $\boldsymbol{\gamma}_{F,m}$ is of order $\Var(\Gamma(F,F))^{({m+1})/{2}}$, where $\Gamma(F,F)$ denotes the carr\'e-du-champ operator of $F$. The variance of $\Gamma(F,F)$ is known to govern Gaussian fluctuations and can be bounded from above by $\kappa_4(F)$, the fourth cumulant of $F$, as established in the seminal work \cite{NP2009a}. Our result thus provides a genuine Edgeworth expansion in the setting of central convergence on Wiener chaoses. In this context, the quantity $\Var(\Gamma(F,F))$ plays the role of the small parameter that governs the accuracy of the approximation, in the same way that $1/\sqrt{n}$ does in the classical central limit theorem. To the best of our knowledge, our work is the first to establish Edgeworth expansions for Wiener chaoses in full generality and at arbitrary order, together with explicit remainder bounds that systematically improve with the order of the expansion--exactly as one would expect from an Edgeworth approximation. Our results apply verbatim to every situation where a central limit theorem is available for chaos elements, since no structural assumption is required beyond belonging to a fixed Wiener chaos. As a byproduct, we recover the celebrated optimal fourth moment theorem from \cite{NP2015} by combining the expansions at the first and second orders, with sharper quantitative bounds. Previous works on Edgeworth expansions for Wiener chaoses were essentially restricted to the first order.

math.PR

Stein's method for Fr\'echet approximation: a regularly varying functions approach

We develop a variant of Stein's method of comparison of generators to bound the Kolmogorov, total variation, and Wasserstein-1 distances between distributions on the real line. Our discrepancy is expressed in terms of the ratio of reverse hazard rates; it therefore remains tractable even when density derivatives are intractable. Our main application concerns the approximation of normalized extremes by Fr\'echet laws. In this setting, the new discrepancy provides a quantitative measure of distributional proximity in terms of the average regular variation at infinity of the underlying cumulative distribution function. We illustrate the approach through explicit computations for maxima of Pareto, Cauchy, and Burr~XII distributions. Our new discrepancy also opens the way to statistical applications which we outline.

math.PR

Split Conformal Prediction under Data Contamination

Conformal prediction is a non-parametric technique for constructing prediction intervals or sets from arbitrary predictive models under the assumption that the data is exchangeable. It is popular as it comes with theoretical guarantees on the marginal coverage of the prediction sets and the split conformal prediction variant has a very low computational cost compared to model training. We study the robustness of split conformal prediction in a data contamination setting, where we assume a small fraction of the calibration scores are drawn from a different distribution than the bulk. We quantify the impact of the corrupted data on the coverage and efficiency of the constructed sets when evaluated on "clean" test points, and verify our results with numerical experiments. Moreover, we propose an adjustment in the classification setting which we call Contamination Robust Conformal Prediction, and verify the efficacy of our approach using both synthetic and real datasets.

stat.ML

Stein's Method of Moments on the Sphere

We use Stein characterizations to obtain new moment-type estimators for the parameters of three classical spherical distributions (namely the Fisher-Bingham, the von Mises-Fisher, and the Watson distributions) in the i.i.d. case. This leads to explicit estimators which have good asymptotic properties (close to efficiency) and therefore lead to interesting alternatives to classical maximum likelihood methods or more recent score matching estimators. We perform competitive simulation studies to assess the quality of the new estimators. Finally, the practical relevance of our estimators is illustrated on a real data application in spherical latent representations of handwritten numbers.

math.ST

Stein's method of moments for truncated multivariate distributions

We use Stein characterisations to derive new moment-type estimators for the parameters of several truncated multivariate distributions in the i.i.d. case; we also derive the asymptotic properties of these estimators. Our examples include the truncated multivariate normal distribution and truncated products of independent univariate distributions. The estimators are explicit and therefore provide an interesting alternative to the maximum-likelihood estimator (MLE). The quality of these estimators is assessed through competitive simulation studies, in which we compare their behaviour to the performance of the MLE and the score matching approach.

math.ST

Derivatives of entropy and the MMSE conjecture

We investigate the entropy $H(\mu,t)$ of a probability measure $\mu$ along the heat flow and more precisely we seek for closed algebraic representations of its derivatives. Provided that $\mu$ admits moments of any order, it is indeed proved in [Guo et al., 2010] that $t\mapsto H(\mu,t)$ is smooth, and in [Ledoux, 2016] that its derivatives at zero can be expressed into multivariate polynomials evaluated in the moments (or cumulants) of $\mu$. In the seminal contribution \cite{Led}, these algebraic expressions are derived through $\Gamma$-calculus techniques which provide implicit recursive formulas for these polynomials. Our main contribution consists in a fine combinatorial analysis of these inductive relations and for the first time to derive closed formulas for the leading coefficients of these polynomials expressions. Building upon these explicit formulas we revisit the so-called "MMSE conjecture" from [Guo et al., 2010] which asserts that two distributions on the real line with the same entropy along the heat flow must coincide up to translation and symmetry. Our approach enables us to provide new conditions on the source distributions ensuring that the MMSE conjecture holds and to refine several criteria proved in [Ledoux, 2016]. As illustrating examples, our findings cover the cases of uniform and Rademacher distributions, for which previous results in the literature were inapplicable.

cs.IT

One-dimensional Stein's method with bespoke derivatives

We introduce a version of Stein's method of comparison of operators specifically tailored to the problem of bounding the Wasserstein-1 distance between continuous and discrete distributions on the real line. Our approach rests on a new family of weighted discrete derivative operators, which we call bespoke derivatives. We also propose new bounds on the derivatives of the solutions of Stein equations for Integrated Pearson random variables; this is a crucial step in Stein's method. We apply our result to several examples, including the Central Limit Theorem, Polya-Eggenberger urn models, the empirical distribution of the ground state of a many-interacting-worlds harmonic oscillator, the stationary distribution for the number of genes in the Moran model, and the stationary distribution of the Erlang-C system. Whenever our bounds can be compared with bounds from the literature, our constants are sharper.

math.PR

Stein's Method of Moments

Stein operators allow to characterise probability distributions via differential operators. Based on these characterisations, we develop a new method of point estimation for marginal parameters of strictly stationary and ergodic processes, which we call \emph{Stein's Method of Moments} (SMOM). These SMOM estimators satisfy the desirable classical properties such as consistency and asymptotic normality. As a consequence of the usually simple form of the operator, we obtain explicit estimators in cases where standard methods such as (pseudo-) maximum likelihood estimation require a numerical procedure to calculate the estimate. In addition, with our approach, one can choose from a large class of test functions which allows to improve significantly on the moment estimator. Moreover, for i.i.d.\ observations, we retrieve data-dependent functions that result in asymptotically efficient estimators and give a sequence of explicit SMOM estimators that converge to the maximum likelihood estimator. Our simulation study demonstrates that for a number of important univariate continuous probability distributions our SMOM estimators possess excellent small sample behaviour, often outperforming the maximum likelihood estimator and other widely-used methods in terms of lower bias and mean squared error. We also illustrate the pertinence of our approach on a real data set related to rainfall modelisation.

math.ST

Independent additive weighted bias distributions and associated goodness-of-fit tests

We use a Stein identity to define a new class of parametric distributions which we call ``independent additive weighted bias distributions.'' We investigate related $L^2$-type discrepancy measures, empirical versions of which not only encompass traditional ODE-based procedures but also offer novel methods for conducting goodness-of-fit tests in composite hypothesis testing problems. We determine critical values for these new procedures using a parametric bootstrap approach and evaluate their power through Monte Carlo simulations. As an illustration, we apply these procedures to examine the compatibility of two real data sets with a compound Poisson Gamma distribution.

stat.ME

Normal approximation for the posterior in exponential families

In this paper, we obtain quantitative, non-asymptotic, and data-dependent \textit{Bernstein-von Mises type} bounds on the normal approximation of the posterior distribution in exponential family models with arbitrary centring and scaling. Our bounds, stated in the total variation and Wasserstein distances, are valid for univariate and multivariate posteriors alike, and do not require a conjugate prior setting. They are obtained through a refined version of Stein's method of comparison of operators that allows for improved dimensional dependence in high-dimensional settings and may also be of interest in other problems. Our approach is rather flexible and, in certain settings, allows for the derivation of bounds with rates of convergence faster than the usual \( O(n^{-1/2}) \) rate (when \( n \) is the sample size). We illustrate our findings on a variety of exponential family distributions, including the Weibull, multinomial, and linear regression with unknown variance. The resulting bounds have an explicit dependence on the prior distribution and on sufficient statistics of the data from the sample, and thus provide insight into how these factors affect the quality of the normal approximation. Insights from our examples include identification of conditions under which faster \( O(n^{-1}) \) convergence rates occur for Bernoulli data, illustrations of how the quality of the normal approximation is influenced by the choice of standardisation, and dimensional dependence in high-dimensional settings.

math.ST

A note on one-dimensional Poincar\'e inequalities by Stein-type integration

We study the weighted Poincar\'e constant $C(p,w)$ of a probability density $p$ with weight function $w$ using integration methods inspired by Stein's method. We obtain a new version of the Chen-Wang variational formula which, as a byproduct, yields simple upper and lower bounds on $C(p,w)$ in terms of the so-called Stein kernel of $p$. We also iterate these variational formulas so as to build sequences of nested intervals containing the Poincar\'e constant, sequences of functions converging to said constant, as well as sequences of functions converging to the solutions of the corresponding spectral problem. Our results rely on the properties of a pseudo inverse operator of the classical Sturm-Liouville operator. We illustrate our methods on a variety of examples: Gaussian functionals, weighted Gaussian, beta, gamma, Subbotin, and Weibull distributions.

math.PR

Stein's method and approximating the multidimensional quantum harmonic oscillator

Stein's method is used to study discrete representations of multidimensional distributions that arise as approximations of states of quantum harmonic oscillators. These representations model how quantum effects result from the interaction of finitely many classical "worlds," with the role of sample size played by the number of worlds. Each approximation arises as the ground state of a Hamiltonian involving a particular interworld potential function. Such approximations have previously been studied for one-dimensional quantum harmonic oscillators, but the multidimensional case has remained unresolved. Our approach, framed in terms of spherical coordinates, provides the rate of convergence of the discrete approximation to the ground state in terms of Wasserstein distance. The fastest rate of convergence to the ground state is found to occur in three dimensions. This result is obtained using a discrete density approach to Stein's method applied to the radial component of the ground state solution.

math.PR

Stein's Method Meets Computational Statistics: A Review of Some Recent Developments

Stein's method compares probability distributions through the study of a class of linear operators called Stein operators. While mainly studied in probability and used to underpin theoretical statistics, Stein's method has led to significant advances in computational statistics in recent years. The goal of this survey is to bring together some of these recent developments and, in doing so, to stimulate further research into the successful field of Stein's method and statistics. The topics we discuss include tools to benchmark and compare sampling methods such as approximate Markov chain Monte Carlo, deterministic alternatives to sampling methods, control variate techniques, parameter estimation and goodness-of-fit testing.

stat.ME

Stein's density method for multivariate continuous distributions

This paper provides a general framework for Stein's density method for multivariate continuous distributions. The approach associates to any probability density function a canonical operator and Stein class, as well as an infinite collection of operators and classes which we call standardizations. These in turn spawn an entire family of Stein identities and characterizations for any continuous distribution on $\mathbb{R}^d$, among which we highlight those based on the score function and the Stein kernel. A feature of these operators is that they do not depend on normalizing constants. A new definition of Stein kernel is introduced and examined; integral formulas are obtained through a connection with mass transport, as well as ready-to-use explicit formulas for elliptical distributions. The flexibility of the kernels is used to compare in Stein discrepancy (and therefore 2-Wasserstein distance) between two normal distributions, Student and normal distributions, as well as two normal-gamma distributions. Upper and lower bounds on the 1-Wasserstein distance between continuous distributions are provided, and computed for a variety of examples: comparison between different normal distributions (improving on existing bounds in some regimes), posterior distributions with different priors in a Bayesian setting (including logistic regression), centred Azzalini--Dalla Valle distributions. Finally the notion of weak Stein equation and weak Stein factors is introduced. Bounds for solutions of the weak Stein equation are obtained for Lipschitz test functions if the distribution admits a Poincar\'e constant. We use these bounds to compare different copulas on the unit square in 1-Wasserstein distance.

math.PR

Stein operators, kernels and discrepancies for multivariate continuous distributions

In this paper we present a general framework for Stein's method for multivariate continuous distributions. The approach gives a collection of Stein characterisations, among which we highlight score-Stein operators and kernel Stein operators. Applications include copulas and distance between posterior distributions. We give a general construction for Stein kernels for elliptical distributions and discuss Stein kernels in generality, highlighting connections with Fisher information and mass transport. Finally, a goodness-of-fit test based on Stein discrepancies is given.

math.PR

Distances between distributions via Stein's method

We build on the formalism developed in [arXiv:1906.08372v1] to propose new representations of solutions to Stein equations. We provide new uniform and non uniform bounds on these solutions (a.k.a.\ Stein factors). We use these representations to obtain representations for differences between expectations in terms of solutions to the Stein equations. We apply these to compute abstract Stein-type bounds on Kolmogorov, Total Variation and Wasserstein distances between arbitrary distributions. We apply our results to several illustrative examples, and compare our results with current literature on the same topic, whenever possible. In all occurrences our results are competitive.

math.PR

Simple variance bounds with applications to Bayesian posteriors and intractable distributions

Using coupling techniques based on Stein's method for probability approximation, we revisit classical variance bounding inequalities of Chernoff, Cacoullos, Chen and Klaassen. Taking advantage of modern coupling techniques allows us to establish novel variance bounds in settings where the underlying density function is unknown or intractable. Applications include bounds for asymptotically Gaussian random variables using zero-biased couplings, bounds for random variables which are New Better (Worse) than Used in Expectation, and analysis of the posterior in Bayesian statistics.

math.PR

Some new Stein operators for product distributions

We provide a general result for finding Stein operators for the product of two independent random variables whose Stein operators satisfy a certain assumption, extending a recent result of Gaunt, Mijoule and Swan \cite{gms18}. This framework applies to non-centered normal and non-centered gamma random variables, as well as a general sub-family of the variance-gamma distributions. Curiously, there is an increase in complexity in the Stein operators for products of independent normals as one moves, for example, from centered to non-centered normals. As applications, we give a simple derivation of the characteristic function of the product of independent normals, and provide insight into why the probability density function of this distribution is much more complicated in the non-centered case than the centered case.

math.PR