arXiv subjects

Stefano Favaro

Publications and source records attributed to Stefano Favaro.

At least 37 records · Page 2Linked to original sources

arXiv2025

Non-asymptotic approximations of Gaussian neural networks via second-order Poincaré inequalities

There is a recent and growing literature on large-width asymptotic and non-asymptotic properties of deep Gaussian neural networks (NNs), namely NNs with weights initialized as Gaussian distributions. For a Gaussian NN of depth $L\geq1$ and width $n\geq1$, it is well-known that, as $n\rightarrow+\infty$, the NN's output converges (in distribution) to a Gaussian process. Recently, some quantitative versions of this result, also known as quantitative central limit theorems (QCLTs), have been obtained, showing that the rate of convergence is $n^{-1}$, in the $2$-Wasserstein distance, and that such a rate is optimal. In this paper, we investigate the use of second-order Poincaré inequalities as an alternative approach to establish QCLTs for the NN's output. Previous approaches consist of a careful analysis of the NN, by combining non-trivial probabilistic tools with ad-hoc techniques that rely on the recursive definition of the network, typically by means of an induction argument over the layers, and it is unclear if and how they still apply to other NN's architectures. Instead, the use of second-order Poincaré inequalities rely only on the fact that the NN is a functional of a Gaussian process, reducing the problem of establishing QCLTs to the algebraic problem of computing the gradient and Hessian of the NN's output, which still applies to other NN's architectures. We show how our approach is effective in establishing QCLTs for the NN's output, though it leads to suboptimal rates of convergence. We argue that such a worsening in the rates is peculiar to second-order Poincaré inequalities, and it should be interpreted as the "cost" for having a straightforward, and general, procedure for obtaining QCLTs.

cs.LG↗

arXiv2025

A smoothed-Bayesian approach to frequency recovery from sketched data

We provide a novel statistical perspective on a classical problem at the intersection of computer science and information theory: recovering the empirical frequency of a symbol in a large discrete dataset using only a compressed representation, or sketch, obtained via random hashing. Departing from traditional algorithmic approaches, recent works have proposed Bayesian nonparametric (BNP) methods that can provide more informative frequency estimates by leveraging modeling assumptions about the distribution of the sketched data. In this paper, we propose a smoothed-Bayesian method, inspired by existing BNP approaches but designed in a frequentist framework to overcome the computational limitations of the BNP approaches when dealing with large-scale data from realistic distributions, including those with power-law tail behaviors. For sketches obtained with a single hash function, our approach is supported by rigorous frequentist properties, including unbiasedness and optimality under a squared error loss function within an intuitive class of linear estimators. For sketches with multiple hash functions, we introduce an approach based on multi-view learning to construct computationally efficient frequency estimators. We validate our method on synthetic and real data, comparing its performance to that of existing alternatives.

stat.ME↗

arXiv2025

Function-Space MCMC for Bayesian Wide Neural Networks

Bayesian Neural Networks represent a fascinating confluence of deep learning and probabilistic reasoning, offering a compelling framework for understanding uncertainty in complex predictive models. In this paper, we investigate the use of the preconditioned Crank-Nicolson algorithm and its Langevin version to sample from a reparametrised posterior distribution of the neural network's weights, as the widths grow larger. In addition to being robust in the infinite-dimensional setting, we prove that the acceptance probabilities of the proposed algorithms approach 1 as the width of the network increases, independently of any stepsize tuning. Moreover, we examine and compare how the mixing speeds of the underdamped Langevin Monte Carlo, the preconditioned Crank-Nicolson and the preconditioned Crank-Nicolson Langevin samplers are influenced by changes in the network width in some real-world cases. Our findings suggest that, in wide Bayesian Neural Networks configurations, the preconditioned Crank-Nicolson algorithm allows for a scalable and more efficient sampling of the reparametrised posterior distribution, as also evidenced by a higher effective sample size and improved diagnostic results compared with the other analysed algorithms.

cs.LG↗

arXiv2025

A new look on large deviations and concentration inequalities for the Ewens-Pitman model

The Ewens-Pitman model is a probability distribution for random partitions of the set $[n]=\{1,\ldots,n\}$, parameterized by $α\in[0,1)$ and $θ>-α$, with $α=0$ corresponding to the Ewens model in population genetics. The goal of this paper is to provide an alternative and concise proof of the Feng-Hoppe large deviation principle for the number $K_{n}$ of partition sets in the Ewens-Pitman model with $α\in(0,1)$ and $θ>-α$. Our approach leverages an integral representation of the moment-generating function of $K_{n}$ in terms of the (one-parameter) Mittag-Leffler function, along with a sharp asymptotic expansion of it. This approach significantly simplifies the original proof of Feng-Hoppe large deviation principle, as it avoids all the technical difficulties arising from a continuity argument with respect to rational and non-rational values of $α$. Beyond large deviations for $K_{n}$, our approach allows to establish a sharp concentration inequality for $K_n$ involving the rate function of the large deviation principle.

math.PR↗

arXiv2025

Student-t processes as infinite-width limits of posterior Bayesian neural networks

The asymptotic properties of Bayesian Neural Networks (BNNs) have been extensively studied, particularly regarding their approximations by Gaussian processes in the infinite-width limit. We extend these results by showing that posterior BNNs can be approximated by Student-t processes, which offer greater flexibility in modeling uncertainty. Specifically, we show that, if the parameters of a BNN follow a Gaussian prior distribution, and the variance of both the last hidden layer and the Gaussian likelihood function follows an Inverse-Gamma prior distribution, then the resulting posterior BNN converges to a Student-t process in the infinite-width limit. Our proof leverages the Wasserstein metric to establish control over the convergence rate of the Student-t process approximation.

stat.ML↗

arXiv2025

Gaussian credible intervals in Bayesian nonparametric estimation of the unseen

The unseen-species problem assumes $n\geq1$ samples from a population of individuals belonging to different species, possibly infinite, and calls for estimating the number $K_{n,m}$ of hitherto unseen species that would be observed if $m\geq1$ new samples were collected from the same population. This is a long-standing problem in statistics, which has gained renewed relevance in biological and physical sciences, particularly in settings with large values of $n$ and $m$. In this paper, we adopt a Bayesian nonparametric approach to the unseen-species problem under the Pitman-Yor prior, and propose a novel methodology to derive large $m$ asymptotic credible intervals for $K_{n,m}$, for any $n\geq1$. By leveraging a Gaussian central limit theorem for the posterior distribution of $K_{n,m}$, our method improves upon competitors in two key aspects: firstly, it enables the full parameterization of the Pitman-Yor prior, including the Dirichlet prior; secondly, it avoids the need of Monte Carlo sampling, enhancing computational efficiency. We validate the proposed method on synthetic and real data, demonstrating that it improves the empirical performance of competitors by significantly narrowing the gap between asymptotic and exact credible intervals for any $m\geq1$.

stat.ME↗

arXiv2024

Laws of large numbers and central limit theorem for Ewens-Pitman model

The Ewens-Pitman model is a distribution for random partitions of the set $\{1,\ldots,n\}$, with $n\in\mathbb{N}$, indexed by parameters $α\in [0,1)$ and $θ>-α$, such that $α=0$ is the Ewens model in population genetics. The large $n$ asymptotic behaviour of the number $K_{n}$ of blocks in the Ewens-Pitman random partition has been extensively investigated in terms of almost-sure and Gaussian fluctuations, which show that $K_{n}$ scales as $\log n$ and $n^α$ depending on whether $α=0$ or $α\in(0,1)$, providing non-random and random limiting behaviours, respectively. In this paper, we study the large $n$ asymptotic behaviour of $K_{n}$ when the parameter $θ$ is allowed to depend linearly on $n\in\mathbb{N}$, a non-standard asymptotic regime first considered for $α=0$ in Feng (\textit{The Annals of Applied Probability}, \textbf{17}, 2007). In particular, for $α\in[0,1)$ and $θ=λn$, with $λ>0$, we establish a law of large numbers (LLN) and a central limit theorem (CLT) for $K_{n}$, which show that $K_{n}$ scales as $n$, providing non-random limiting behaviours. Depending on whether $α=0$ or $α\in(0,1)$, our results rely on different arguments. For $α=0$ we rely on the representation of $K_{n}$ as a sum of independent, but not identically distributed, Bernoulli random variables, which leads to a refinement of the CLT in terms of a Berry-Esseen theorem. Instead, for $α\in(0,1)$, we rely on a compound Poisson construction of $K_{n}$, leading to prove LLNs, CLTs and Berry-Esseen theorems for the number of blocks of the negative-Binomial compound Poisson random partition, which are of independent interest.

math.PR↗

arXiv2024

On strong posterior contraction rates for Besov-Laplace priors in the white noise model

In this article, we investigate the problem of estimating a spatially inhomogeneous function and its derivatives in the white noise model using Besov-Laplace priors. We show that smoothness-matching priors attains minimax optimal posterior contraction rates, in strong Sobolev metrics, over the Besov spaces $B^β_{11}$, $β> d/2$, closing a gap in the existing literature. Our strong posterior contraction rates also imply that the posterior distributions arising from Besov-Laplace priors with matching regularity enjoy a desirable plug-in property for derivative estimation, entailing that the push-forward measures under differential operators optimally recover the derivatives of the unknown regression function. The proof of our results relies on the novel approach to posterior contraction rates, based on Wasserstein distance, recently developed by Dolera, Favaro and Mainini (Probability Theory and Related Fields, 2024). We show how this approach allows to overcome some technical challenges that emerge in the frequentist analysis of smoothness-matching Besov-Laplace priors.

math.ST↗

arXiv2024

A martingale approach to Gaussian fluctuations and laws of iterated logarithm for Ewens-Pitman model

The Ewens-Pitman model refers to a distribution for random partitions of $[n]=\{1,\ldots,n\}$, which is indexed by a pair of parameters $α\in [0,1)$ and $θ>-α$, with $α=0$ corresponding to the Ewens model in population genetics. The large $n$ asymptotic properties of the Ewens-Pitman model have been the subject of numerous studies, with the focus being on the number $K_{n}$ of partition sets and the number $K_{r,n}$ of partition subsets of size $r$, for $r=1,\ldots,n$. While for $α=0$ asymptotic results have been obtained in terms of almost-sure convergence and Gaussian fluctuations, for $α\in(0,1)$ only almost-sure convergences are available, with the proof for $K_{r,n}$ being given only as a sketch. In this paper, we make use of martingales to develop a unified and comprehensive treatment of the large $n$ asymptotic behaviours of $K_{n}$ and $K_{r,n}$ for $α\in(0,1)$, providing alternative, and rigorous, proofs of the almost-sure convergences of $K_{n}$ and $K_{r,n}$, and covering the gap of Gaussian fluctuations. We also obtain new laws of the iterated logarithm for $K_{n}$ and $K_{r,n}$.

math.PR↗

arXiv2024

Optimal estimation of high-order missing masses, and the rare-type match problem

Consider a random sample $(X_{1},\ldots,X_{n})$ from an unknown discrete distribution $P=\sum_{j\geq1}p_{j}δ_{s_{j}}$ on a countable alphabet $\mathbb{S}$, and let $(Y_{n,j})_{j\geq1}$ be the empirical frequencies of distinct symbols $s_{j}$'s in the sample. We consider the problem of estimating the $r$-order missing mass, which is a discrete functional of $P$ defined as $$θ_{r}(P;\mathbf{X}_{n})=\sum_{j\geq1}p^{r}_{j}I(Y_{n,j}=0).$$ This is generalization of the missing mass whose estimation is a classical problem in statistics, being the subject of numerous studies both in theory and methods. First, we introduce a nonparametric estimator of $θ_{r}(P;\mathbf{X}_{n})$ and a corresponding non-asymptotic confidence interval through concentration properties of $θ_{r}(P;\mathbf{X}_{n})$. Then, we investigate minimax estimation of $θ_{r}(P;\mathbf{X}_{n})$, which is the main contribution of our work. We show that minimax estimation is not feasible over the class of all discrete distributions on $\mathbb{S}$, and not even for distributions with regularly varying tails, which only guarantee that our estimator is consistent for $θ_{r}(P;\mathbf{X}_{n})$. This leads to introduce a stronger assumption for the tail behaviour of $P$, which is proved to be sufficient for minimax estimation of $θ_r(P;\mathbf{X}_{n})$, making the proposed estimator an optimal minimax estimator of $θ_{r}(P;\mathbf{X}_{n})$. Our interest in the $r$-order missing mass arises from forensic statistics, where the estimation of the $2$-order missing mass appears in connection to the estimation of the likelihood ratio $T(P,\mathbf{X}_{n})=θ_{1}(P;\mathbf{X}_{n})/θ_{2}(P;\mathbf{X}_{n})$, known as the "fundamental problem of forensic mathematics". We present theoretical guarantees to nonparametric estimation of $T(P,\mathbf{X}_{n})$.

math.ST↗

arXiv2024

Quantitative CLTs in Deep Neural Networks

We study the distribution of a fully connected neural network with random Gaussian weights and biases in which the hidden layer widths are proportional to a large constant $n$. Under mild assumptions on the non-linearity, we obtain quantitative bounds on normal approximations valid at large but finite $n$ and any fixed network depth. Our theorems show both for the finite-dimensional distributions and the entire process, that the distance between a random fully connected network (and its derivatives) to the corresponding infinite width Gaussian process scales like $n^{-γ}$ for $γ>0$, with the exponent depending on the metric used to measure discrepancy. Our bounds are strictly stronger in terms of their dependence on network width than any previously available in the literature; in the one-dimensional case, we also prove that they are optimal, i.e., we establish matching lower bounds.

cs.LG↗

arXiv2024

Random measure priors in Bayesian recovery from sketches

This paper introduces a Bayesian nonparametric approach to frequency recovery from lossy-compressed discrete data, leveraging all information contained in a sketch obtained through random hashing. By modeling the data points as random samples from an unknown discrete distribution endowed with a Poisson-Kingman prior, we derive the posterior distribution of a symbol's empirical frequency given the sketch. This leads to principled frequency estimates through mean functionals, e.g., the posterior mean, median and mode. We highlight applications of this general result to Dirichlet process and Pitman-Yor process priors. Notably, we prove that the former prior uniquely satisfies a sufficiency property that simplifies the posterior distribution, while the latter enables a convenient large-sample asymptotic approximation. Additionally, we extend our approach to the problem of cardinality recovery, estimating the number of distinct symbols in the sketched dataset. Our approach to frequency recovery also adapts to a more general ``traits'' setting, where each data point has integer levels of association with multiple symbols, typically referred to as ``traits''. By employing a generalized Indian buffet process, we compute the posterior distribution of a trait's frequency using both the Poisson and Bernoulli distributions for the trait association levels, respectively yielding exact and approximate posterior frequency distributions.

math.ST↗

arXiv2024

MCMC for Bayesian nonparametric mixture modeling under differential privacy

Estimating the probability density of a population while preserving the privacy of individuals in that population is an important and challenging problem that has received considerable attention in recent years. While the previous literature focused on frequentist approaches, in this paper, we propose a Bayesian nonparametric mixture model under differential privacy (DP) and present two Markov chain Monte Carlo (MCMC) algorithms for posterior inference. One is a marginal approach, resembling Neal's algorithm 5 with a pseudo-marginal Metropolis-Hastings move, and the other is a conditional approach. Although our focus is primarily on local DP, we show that our MCMC algorithms can be easily extended to deal with global differential privacy mechanisms. Moreover, for some carefully chosen mechanisms and mixture kernels, we show how auxiliary parameters can be analytically marginalized, allowing standard MCMC algorithms (i.e., non-privatized, such as Neal's Algorithm 2) to be efficiently employed. Our approach is general and applicable to any mixture model and privacy mechanism. In several simulations and a real case study, we discuss the performance of our algorithms and evaluate different privacy mechanisms proposed in the frequentist literature.

stat.CO↗

arXiv2024

Bayesian Nonparametric Inference for "Species-sampling" Problems

Given an observed sample from a population of individuals belonging to species, "species-sampling" problems (SSPs) call for estimating some features of the unknown species composition of additional unobservable samples from the same population. Within SSPs, the problems of estimating coverage probabilities, the number of unseen species and coverages of prevalences have emerged in the past three decades for being the subject of numerous methodological and applied works, mostly in biological sciences but also in statistical machine learning, electrical engineering, theoretical computer science, information theory and forensic statistics. In this paper, we focus on these popular SSPs, and present an overview of their Bayesian nonparametric (BNP) analysis under the Pitman--Yor process (PYP) prior. While reviewing the literature, we improve on computation and interpretability of existing posterior inferences, typically expressed through complicated combinatorial numbers, by establishing novel posterior representations in terms of simple compound Binomial and Hypergeometric distributions. We also consider the problem of estimating the discount and scale parameters of the PYP prior, showing a property of Bayesian consistency with respect to estimation through the hierarchical Bayes and empirical Bayes approaches, that is: the discount parameter can be estimated consistently, whereas the scale parameter cannot be estimated consistently, thus advising caution in posterior inference. We conclude our work by discussing some generalizations of SSPs, mostly in the field of biological sciences, which deal with "feature-sampling", multiple populations of individuals sharing species and classes of Markov chains.

math.ST↗

arXiv2024

A Bayesian Nonparametric Approach to Species Sampling Problems with Ordering

Species-sampling problems (SSPs) refer to a vast class of statistical problems calling for the estimation of (discrete) functionals of the unknown species composition of an unobservable population. A common feature of SSPs is their invariance with respect to species labeling, which is at the core of the Bayesian nonparametric (BNP) approach to SSPs under the popular Pitman-Yor process (PYP) prior. In this paper, we develop a BNP approach to SSPs that are not "invariant" to species labeling, in the sense that an ordering or ranking is assigned to species' labels. Inspired by the population genetics literature on age-ordered alleles' compositions, we study the following SSP with ordering: given an observable sample from an unknown population of individuals belonging to species (alleles), with species' labels being ordered according to weights (ages), estimate the frequencies of the first $r$ order species' labels in an enlarged sample obtained by including additional unobservable samples. By relying on an ordered PYP prior, we obtain an explicit posterior distribution of the first $r$ order frequencies, with estimates being of easy implementation and computationally efficient. We apply our approach to the analysis of genetic variation, showing its effectiveness in estimating the frequency of the oldest allele, and then we discuss other potential applications.

stat.ME↗

arXiv2024

Improved prediction of future user activity in online A/B testing

In online randomized experiments or A/B tests, accurate predictions of participant inclusion rates are of paramount importance. These predictions not only guide experimenters in optimizing the experiment's duration but also enhance the precision of treatment effect estimates. In this paper we present a novel, straightforward, and scalable Bayesian nonparametric approach for predicting the rate at which individuals will be exposed to interventions within the realm of online A/B testing. Our approach stands out by offering dual prediction capabilities: it forecasts both the quantity of new customers expected in future time windows and, unlike available alternative methods, the number of times they will be observed. We derive closed-form expressions for the posterior distributions of the quantities needed to form predictions about future user activity, thereby bypassing the need for numerical algorithms such as Markov chain Monte Carlo. After a comprehensive exposition of our model, we test its performance on experiments on real and simulated data, where we show its superior performance with respect to existing alternatives in the literature.

stat.ME↗

arXiv2024

A Nonparametric Bayes Approach to Online Activity Prediction

Accurately predicting the onset of specific activities within defined timeframes holds significant importance in several applied contexts. In particular, accurate prediction of the number of future users that will be exposed to an intervention is an important piece of information for experimenters running online experiments (A/B tests). In this work, we propose a novel approach to predict the number of users that will be active in a given time period, as well as the temporal trajectory needed to attain a desired user participation threshold. We model user activity using a Bayesian nonparametric approach which allows us to capture the underlying heterogeneity in user engagement. We derive closed-form expressions for the number of new users expected in a given period, and a simple Monte Carlo algorithm targeting the posterior distribution of the number of days needed to attain a desired number of users; the latter is important for experimental planning. We illustrate the performance of our approach via several experiments on synthetic and real world data, in which we show that our novel method outperforms existing competitors.

stat.ME↗

arXiv2023

Strong posterior contraction rates via Wasserstein dynamics

In Bayesian statistics, posterior contraction rates (PCRs) quantify the speed at which the posterior distribution concentrates on arbitrarily small neighborhoods of a true model, in a suitable way, as the sample size goes to infinity. In this paper, we develop a new approach to PCRs, with respect to strong norm distances on parameter spaces of functions. Critical to our approach is the combination of a local Lipschitz-continuity for the posterior distribution with a dynamic formulation of the Wasserstein distance, which allows to set forth an interesting connection between PCRs and some classical problems arising in mathematical analysis, probability and statistics, e.g., Laplace methods for approximating integrals, Sanov's large deviation principles in the Wasserstein distance, rates of convergence of mean Glivenko-Cantelli theorems, and estimates of weighted Poincaré-Wirtinger constants. We first present a theorem on PCRs for a model in the regular infinite-dimensional exponential family, which exploits sufficient statistics of the model, and then extend such a theorem to a general dominated model. These results rely on the development of novel techniques to evaluate Laplace integrals and weighted Poincaré-Wirtinger constants in infinite-dimension, which are of independent interest. The proposed approach is applied to the regular parametric model, the multinomial model, the finite-dimensional and the infinite-dimensional logistic-Gaussian model and the infinite-dimensional linear regression. In general, our approach leads to optimal PCRs in finite-dimensional models, whereas for infinite-dimensional models it is shown explicitly how the prior distribution affect PCRs.

math.ST↗