arXiv subjects

Sonia Petrone

Publications and source records attributed to Sonia Petrone.

13 recordsLinked to original sources

arXiv2026

Nonparametric quantile inference using Dirichlet processes

This chapter deals with nonparametric inference for quantiles from a Bayesian perspective, using the Dirichlet process. The posterior distribution for quantiles is characterised, enabling also explicit formulae for posterior mean and variance. Unlike the Bayes estimator for the distribution function, our Bayes estimator for the quantile function is a smooth curve. A Bernstein--von Mises type theorem is given, exhibiting the limiting posterior distribution of the quantile process. Links to kernel-smoothed quantile estimators are provided. As a side product we develop an automatic nonparametric density estimator, free of smoothing parameters, with support exactly matching that of the data range. Nonparametric Bayes estimators are also provided for other quantile-related quantities, including the Lorenz curve and the Gini index, for Doksum's shift curve and for Parzen's comparison distribution in two-sample situations, and finally for the quantile regression function in situations with covariates.

math.ST↗

arXiv2026

Uncertainty Decomposition for Bayes-Filtered Transformers via Bayesian Predictive Inference

Bayes-filtered transformers are transformers meta-learned on sequences from a prior predictive distribution to approximate the corresponding posterior predictive distribution. They output total predictive uncertainty in a single forward pass but never explicitly represent a posterior distribution, making the standard route to separating aleatoric from epistemic uncertainty unavailable. We address this challenge through the lens of Bayesian predictive inference (BPI). Our main result is a predictive Central Limit Theorem (CLT) for supervised settings under conditions that are among the weakest known in the BPI literature. The CLT characterises the posterior of the limiting predictive distribution given an observed context as asymptotically Gaussian; the variance of this Gaussian quantifies epistemic uncertainty. We apply the framework to TabPFN, a Bayes-filtered transformer that is a state-of-the-art foundation model for tabular prediction. The resulting credible bands achieve near-nominal frequentist coverage as context length grows, and the decomposition largely matches standard desiderata: epistemic uncertainty shrinks with context length and is highest in sparsely observed regions within the span of the context data, while aleatoric uncertainty dominates near decision boundaries where classes overlap.

stat.ML↗

arXiv2024

Characterizing the Effects of Environmental Exposures on Social Mobility: Bayesian Semi-parametrics for Principal Stratification

Understanding the causal effects of air pollution exposures on social mobility is attracting increasing attention. At the same time, education is widely recognized as a key driver of social mobility. However, the causal pathways linking fine particulate matter (PM2.5) exposure, educational attainment, and social mobility remain largely unexplored. To address this, we adopt the principal stratification approach, which rigorously defines causal effects when a post-treatment variable--educational attainment--is affected by exposure--PM2.5--and may, in turn, affect the primary outcome--social mobility. To estimate the causal effects, we propose a Bayesian semi-parametric method leveraging infinite mixtures for modeling the primary outcome. The proposed method (i) allows flexible modeling of the distribution of the primary potential outcomes, (ii) improves the accuracy of counterfactual imputation--a fundamental problem in causal inference framework--, and (iii) enables the characterization of treatment effects across different values of the post-treatment variable. We evaluate the performance of the proposed methodology through a Monte Carlo simulation study, demonstrating its advantages over existing approaches. Finally, we apply our method to a national dataset of 3,009 counties in the United States to estimate the causal effect of PM2.5 on social mobility, taking into account educational attainment as a post-treatment variable. Our findings indicate that in counties where higher PM2.5 exposure significantly reduces educational attainment social mobility decreases by approximately 5% compared to counties with lower PM2.5 exposure. We also find that in counties where exposure to PM2.5 does not affect educational attainment, social mobility is reduced by approximately 2% hinting at the possibility of further, yet unexplored, pathways connecting air pollution and social mobility.

stat.ME↗

arXiv2024

Empirical Bayes in Bayesian learning: understanding a common practice

In applications of Bayesian procedures, once a class of priors has been chosen, it may be tempting to fix the prior's hyperparameters from the data, in an empirical Bayes (EB) fashion, usually by their maximum marginal likelihood estimates (MMLE). This is a quite common but questionable practice, lacking a rigorous theoretical basis. We provide a theoretical framework where this form of EB is regarded as a computational strategy for approximating a genuine Bayesian posterior distribution and prove its general properties for parametric models. While computing the MMLE may still be demanding, we prove novel results that allow us to provide a simple proxy. These results establish the limit behavior of the MMLE in quite general settings, including both identifiable and non-identifiable models - specifically, overfitted mixture models - significantly filling a gap in the literature. Moreover, we study higher order merging, showing that, when not degenerate, the EB posterior approximates at a faster rate an oracle-Bayes posterior distribution based on the prior law that, within the given class of priors, expresses the most information on the true model's parameters. This is a faster approximation than classic Bernstein-von Mises results. Our work provides formal content to common beliefs on this popular practice.

math.ST↗

arXiv2024

Exchangeability, prediction and predictive modeling in Bayesian statistics

There is currently a renewed interest in the Bayesian predictive approach to statistics. This paper offers a review on foundational concepts and focuses on predictive modeling, which by directly reasoning on prediction, bypasses inferential models or may characterize them. We detail predictive characterizations in exchangeable and partially exchangeable settings, for a large variety of data structures, and hint at new directions. The underlying concept is that Bayesian predictive rules are probabilistic learning rules, formalizing through conditional probability how we learn on future events given the available information. This concept has implications in any statistical problem and in inference, from classic contexts to less explored challenges, such as providing Bayesian uncertainty quantification to predictive algorithms in data science, as we show in the last part of the paper. The paper gives a historical overview, but also includes a few new results, presents some recent developments and poses some open questions.

math.ST↗

arXiv2023

Bayesian dependent mixture models: A predictive comparison and survey

For exchangeable data, mixture models are an extremely useful tool for density estimation due to their attractive balance between smoothness and flexibility. When additional covariate information is present, mixture models can be extended for flexible regression by modeling the mixture parameters, namely the weights and atoms, as functions of the covariates. These types of models are interpretable and highly flexible, allowing non only the mean but the whole density of the response to change with the covariates, which is also known as density regression. This article reviews Bayesian covariate-dependent mixture models and highlights which data types can be accommodated by the different models along with the methodological and applied areas where they have been used. In addition to being highly flexible, these models are also numerous; we focus on nonparametric constructions and broadly organize them into three categories: 1) joint models of the responses and covariates, 2) conditional models with single-weights and covariate-dependent atoms, and 3) conditional models with covariate-dependent weights. The diversity and variety of the available models in the literature raises the question of how to choose among them for the application at hand. We attempt to shed light on this question through a careful analysis of the predictive equations for the conditional mean and density function as well as predictive comparisons in three simulated data examples.

stat.ME↗

arXiv2022

Enriched Pitman-Yor processes

In Bayesian nonparametrics there exists a rich variety of discrete priors, including the Dirichlet process and its generalizations, which are nowadays well-established tools. Despite the remarkable advances, few proposals are tailored for modeling observations lying on product spaces, such as $\mathbb{R}^p$. Indeed, for multivariate random measures, most available priors lack flexibility and do not allow for separate partition structures among the spaces. We introduce a discrete nonparametric prior, termed enriched Pitman-Yor process (EPY), aimed at addressing these issues. Theoretical properties of this novel prior are extensively investigated. We discuss its formal link with the enriched Dirichlet process and normalized random measures, we describe a square-breaking representation and we obtain closed-form expressions for the posterior law and the involved urn schemes. In second place, we show that several existing approaches, including Dirichlet processes with a spike and slab base measure and mixture of mixtures models, implicitly rely on special cases of the EPY, which therefore constitutes a unified probabilistic framework for many Bayesian nonparametric priors. Interestingly, our unifying formulation will allow us to naturally extend these models while preserving their analytical tractability. As an illustration, we employ the EPY for a species sampling problem in ecology and for functional clustering in an e-commerce application.

stat.ME↗

arXiv2021

Infinite-color randomly reinforced urns with dominant colors

We define and prove limit results for a class of dominant P\'olya sequences, which are randomly reinforced urn processes with color-specific random weights and unbounded number of possible colors. Under fairly mild assumptions on the expected reinforcement, we show that the predictive and the empirical distributions converge almost surely (a.s.) in total variation to the same random probability measure $\tilde{P}$; moreover, $\tilde{P}(\mathcal{D})=1$ a.s., where $\mathcal{D}$ denotes the set of dominant colors for which the expected reinforcement is maximum. In the general case, the predictive probabilities and the empirical frequencies of any $\delta$-neighborhood of $\mathcal{D}$ converge a.s. to one. That is, although non-dominant colors continue to be regularly observed, their distance to $\mathcal{D}$ converges in probability to zero. We refine the above results with rates of convergence. We further hint potential applications of dominant P\'olya sequences in randomized clinical trials and species sampling, and use our central limit results for Bayesian inference.

math.PR↗

arXiv2021

A closed-form filter for binary time series

Non-Gaussian state-space models arise in several applications, and within this framework the binary time series setting provides a relevant example. However, unlike for Gaussian state-space models - where filtering, predictive and smoothing distributions are available in closed form - binary state-space models require approximations or sequential Monte Carlo strategies for inference and prediction. This is due to the apparent absence of conjugacy between the Gaussian states and the likelihood induced by the observation equation for the binary data. In this article we prove that the filtering, predictive and smoothing distributions in dynamic probit models with Gaussian state variables are, in fact, available and belong to a class of unified skew-normals (SUN) whose parameters can be updated recursively in time via analytical expressions. Also the key functionals of these distributions are, in principle, available, but their calculation requires the evaluation of multivariate Gaussian cumulative distribution functions. Leveraging SUN properties, we address this issue via novel Monte Carlo methods based on independent samples from the smoothing distribution, that can easily be adapted to the filtering and predictive case, thus improving state-of-the-art approximate and sequential Monte Carlo inference in small-to-moderate dimensional studies. Novel sequential Monte Carlo procedures that exploit the SUN properties are also developed to deal with online inference in high dimensions. Performance gains over competitors are outlined in a financial application.

stat.ME↗

arXiv2019

Quasi-Bayes properties of a recursive procedure for mixtures

Bayesian methods are often optimal, yet increasing pressure for fast computations, especially with streaming data, brings renewed interest in faster, possibly sub-optimal, solutions. The extent to which these algorithms approximate Bayesian solutions is a question of interest, but often unanswered. We propose a methodology to address this question in predictive settings, when the algorithm can be reinterpreted as a probabilistic predictive rule. We specifically develop the proposed methodology for a recursive procedure for online learning in nonparametric mixture models, often refereed to as Newton's algorithm. This algorithm is simple and fast; however, its approximation properties are unclear. By reinterpreting it as a predictive rule, we can show that it underlies a statistical model which is, asymptotically, a Bayesian, exchangeable mixture model. In this sense, the recursive rule provides a quasi-Bayes solution. While the algorithm only offers a point estimate, our clean statistical formulation allows us to provide the asymptotic posterior distribution and asymptotic credible intervals for the mixing distribution. Moreover, it gives insights for tuning the parameters, as we illustrate in simulation studies, and paves the way to extensions in various directions. Beyond mixture models, our approach can be applied to other predictive algorithms.

math.ST↗

arXiv2017

On a notion of partially conditionally identically distributed sequences

A notion of conditionally identically distributed (c.i.d.) sequences has been studied as a form of stochastic dependence that is weaker than exchangeability, but is equivalent to exchangeability for stationary sequences. In this article we extend this notion to families of sequences. Paralleling the extension from exchangeability to partial exchangeability in the sense of de Finetti, we propose a notion of partially c.i.d. dependence, that is equivalent to partial exchangeability for stationary processes. Partially c.i.d. families of sequences preserve attractive limit properties of partial exchangeability, and are asymptotically partially exchangeable. Moreover, we provide strong laws of large numbers and two central limit theorems. Our focus is on the asymptotic agreement of predictions and empirical means, which lies in the foundations of Bayesian statistics. Natural examples are interacting randomly reinforced processes satisfying certain conditions on the reinforcement.

math.PR↗

arXiv2015

Predictive Characterization of Mixtures of Markov Chains

Predictive constructions are a powerful way of characterizing the probability law of stochastic processes with certain forms of invariance, such as exchangeability or Markov exchangeability. When de Finetti-like representation theorems are available, the predictive characterization implicitly defines the prior distribution, starting from assumptions on the observables; moreover, it often helps designing efficient computational strategies. In this paper we give necessary and sufficient conditions on the sequence of predictive distributions such that they characterize a Markov exchangeable probability law for a discrete valued process X. Under recurrence, Markov exchangeable processes are mixtures of Markov chains. Thus, our results help checking when a predictive scheme characterizes a prior for Bayesian inference on the unknown transition matrix of a Markov chain. Our predictive conditions are in some sense minimal sufficient conditions for Markov exchangeability; we also provide predictive conditions for recurrence. We illustrate their application in relevant examples from the literature and in novel constructions.

stat.ME↗

arXiv2012

Bayes and empirical Bayes: do they merge?

Bayesian inference is attractive for its coherence and good frequentist properties. However, it is a common experience that eliciting a honest prior may be difficult and, in practice, people often take an {\em empirical Bayes} approach, plugging empirical estimates of the prior hyperparameters into the posterior distribution. Even if not rigorously justified, the underlying idea is that, when the sample size is large, empirical Bayes leads to "similar" inferential answers. Yet, precise mathematical results seem to be missing. In this work, we give a more rigorous justification in terms of merging of Bayes and empirical Bayes posterior distributions. We consider two notions of merging: Bayesian weak merging and frequentist merging in total variation. Since weak merging is related to consistency, we provide sufficient conditions for consistency of empirical Bayes posteriors. Also, we show that, under regularity conditions, the empirical Bayes procedure asymptotically selects the value of the hyperparameter for which the prior mostly favors the "truth". Examples include empirical Bayes density estimation with Dirichlet process mixtures.

math.ST↗