arXiv subjects

B. J. K. Kleijn

Publications and source records attributed to B. J. K. Kleijn.

13 recordsLinked to original sources

arXiv2025

Existence and phase structure of random inverse limit measures

Analogous to Kolmogorov's theorem for the existence of stochastic processes describing random functions, we consider theorems for the existence of stochastic processes describing random measures, as limits of inverse measure systems. Specifically, given a coherent inverse system of random (bounded/signed/positive/probability) histograms on refining partitions, we study conditions for the existence and uniqueness of a corresponding random inverse limit, a Radon probability measure on the space of (bounded/signed/positive/probability) measures. Depending on the topology (vague/tight/weak/total-variational) and Kingman's notion of complete randomness, the limiting random measure is in one of four phases, distinguished by their degrees of concentration (support/domination/discreteness). Results are applied in the well-known Dirichlet and Polya tree families of random probability measures and in a new Gaussian family of signed inverse limit measures. In these three families, examples of all four phases occur and we describe the corresponding conditions on defining parameters.

math.PR↗

arXiv2024

Contiguity and remote contiguity of some random graphs

Asymptotic properties of random graph sequences, like occurrence of a giant component or full connectivity in Erdős-Rényi graphs, are usually derived with very specific choices for defining parameters. The question arises to which extent those parameters choices may be perturbed, without losing the asymptotic property. Writing $(P_n)$ and $(Q_n)$ for two sequences of graph distributions, asymptotic equivalence (convergence in total-variation) and contiguity ($P_n(A_n)=o(1) \implies Q_n(A_n)=o(1)$) have been considered by (Janson, 2010) and others; here we use so-called remote contiguity (for some fixed $a_n\downarrow 0$, $P_n(A_n)=o(a_n) \implies Q_n(A_n)=o(1)$) to show that connectivity properties are preserved in more heavily perturbed Erdős-Rényi graphs. The techniques we demonstrate with random graphs here, extend to general asymptotic properties, e.g. in more complex large-graph limits, scaling limits, large-sample limits, etc.

math.PR↗

arXiv2023

Asymptotic uncertainty quantification for communities in sparse planted bi-section models

Posterior distributions for community structure in sparse planted bi-section models are shown to achieve exact (resp. almost-exact) recovery, with sharp bounds for the sparsity regimes where edge probabilities decrease as $O(\log(n)/n)$ (resp. $O(1/n)$). Assuming posterior recovery, one may interpret credible sets (resp. enlarged credible sets) as asymptotically consistent confidence sets; the diameters of those credible sets are controlled by the rate of posterior concentration. If credible levels are chosen to grow to one quickly enough, corresponding credible sets can be interpreted as frequentist confidence sets without conditions on posterior concentration. In the regimes with $O(1/n)$ edge sparsity, or when within-community and between-community edge probabilities are very close, credible sets may be enlarged to achieve frequentist asymptotic coverage, also without conditions on posterior concentration.

math.ST↗

arXiv2021

Confidence sets in a sparse stochastic block model with two communities of unknown sizes

In a sparse stochastic block model with two communities of unequal sizes we derive two posterior concentration inequalities, that imply (1) posterior (almost-)exact recovery of the community structure under sparsity bounds comparable to well-known sharp bounds in the planted bi-section model; (2) a construction of confidence sets for the community assignment from credible sets, with finite graph sizes. The latter enables exact frequentist uncertain quantification with Bayesian credible sets at non-asymptotic graph sizes, where posteriors can be simulated well. There turns out to be no proportionality between credible and confidence levels: for given edge probabilities and a desired confidence level, there exists a critical graph size where the required credible level drops sharply from close to one to close to zero. At such graph sizes the frequentist decides to include not most of the posterior support for the construction of his confidence set, but only a small subset of community assignments containing the highest amounts of posterior probability (like the maximum-a-posteriori estimator). It is argued that for the proposed construction of confidence sets, a form of early stopping applies to MCMC sampling of the posterior, which would enable the computation of confidence sets at larger graph sizes.

math.ST↗

arXiv2021

Uncertainty quantification and testing in a stochastic block model with two unequal communities

We show posterior convergence for the community structure in the planted bi-section model, for several interesting priors. Examples include where the label on each vertex is iid Bernoulli distributed, with some parameter $r\in(0,1)$. The parameter $r$ may be fixed, or equipped with a beta distribution. We do not have constraints on the class sizes, which might be as small as zero, or include all vertices, and everything in between. This enables us to test between a uniform (Erdös-Rényi) random graph with no distinguishable community or the planted bi-section model. The exact bounds for posterior convergence enable us to convert credible sets into confidence sets. Symmetric testing with posterior odds is shown to be consistent.

math.ST↗

arXiv2020

Uncertainty quantification in the stochastic block model with an unknown number of classes

We study the frequentist properties of Bayesian statistical inference for the stochastic block model, with an unknown number of classes of varying sizes. We equip the space of vertex labellings with a prior on the number of classes and, conditionally, a prior on the labels. The number of classes may grow to infinity as a function of the number of vertices, depending on the sparsity of the graph. We derive non-asymptotic posterior contraction rates of the form $P_{θ_{0,n}}Π_n(B_n\mid X^n)\le ε_n$, where $X^n$ is the observed graph, generated according to $P_{θ_{0,n}}$, $B_n$ is either $\{θ_{0, n}\}$ or, in the very sparse case, a ball around $θ_{0,n}$ of known extent, and $ε_n$ is an explicit rate of convergence. These results enable conversion of credible sets to confidence sets. In the sparse case, credible tests are shown to be confidence sets. In the very sparse case, credible sets are enlarged to form confidence sets. Confidence levels are explicit, for each $n$, as a function of the credible level and the rate of convergence. Hypothesis testing between the number of classes is considered with the help of posterior odds, and is shown to be consistent. Explicit upper bounds on errors of the first and second type and an explicit lower bound on the power of the tests are given.

math.ST↗

arXiv2018

Criteria for posterior consistency

Frequentist conditions for asymptotic suitability of Bayesian procedures focus on lower bounds for prior mass in Kullback-Leibler neighbourhoods of the data distribution. The goal of this paper is to investigate the flexibility in criteria for posterior consistency with i.i.d. data. We formulate a versatile posterior consistency theorem that applies both to well- and mis-specified models and which we use to re-derive Schwartz's theorem, consider Kullback-Leibler consistency and formulate consistency theorems in which priors charge metric balls. It is generalized to sieved models with Barron's negligible prior mass condition and to separable models with variations on Walker's consistency theorem. Results also apply to marginal semi-parametric consistency: support boundary estimation is considered explicitly and consistency is proved in a model for which Kullback-Leibler priors do not exist. Other examples include consistent density estimation in mixture models with Dirichlet or Gibbs-type priors of full weak support. Regarding posterior convergence at a rate, it is shown that under a mild integrability condition, the second-order Ghosal-Ghosh-van der Vaart prior mass condition can be relaxed to a lower bound to the prior mass in Schwartz's Kullback-Leibler neighbourhoods. The posterior rate of convergence is derived in a simple, parametric model for heavy-tailed distributions in which the Ghosal-Ghosh-van der Vaart condition cannot be satisfied by any prior.

math.ST↗

arXiv2017

On the frequentist validity of Bayesian limits

To the frequentist who computes posteriors, not all priors are useful asymptotically: in this paper Schwartz's 1965 Kullback-Leibler condition is generalised to enable frequentist interpretation of convergence of posterior distributions with the complex models and often dependent datasets in present-day statistical applications. We prove four simple and fully general frequentist theorems, for posterior consistency; for posterior rates of convergence; for consistency of the Bayes factor in hypothesis testing or model selection; and a theorem to obtain confidence sets from credible sets. The latter has a significant methodological consequence in frequentist uncertainty quantification: use of a suitable prior allows one to convert credible sets of a calculated, simulated or approximated posterior into asymptotically consistent confidence sets, in full generality. This extends the main inferential implication of the Bernstein-von Mises theorem to non-parametric models without smoothness conditions. Proofs require the existence of a Bayesian type of test sequence and priors giving rise to local prior predictive distributions that satisfy a weakened form of Le~Cam's contiguity with respect to the data distribution. Results are applied in a wide range of examples and counterexamples.

math.ST↗

arXiv2015

The Bayesian Analysis of Complex, High-Dimensional Models: Can It Be CODA?

We consider the Bayesian analysis of a few complex, high-dimensional models and show that intuitive priors, which are not tailored to the fine details of the model and the estimated parameters, produce estimators which perform poorly in situations in which good, simple frequentist estimators exist. The models we consider are: stratified sampling, the partial linear model, linear and quadratic functionals of white noise and estimation with stopping times. We present a strong version of Doob's consistency theorem which demonstrates that the existence of a uniformly $\sqrt{n}$-consistent estimator ensures that the Bayes posterior is $\sqrt{n}$-consistent for values of the parameter in subsets of prior probability 1. We also demonstrate that it is, at least, in principle, possible to construct Bayes priors giving both global and local minimax rates, using a suitable combination of loss functions. We argue that there is no contradiction in these apparently conflicting findings.

math.ST↗

arXiv2014

The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors

We consider the horseshoe estimator due to Carvalho, Polson and Scott (2010) for the multivariate normal mean model in the situation that the mean vector is sparse in the nearly black sense. We assume the frequentist framework where the data is generated according to a fixed mean vector. We show that if the number of nonzero parameters of the mean vector is known, the horseshoe estimator attains the minimax $\ell_2$ risk, possibly up to a multiplicative constant. We provide conditions under which the horseshoe estimator combined with an empirical Bayes estimate of the number of nonzero means still yields the minimax risk. We furthermore prove an upper bound on the rate of contraction of the posterior distribution around the horseshoe estimator, and a lower bound on the posterior variance. These bounds indicate that the posterior distribution of the horseshoe prior may be more informative than that of other one-component priors, including the Lasso.

math.ST↗

arXiv2013

Semiparametric posterior limits

We review the Bayesian theory of semiparametric inference following Bickel and Kleijn (2012) and Kleijn and Knapik (2013). After an overview of efficiency in parametric and semiparametric estimation problems, we consider the Bernstein-von Mises theorem (see, e.g., Le Cam and Yang (1990)) and generalize it to (LAN) regular and (LAE) irregular semiparametric estimation problems. We formulate a version of the semiparametric Bernstein-von Mises theorem that does not depend on least-favourable submodels, thus bypassing the most restrictive condition in the presentation of Bickel and Kleijn (2012). The results are applied to the (regular) estimation of the linear coefficient in partial linear regression (with a Gaussian nuisance prior) and of the kernel bandwidth in a model of normal location mixtures (with a Dirichlet nuisance prior), as well as the (irregular) estimation of the boundary of the support of a monotone family of densities (with a Gaussian nuisance prior).

math.ST↗

arXiv2012

The semiparametric Bernstein-von Mises theorem

In a smooth semiparametric estimation problem, the marginal posterior for the parameter of interest is expected to be asymptotically normal and satisfy frequentist criteria of optimality if the model is endowed with a suitable prior. It is shown that, under certain straightforward and interpretable conditions, the assertion of Le Cam's acclaimed, but strictly parametric, Bernstein-von Mises theorem [Univ. California Publ. Statist. 1 (1953) 277-329] holds in the semiparametric situation as well. As a consequence, Bayesian point-estimators achieve efficiency, for example, in the sense of Hájek's convolution theorem [Z. Wahrsch. Verw. Gebiete 14 (1970) 323-330]. The model is required to satisfy differentiability and metric entropy conditions, while the nuisance prior must assign nonzero mass to certain Kullback-Leibler neighborhoods [Ghosal, Ghosh and van der Vaart Ann. Statist. 28 (2000) 500-531]. In addition, the marginal posterior is required to converge at parametric rate, which appears to be the most stringent condition in examples. The results are applied to estimation of the linear coefficient in partial linear regression, with a Gaussian prior on a smoothness class for the nuisance.

math.ST↗

arXiv2006

Misspecification in infinite-dimensional Bayesian statistics

We consider the asymptotic behavior of posterior distributions if the model is misspecified. Given a prior distribution and a random sample from a distribution $P_0$, which may not be in the support of the prior, we show that the posterior concentrates its mass near the points in the support of the prior that minimize the Kullback--Leibler divergence with respect to $P_0$. An entropy condition and a prior-mass condition determine the rate of convergence. The method is applied to several examples, with special interest for infinite-dimensional models. These include Gaussian mixtures, nonparametric regression and parametric models.

math.ST↗