Searcharxiv⌕ Search

arXiv subjects

A. W. van der Vaart

Publications and source records attributed to A. W. van der Vaart.

13 recordsLinked to original sources
arXiv2022

Empirical and Full Bayes estimation of the type of a Pitman-Yor process

The Pitman-Yor process is a random discrete probability distribution of which the atoms can be used to model the relative abundance of species. The process is indexed by a type parameter $σ$, which controls the number of different species in a finite sample from a realization of the distribution. A random sample of size $n$ from the Pitman-Yor process of type $σ>0$ will contain of the order $n^σ$ distinct values (``species''). In this paper we consider the estimation of the type parameter by both empirical Bayes and full Bayes methods. We derive the asymptotic normality of the empirical Bayes estimator and a Bernstein-von Mises theorem for the full Bayes posterior, in the frequentist setup that the observations are a random sample from a given true distribution. We also consider the estimation of the second parameter of the Pitman-Yor process, the prior precision. We apply our results to derive the limit behaviour of the likelihood ratio in a setting of forensic statistics.

math.ST↗
arXiv2021

The Bernstein-von Mises theorem for the Pitman-Yor process of nonnegative type

The Pitman-Yor process is a random probability distribution, that can be used as a prior distribution in a nonparametric Bayesian analysis. The process is of species sampling type and generates discrete distributions, which yield of the order $n^σ$ different values ("species") in a random sample of size $n$, if the type $σ$ is positive. Thus this type parameter can be set to target true distributions of various levels of discreteness, making the Pitman-Yor process an interesting prior in this case. It was previously shown that the resulting posterior distribution is consistent if and only if the true distribution of the data is discrete. In this paper we derive the distributional limit of the posterior distribution, in the form of a (corrected) Bernstein-von Mises theorem, which previously was known only in the continuous, inconsistent case. It turns out that the Pitman-Yor posterior distribution has good behaviour if the true distribution of the data is discrete with atoms that decrease not too slowly. Credible sets derived from the posterior distribution provide valid frequentist confidence sets in this case. For a general discrete distribution, the posterior distribution, although consistent, may contain a bias which does not converge to zero at the $\sqrt{n}$ rate and invalidates posterior inference. We propose a bias correction that solves this problem. We also consider the effect of estimating the type parameter from the data, both by empirical Bayes and full Bayes methods. In a small simulation study we illustrate that without bias correction the coverage of credible sets can be arbitrarily low, also for some discrete distributions.

math.ST↗
arXiv2015

Frequentist coverage of adaptive nonparametric Bayesian credible sets

We investigate the frequentist coverage of Bayesian credible sets in a nonparametric setting. We consider a scale of priors of varying regularity and choose the regularity by an empirical Bayes method. Next we consider a central set of prescribed posterior probability in the posterior distribution of the chosen regularity. We show that such an adaptive Bayes credible set gives correct uncertainty quantification of "polished tail" parameters, in the sense of high probability of coverage of such parameters. On the negative side, we show by theory and example that adaptation of the prior necessarily leads to gross and haphazard uncertainty quantification for some true parameters that are still within the hyperrectangle regularity scale.

math.ST↗
arXiv2014

The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors

We consider the horseshoe estimator due to Carvalho, Polson and Scott (2010) for the multivariate normal mean model in the situation that the mean vector is sparse in the nearly black sense. We assume the frequentist framework where the data is generated according to a fixed mean vector. We show that if the number of nonzero parameters of the mean vector is known, the horseshoe estimator attains the minimax $\ell_2$ risk, possibly up to a multiplicative constant. We provide conditions under which the horseshoe estimator combined with an empirical Bayes estimate of the number of nonzero means still yields the minimax risk. We furthermore prove an upper bound on the rate of contraction of the posterior distribution around the horseshoe estimator, and a lower bound on the posterior variance. These bounds indicate that the posterior distribution of the horseshoe prior may be more informative than that of other one-component priors, including the Lasso.

math.ST↗
arXiv2013

Bayes procedures for adaptive inference in inverse problems for the white noise model

We study empirical and hierarchical Bayes approaches to the problem of estimating an infinite-dimensional parameter in mildly ill-posed inverse problems. We consider a class of prior distributions indexed by a hyperparameter that quantifies regularity. We prove that both methods we consider succeed in automatically selecting this parameter optimally, resulting in optimal convergence rates for truths with Sobolev or analytic "smoothness", without using knowledge about this regularity. Both methods are illustrated by simulation examples.

math.ST↗
arXiv2013

Bayesian recovery of the initial condition for the heat equation

We study a Bayesian approach to recovering the initial condition for the heat equation from noisy observations of the solution at a later time. We consider a class of prior distributions indexed by a parameter quantifying "smoothness" and show that the corresponding posterior distributions contract around the true parameter at a rate that depends on the smoothness of the true initial condition and the smoothness and scale of the prior. Correct combinations of these characteristics lead to the optimal minimax rate. One type of priors leads to a rate-adaptive Bayesian procedure. The frequentist coverage of credible sets is shown to depend on the combination of the prior and true parameter as well, with smoother priors leading to zero coverage and rougher priors to (extremely) conservative results. In the latter case credible sets are much larger than frequentist confidence sets, in that the ratio of diameters diverges to infinity. The results are numerically illustrated by a simulated data example.

math.ST↗
arXiv2012

Bayesian inverse problems with Gaussian priors

The posterior distribution in a nonparametric inverse problem is shown to contract to the true parameter at a rate that depends on the smoothness of the parameter, and the smoothness and scale of the prior. Correct combinations of these characteristics lead to the minimax rate. The frequentist coverage of credible sets is shown to depend on the combination of prior and true parameter, with smoother priors leading to zero coverage and rougher priors to conservative coverage. In the latter case credible sets are of the correct order of magnitude. The results are numerically illustrated by the problem of recovering a function from observation of a noisy version of its primitive.

math.ST↗
arXiv2009

Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth

We consider nonparametric Bayesian estimation inference using a rescaled smooth Gaussian field as a prior for a multidimensional function. The rescaling is achieved using a Gamma variable and the procedure can be viewed as choosing an inverse Gamma bandwidth. The procedure is studied from a frequentist perspective in three statistical settings involving replicated observations (density estimation, regression and classification). We prove that the resulting posterior distribution shrinks to the distribution that generates the data at a speed which is minimax-optimal up to a logarithmic factor, whatever the regularity level of the data-generating distribution. Thus the hierachical Bayesian procedure, with a fixed prior, is shown to be fully adaptive.

math.ST↗
arXiv2008

Rates of contraction of posterior distributions based on Gaussian process priors

We derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on Gaussian processes. The rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel Hilbert space of the Gaussian process and the small ball probabilities of the Gaussian process. We determine these quantities for a range of examples of Gaussian priors and in several statistical settings. For instance, we consider the rate of contraction of the posterior distribution based on sampling from a smooth density model when the prior models the log density as a (fractionally integrated) Brownian motion. We also consider regression with Gaussian errors and smooth classification under a logistic or probit link function combined with various priors.

math.ST↗
arXiv2008

Reproducing kernel Hilbert spaces of Gaussian priors

We review definitions and properties of reproducing kernel Hilbert spaces attached to Gaussian variables and processes, with a view to applications in nonparametric Bayesian statistics using Gaussian priors. The rate of contraction of posterior distributions based on Gaussian priors can be described through a concentration function that is expressed in the reproducing Hilbert space. Absolute continuity of Gaussian measures and concentration inequalities play an important role in understanding and deriving this result. Series expansions of Gaussian variables and transformations of their reproducing kernel Hilbert spaces under linear maps are useful tools to compute the concentration function.

math.FA↗
arXiv2006

Misspecification in infinite-dimensional Bayesian statistics

We consider the asymptotic behavior of posterior distributions if the model is misspecified. Given a prior distribution and a random sample from a distribution $P_0$, which may not be in the support of the prior, we show that the posterior concentrates its mass near the points in the support of the prior that minimize the Kullback--Leibler divergence with respect to $P_0$. An entropy condition and a prior-mass condition determine the rate of convergence. The method is applied to several examples, with special interest for infinite-dimensional models. These include Gaussian mixtures, nonparametric regression and parametric models.

math.ST↗
arXiv2004

Estimating the causal effect of a time-varying treatment on time-to-event using structural nested failure time models

In this paper we review an approach to estimating the causal effect of a time-varying treatment on time to some event of interest. This approach is designed for the situation where the treatment may have been repeatedly adapted to patient characteristics, which themselves may also be time-dependent. In this situation the effect of the treatment cannot simply be estimated by conditioning on the patient characteristics, as these may themselves be indicators of the treatment effect. This so-called time-dependent confounding is typical in observational studies. We discuss a new class of failure time models, structural nested failure time models, which can be used to estimate the causal effect of a time-varying treatment, and present methods for estimating and testing the parameters of these models.

math.ST↗