arXiv subjects

Lucien Birgé

Publications and source records attributed to Lucien Birgé.

12 recordsLinked to original sources

arXiv2020

Robust Bayes-Like Estimation: Rho-Bayes estimation

We consider the problem of estimating the joint distribution $P$ of $n$ independent random variables within the Bayes paradigm from a non-asymptotic point of view. Assuming that $P$ admits some density $s$ with respect to a given reference measure, we consider a density model $\overline S$ for $s$ that we endow with a prior distribution $π$ (with support $\overline S$) and we build a robust alternative to the classical Bayes posterior distribution which possesses similar concentration properties around $s$ whenever it belongs to the model $\overline S$. Furthermore, in density estimation, the Hellinger distance between the classical and the robust posterior distributions tends to 0, as the number of observations tends to infinity, under suitable assumptions on the model and the prior, provided that the model $\overline S$ contains the true density $s$. However, unlike what happens with the classical Bayes posterior distribution, we show that the concentration properties of this new posterior distribution are still preserved in the case of a misspecification of the model, that is when $s$ does not belong to $\overline S$ but is close enough to it with respect to the Hellinger distance.

math.ST↗

arXiv2017

Rho-estimators revisited: General theory and applications

Following Baraud, Birgé and Sart (2017), we pursue our attempt to design a robust universal estimator of the joint ditribution of $n$ independent (but not necessarily i.i.d.) observations for an Hellinger-type loss. Given such observations with an unknown joint distribution $\mathbf{P}$ and a dominated model $\mathscr{Q}$ for $\mathbf{P}$, we build an estimator $\widehat{\mathbf{P}}$ based on $\mathscr{Q}$ and measure its risk by an Hellinger-type distance. When $\mathbf{P}$ does belong to the model, this risk is bounded by some quantity which relies on the local complexity of the model in a vicinity of $\mathbf{P}$. In most situations this bound corresponds to the minimax risk over the model (up to a possible logarithmic factor). When $\mathbf{P}$ does not belong to the model, its risk involves an additional bias term proportional to the distance between $\mathbf{P}$ and $\mathscr{Q}$, whatever the true distribution $\mathbf{P}$. From this point of view, this new version of $ρ$-estimators improves upon the previous one described in Baraud, Birgé and Sart (2017) which required that $\mathbf{P}$ be absolutely continuous with respect to some known reference measure. Further additional improvements have been brought as compared to the former construction. In particular, it provides a very general treatment of the regression framework with random design as well as a computationally tractable procedure for aggregating estimators. We also give some conditions for the Maximum Likelihood Estimator to be a $ρ$-estimator. Finally, we consider the situation where the Statistician has at disposal many different models and we build a penalized version of the $ρ$-estimator for model selection and adaptation purposes. In the regression setting, this penalized estimator not only allows to estimate the regression function but also the distribution of the errors.

math.ST↗

arXiv2017

Une alternative robuste au maximum de vraisemblance: la $ρ$-estimation

This paper is based on our personal notes for the short course we gave on January 5, 2017 at Institut Henri Poincaré, after an invitation of the SFdS. Our purpose is to give an overview of the method of $ρ$-estimation and of the optimality and robustness properties of the estimators built according to this procedure. This method can be viewed as the sequel of a long series of researches which were devoted to the construction of estimators with good properties in various statistical frameworks. We shall emphasize the connection between the $ρ$-estimators and the previous ones, in particular the maximum likelihood estimator, and we shall show, via some typical examples, that the $ρ$-estimators perform better from various points of view. ------ Cet article est fondé sur les notes du mini-cours que nous avons donné le 5 janvier 2017 à l'Institut Henri Poincaré à l'occasion d'une journée organisée par la SFdS et consacrée à la Statistique Mathématique. Il vise à donner un aperçu de la méthode de $ρ$-estimation ainsi que des propriétés d'optimalité et de robustesse des estimateurs construits selon cette procédure. Cette méthode s'inscrit dans une longue lignée de recherches dont l'objectif a été de produire des estimateurs possédant de bonnes propriétés pour un ensemble de cadres statistiques aussi vaste que possible. Nous mettrons en lumière les liens forts qui existent entre les $ρ$-estimateurs et ces prédécesseurs, notamment les estimateurs du maximum de vraisemblance, mais montrerons également, au travers d'exemples choisis, que les $ρ$-estimateurs les surpassent sur bien des aspects.

math.ST↗

arXiv2016

A new method for estimation and model selection: $ρ$-estimation

The aim of this paper is to present a new estimation procedure that can be applied in many statistical frameworks including density and regression and which leads to both robust and optimal (or nearly optimal) estimators. In density estimation, they asymptotically coincide with the celebrated maximum likelihood estimators at least when the statistical model is regular enough and contains the true density to estimate. For very general models of densities, including non-compact ones, these estimators are robust with respect to the Hellinger distance and converge at optimal rate (up to a possible logarithmic factor) in all cases we know. In the regression setting, our approach improves upon the classical least squares from many aspects. In simple linear regression for example, it provides an estimation of the coefficients that are both robust to outliers and simultaneously rate-optimal (or nearly rate-optimal) for large class of error distributions including Gaussian, Laplace, Cauchy and uniform among others.

math.ST↗

arXiv2016

Rates of convergence of rho-estimators for sets of densities satisfying shape constraints

The purpose of this paper is to pursue our study of rho-estimators built from i.i.d. observations that we defined in Baraud et al. (2014). For a ρ-estimator based on some model S (which means that the estimator belongs to S) and a true distribution of the observations that also belongs to S, the risk (with squared Hellinger loss) is bounded by a quantity which can be viewed as a dimension function of the model and is often related to the "metric dimension" of this model, as defined in Birgé (2006). This is a minimax point of view and it is well-known that it is pessimistic. Typically, the bound is accurate for most points in the model but may be very pessimistic when the true distribution belongs to some specific part of it. This is the situation that we want to investigate here. For some models, like the set of decreasing densities on [0,1], there exist specific points in the model that we shall call "extremal" and for which the risk is substantially smaller than the typical risk. Moreover, the risk at a non-extremal point of the model can be bounded by the sum of the risk bound at a well-chosen extremal point plus the square of its distance to this point. This implies that if the true density is close enough to an extremal point, the risk at this point may be smaller than the minimax risk on the model and this actually remains true even if the true density does not belong to the model. The result is based on some refined bounds on the suprema of empirical processes that are established in Baraud (2016).

math.ST↗

arXiv2015

A new V-fold type procedure based on robust tests

We define a general V-fold cross-validation type method based on robust tests, which is an extension of the hold-out defined by Birg{é} [7, Section 9]. We give some theoretical results showing that, under some weak assumptions on the considered statistical procedures, our selected estimator satisfies an oracle type inequality. We also introduce a fast algorithm that implements our method. Moreover we show in our simulations that this V-fold performs generally well for estimating a density for different sample sizes, and can handle well-known problems, such as binwidth selection for histograms or bandwidth selection for kernels. We finally provide a comparison with other classical V-fold methods and study empirically the influence of the value of V on the risk.

math.ST↗

arXiv2014

About the non-asymptotic behaviour of Bayes estimators

This paper investigates the {\em nonasymptotic} properties of Bayes procedures for estimating an unknown distribution from $n$ i.i.d.\ observations. We assume that the prior is supported by a model $(\scr{S},h)$ (where $h$ denotes the Hellinger distance) with suitable metric properties involving the number of small balls that are needed to cover larger ones. We also require that the prior put enough probability on small balls. We consider two different situations. The simplest case is the one of a parametric model containing the target density for which we show that the posterior concentrates around the true distribution at rate $1/\sqrt{n}$. In the general situation, we relax the parametric assumption and take into account a possible mispecification of the model. Provided that the Kullback-Leibler Information between the true distribution and $\scr{S}$ is finite, we establish risk bounds for the Bayes estimators.

math.ST↗

arXiv2013

Estimating composite functions by model selection

We consider the problem of estimating a function $s$ on $[-1,1]^{k}$ for large values of $k$ by looking for some best approximation by composite functions of the form $g\circ u$. Our solution is based on model selection and leads to a very general approach to solve this problem with respect to many different types of functions $g,u$ and statistical frameworks. In particular, we handle the problems of approximating $s$ by additive functions, single and multiple index models, neural networks, mixtures of Gaussian densities (when $s$ is a density) among other examples. We also investigate the situation where $s=g\circ u$ for functions $g$ and $u$ belonging to possibly anisotropic smoothness classes. In this case, our approach leads to a completely adaptive estimator with respect to the regularity of $s$.

math.ST↗

arXiv2013

Model selection for density estimation with L2-loss

We consider here estimation of an unknown probability density s belonging to L2(mu) where mu is a probability measure. We have at hand n i.i.d. observations with density s and use the squared L2-norm as our loss function. The purpose of this paper is to provide an abstract but completely general method for estimating s by model selection, allowing to handle arbitrary families of finite-dimensional (possibly non-linear) models and any density s belonging to L2(mu). We shall, in particular, consider the cases of unbounded densities and bounded densities with unknown bound and investigate how the L-infinity-norm of s may influence the risk. We shall also provide applications to adaptive estimation and aggregation of preliminary estimators. Although of a purely theoretical nature, our method leads to results that cannot presently be reached by more concrete methods.

math.ST↗

arXiv2007

Model selection for Poisson processes

Our purpose in this paper is to apply the general methodology for model selection based on T-estimators developed in Birgé [Ann. Inst. H. Poincaré Probab. Statist. 42 (2006) 273--325] to the particular situation of the estimation of the unknown mean measure of a Poisson process. We introduce a Hellinger type distance between finite positive measures to serve as our loss function and we build suitable tests between balls (with respect to this distance) in the set of mean measures. As a consequence of the existence of such tests, given a suitable family of approximating models, we can build T-estimators for the mean measure based on this family of models and analyze their performances. We provide a number of applications to adaptive intensity estimation when the square root of the intensity belongs to various smoothness classes. We also give a method for aggregation of preliminary estimators.

math.ST↗

arXiv2006

Estimating the intensity of a random measure by histogram type estimators

The purpose of this paper is to estimate the intensity of some random measure by a piecewise constant function on a finite partition of the underlying measurable space. Given a (possibly large) family of candidate partitions, we build a piecewise constant estimator (histogram) on each of them and then use the data to select one estimator in the family. Choosing the square of a Hellinger-type distance as our loss function, we show that each estimator built on a given partition satisfies an analogue of the classical squared bias plus variance risk bound. Moreover, the selection procedure leads to a final estimator satisfying some oracle-type inequality, with, as usual, a possible loss corresponding to the complexity of the family of partitions we consider. When this complexity is not too high, the selected estimator has a risk bounded, up to a universal constant, by the smallest risk bound obtained for the estimators in the family. For suitable choices of the family of partitions, we deduce uniform risk bounds over various classes of intensities. Our approach applies to the estimation of the intensity of an inhomogenous Poisson process, among other counting processes, or the estimation of the mean of a random vector with nonnegative components.

math.ST↗

arXiv2006

The Brouwer Lecture 2005: Statistical estimation with model selection

The purpose of this paper is to explain the interest and importance of (approximate) models and model selection in Statistics. Starting from the very elementary example of histograms we present a general notion of finite dimensional model for statistical estimation and we explain what type of risk bounds can be expected from the use of one such model. We then give the performance of suitable model selection procedures from a family of such models. We illustrate our point of view by two main examples: the choice of a partition for designing a histogram from an n-sample and the problem of variable selection in the context of Gaussian regression.

math.ST↗