arXiv subjects

Vladimir Spokoiny

Publications and source records attributed to Vladimir Spokoiny.

At least 55 records · Page 3Linked to original sources

arXiv2018

Large ball probability, Gaussian comparison and anti-concentration

We derive tight non-asymptotic bounds for the Kolmogorov distance between the probabilities of two Gaussian elements to hit a ball in a Hilbert space. The key property of these bounds is that they are dimension-free and depend on the nuclear (Schatten-one) norm of the difference between the covariance operators of the elements and on the norm of the mean shift. The obtained bounds significantly improve the bound based on Pinsker's inequality via the Kullback-Leibler divergence. We also establish an anti-concentration bound for a squared norm of a non-centered Gaussian element in Hilbert space. The paper presents a number of examples motivating our results and applications of the obtained bounds to statistical inference and to high-dimensional CLT.

math.PR↗

arXiv2017

Adaptive Nonparametric Clustering

This paper presents a new approach to non-parametric cluster analysis called Adaptive Weights Clustering (AWC). The idea is to identify the clustering structure by checking at different points and for different scales on departure from local homogeneity. The proposed procedure describes the clustering structure in terms of weights $ w_{ij} $ each of them measures the degree of local inhomogeneity for two neighbor local clusters using statistical tests of "no gap" between them. % The procedure starts from very local scale, then the parameter of locality grows by some factor at each step. The method is fully adaptive and does not require to specify the number of clusters or their structure. The clustering results are not sensitive to noise and outliers, the procedure is able to recover different clusters with sharp edges or manifold structure. The method is scalable and computationally feasible. An intensive numerical study shows a state-of-the-art performance of the method in various artificial examples and applications to text data. Our theoretical study states optimal sensitivity of AWC to local inhomogeneity.

stat.ML↗

arXiv2017

Construction of Non-asymptotic Confidence Sets in 2-Wasserstein Space

In this paper, we consider a probabilistic setting where the probability measures are considered to be random objects. We propose a procedure of construction non-asymptotic confidence sets for empirical barycenters in 2-Wasserstein space and develop the idea further to construction of a non-parametric two-sample test that is then applied to the detection of structural breaks in data with complex geometry. Both procedures mainly rely on the idea of multiplier bootstrap (Spokoiny and Zhilova (2015), Chernozhukov et al. (2014)). The main focus lies on probability measures that have commuting covariance matrices and belong to the same scatter-location family: we proof the validity of a bootstrap procedure that allows to compute confidence sets and critical values for a Wasserstein-based two-sample test.

math.ST↗

arXiv2017

Bootstrap confidence sets for spectral projectors of sample covariance

Let $X_{1},\ldots,X_{n}$ be i.i.d. sample in $\mathbb{R}^{p}$ with zero mean and the covariance matrix $\mathbfΣ$. The problem of recovering the projector onto an eigenspace of $\mathbfΣ$ from these observations naturally arises in many applications. Recent technique from [Koltchinskii, Lounici, 2015] helps to study the asymptotic distribution of the distance in the Frobenius norm $\| \mathbf{P}_r - \widehat{\mathbf{P}}_r \|_{2}$ between the true projector $\mathbf{P}_r$ on the subspace of the $r$-th eigenvalue and its empirical counterpart $\widehat{\mathbf{P}}_r$ in terms of the effective rank of $\mathbfΣ$. This paper offers a bootstrap procedure for building sharp confidence sets for the true projector $\mathbf{P}_r$ from the given data. This procedure does not rely on the asymptotic distribution of $\| \mathbf{P}_r - \widehat{\mathbf{P}}_r \|_{2}$ and its moments. It could be applied for small or moderate sample size $n$ and large dimension $p$. The main result states the validity of the proposed procedure for finite samples with an explicit error bound for the error of bootstrap approximation. This bound involves some new sharp results on Gaussian comparison and Gaussian anti-concentration in high-dimensional spaces. Numeric results confirm a good performance of the method in realistic examples.

math.ST↗

arXiv2016

Efficient randomized mirror descents in stochastic online convex optimization

In the paper we consider an application of mirror descent (dual averaging) to the stochastic online convex optimization problems. We compare classical mirror descent (Nemirovski-Yudin, 1979) with dual averaging (Nesterov, 2005) and Grigoriadis-Khachiyan algorithm (1995). Grigoriadis-Khachiyan algorithm has just proved to be a randomized mirror descent (dual averaging) with randomization in KL-projection of (sub)gradient to a unit simplex. We found out that this randomization is an optimal way of solving sparse matrix games and some other problems arising in convex optimization and experts weighting.

math.OC↗

arXiv2016

Universal method with inexact oracle and its applications for searching equillibriums in multistage transport problems

In this paper we propose a new efficient approach for numerical calculation of equillibriums in multistage transport problems. In the very core of our approach lies the proper combination of Universal Gradient Method proposed by Yu. Nesterov (2013) and conception of inexact oracle (Devolder--Glineur--Nesterov, 2011). In particular our technique allows us to calculate Wasserstein's Barycenter in a fast manner (this results generalized M. Cuturi et al. (2014)).

math.OC↗

arXiv2016

Efficient numerical algorithms for regularized regression problem with applications to traffic matrix estimations

In this work we collect and compare to each other many different numerical methods for regularized regression problem and for the problem of projection on a hyperplane. Such problems arise, for example, as a subproblem of demand matrix estimation in IP- networks. In this special case matrix of affine constraints has special structure: all elements are 0 or 1 and this matrix is sparse enough. We have to deal with huge-scale convex optimization problem of special type. Using the properties of the problem we try "to look inside the black-box" and to see how the best modern methods work being applied to this problem.

math.OC↗

arXiv2015

Bootstrap confidence sets under model misspecification

A multiplier bootstrap procedure for construction of likelihood-based confidence sets is considered for finite samples and a possible model misspecification. Theoretical results justify the bootstrap validity for a small or moderate sample size and allow to control the impact of the parameter dimension $p$: the bootstrap approximation works if $p^3/n$ is small. The main result about bootstrap validity continues to apply even if the underlying parametric model is misspecified under the so-called small modelling bias condition. In the case when the true model deviates significantly from the considered parametric family, the bootstrap procedure is still applicable but it becomes a bit conservative: the size of the constructed confidence sets is increased by the modelling bias. We illustrate the results with numerical examples for misspecified linear and logistic regressions.

math.ST↗

arXiv2015

Penalized maximum likelihood estimation and effective dimension

This paper extends some prominent statistical results including \emph{Fisher Theorem and Wilks phenomenon} to the penalized maximum likelihood estimation with a quadratic penalization. It appears that sharp expansions for the penalized MLE $\tilde{\thetav}_{G} $ and for the penalized maximum likelihood can be obtained without involving any asymptotic arguments, the results only rely on smoothness and regularity properties of the of the considered log-likelihood function. The error of estimation is specified in terms of the effective dimension $p_G $ of the parameter set which can be much smaller than the true parameter dimension and even allows an infinite dimensional functional parameter. In the i.i.d. case, the Fisher expansion for the penalized MLE can be established under the constraint "$p_G^{2}/n$ is small" while the remainder in the Wilks result is of order $p_G^{3}/n $.

stat.ME↗

arXiv2015

Bootstrap tuning in ordered model selection

In the problem of model selection for a given family of linear estimators, ordered by their variance, we offer a new "smallest accepted" approach motivated by Lepski's method and multiple testing theory. The procedure selects the smallest model which satisfies an acceptance rule based on comparison with all larger models. The method is completely data-driven and does not use any prior information about the variance structure of the noise: its parameters are adjusted to the underlying possibly heterogeneous noise by the so-called "propagation condition" using a wild bootstrap method. The validity of the bootstrap calibration is proved for finite samples with an explicit error bound. We provide a comprehensive theoretical study of the method and describe in detail the set of possible values of the selector $ \hat{m} $. We also establish some precise oracle error bounds for the corresponding estimator $ \hatθ = \tildeθ_{\hat{m}} $ which equally applies to estimation of the whole parameter vectors, some subvector or linear mapping, as well as the estimation of a linear functional.

math.ST↗

arXiv2015

Two convergence results for an alternation maximization procedure

Andresen and Spokoiny's (2013) ``critical dimension in semiparametric estimation`` provide a technique for the finite sample analysis of profile M-estimators. This paper uses very similar ideas to derive two convergence results for the alternating procedure to approximate the maximizer of random functionals such as the realized log likelihood in MLE estimation. We manage to show that the sequence attains the same deviation properties as shown for the profile M-estimator in Andresen and Spokoiny (2013), i.e. a finite sample Wilks and Fisher theorem. Further under slightly stronger smoothness constraints on the random functional we can show nearly linear convergence to the global maximizer if the starting point for the procedure is well chosen.

math.ST↗

arXiv2014

Critical dimension in profile semiparametric estimation

This paper revisits the classical inference results for profile quasi maximum likelihood estimators (profile MLE) in the semiparametric estimation problem. We mainly focus on two prominent theorems: the Wilks phenomenon and Fisher expansion for the profile MLE are stated in a new fashion allowing finite samples and model misspecification. The method of study is also essentially different from the usual analysis of the semiparametric problem based on the notion of the hardest parametric submodel. Instead we derive finite sample deviation bounds for the linear approximation error for the gradient of the loglikelihood. This novel approach particularly allows to address the important issue of the effective target and nuisance dimension. The obtained nonasymptotic results are surprisingly sharp and yield the classical asymptotic statements including the asymptotic normality and efficiency of the profile MLE. The general results are specified to the important special cases of an i.i.d. sample and the analysis is exemplified with a single index model.

math.ST↗

arXiv2014

Finite Sample Bernstein -- von Mises Theorem for Semiparametric Problems

The classical parametric and semiparametric Bernstein -- von Mises (BvM) results are reconsidered in a non-classical setup allowing finite samples and model misspecification. In the case of a finite dimensional nuisance parameter we obtain an upper bound on the error of Gaussian approximation of the posterior distribution for the target parameter which is explicit in the dimension of the nuisance and target parameters. This helps to identify the so called \emph{critical dimension} $ p $ of the full parameter for which the BvM result is applicable. In the important i.i.d. case, we show that the condition "$ p^{3} / n $ is small" is sufficient for BvM result to be valid under general assumptions on the model. We also provide an example of a model with the phase transition effect: the statement of the BvM theorem fails when the dimension $ p $ approaches $ n^{1/3} $. The results are extended to the case of infinite dimensional parameters with the nuisance parameter from a Sobolev class. In particular we show near normality of the posterior if the smoothness parameter $s$ exceeds 3/2.

math.ST↗

arXiv2014

Bernstein - von Mises Theorem for growing parameter dimension

This paper revisits the prominent Fisher, Wilks, and Bernstein -- von Mises (BvM) results from different viewpoints. Particular issues to address are: nonasymptotic framework with just one finite sample, possible model misspecification, and a large parameter dimension. In particular, in the case of an i.i.d. sample, the mentioned results can be stated for any smooth parametric family provided that the dimension $p $ of the parameter space satisfies the condition "$p^{2}/n $ is small" for the Fisher expansion, while the Wilks and the BvM results require "$p^{3}/n $ is small".

math.ST↗

arXiv2013

Concentration inequalities for smooth random fields

In this note we derive a sharp concentration inequality for the supremum of a smooth random field over a finite dimensional set. It is shown that this supremum can be bounded with high probability by the value of the field at some deterministic point plus an intrinsic dimension of the optimisation problem. As an application we prove the exponential inequality for a function of the maximal eigenvalue of a random matrix is proved.

math.ST↗

arXiv2013

Parametric estimation. Finite sample theory

The paper aims at reconsidering the famous Le Cam LAN theory. The main features of the approach which make it different from the classical one are as follows: (1) the study is nonasymptotic, that is, the sample size is fixed and does not tend to infinity; (2) the parametric assumption is possibly misspecified and the underlying data distribution can lie beyond the given parametric family. These two features enable to bridge the gap between parametric and nonparametric theory and to build a unified framework for statistical estimation. The main results include large deviation bounds for the (quasi) maximum likelihood and the local quadratic bracketing of the log-likelihood process. The latter yields a number of important corollaries for statistical inference: concentration, confidence and risk bounds, expansion of the maximum likelihood estimate, etc. All these corollaries are stated in a nonclassical way admitting a model misspecification and finite samples. However, the classical asymptotic results including the efficiency bounds can be easily derived as corollaries of the obtained nonasymptotic statements. At the same time, the new bracketing device works well in the situations with large or growing parameter dimension in which the classical parametric theory fails. The general results are illustrated for the i.i.d. setup as well as for generalized linear and median estimation. The results apply for any dimension of the parameter space and provide a quantitative lower bound on the sample size yielding the root-n accuracy.

math.ST↗

arXiv2013

Sharp deviation bounds for quadratic forms

This note presents sharp inequalities for deviation probability of a general quadratic form of a random vector $\xiv$ with finite exponential moments. The obtained deviation bounds are similar to the case of a Gaussian random vector. The results are stated under general conditions and do not suppose any special structure of the vector $\xiv$. The obtained bounds are exact (non-asymptotic), all constants are explicit and the leading terms in the bounds are sharp.

math.PR↗

arXiv2012

Local Quantile Regression

Quantile regression is a technique to estimate conditional quantile curves. It provides a comprehensive picture of a response contingent on explanatory variables. In a flexible modeling framework, a specific form of the conditional quantile curve is not a priori fixed. % Indeed, the majority of applications do not per se require specific functional forms. This motivates a local parametric rather than a global fixed model fitting approach. A nonparametric smoothing estimator of the conditional quantile curve requires to balance between local curvature and stochastic variability. In this paper, we suggest a local model selection technique that provides an adaptive estimator of the conditional quantile regression curve at each design point. Theoretical results claim that the proposed adaptive procedure performs as good as an oracle which would minimize the local estimation risk for the problem at hand. We illustrate the performance of the procedure by an extensive simulation study and consider a couple of applications: to tail dependence analysis for the Hong Kong stock market and to analysis of the distributions of the risk factors of temperature dynamics.

math.ST↗