arXiv subjects

Alexey Naumov

Publications and source records attributed to Alexey Naumov.

At least 55 records · Page 3Linked to original sources

arXiv2022

Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations

This paper investigates the approximation properties of deep neural networks with piecewise-polynomial activation functions. We derive the required depth, width, and sparsity of a deep neural network to approximate any Hölder smooth function up to a given approximation error in Hölder norms in such a way that all weights of this neural network are bounded by $1$. The latter feature is essential to control generalization errors in many statistical and machine learning applications.

math.NA↗

arXiv2022

Local-Global MCMC kernels: the best of both worlds

Recent works leveraging learning to enhance sampling have shown promising results, in particular by designing effective non-local moves and global proposals. However, learning accuracy is inevitably limited in regions where little data is available such as in the tails of distributions as well as in high-dimensional problems. In the present paper we study an Explore-Exploit Markov chain Monte Carlo strategy ($Ex^2MCMC$) that combines local and global samplers showing that it enjoys the advantages of both approaches. We prove $V$-uniform geometric ergodicity of $Ex^2MCMC$ without requiring a uniform adaptation of the global sampler to the target distribution. We also compute explicit bounds on the mixing rate of the Explore-Exploit strategy under realistic conditions. Moreover, we also analyze an adaptive version of the strategy ($FlEx^2MCMC$) where a normalizing flow is trained while sampling to serve as a proposal for global moves. We illustrate the efficiency of $Ex^2MCMC$ and its adaptive version on classical sampling benchmarks as well as in sampling high-dimensional distributions defined by Generative Adversarial Networks seen as Energy Based Models. We provide the code to reproduce the experiments at the link: https://github.com/svsamsonov/ex2mcmc_new.

stat.ML↗

arXiv2022

Optimistic Posterior Sampling for Reinforcement Learning with Few Samples and Tight Guarantees

We consider reinforcement learning in an environment modeled by an episodic, finite, stage-dependent Markov decision process of horizon $H$ with $S$ states, and $A$ actions. The performance of an agent is measured by the regret after interacting with the environment for $T$ episodes. We propose an optimistic posterior sampling algorithm for reinforcement learning (OPSRL), a simple variant of posterior sampling that only needs a number of posterior samples logarithmic in $H$, $S$, $A$, and $T$ per state-action pair. For OPSRL we guarantee a high-probability regret bound of order at most $\widetilde{\mathcal{O}}(\sqrt{H^3SAT})$ ignoring $\text{poly}\log(HSAT)$ terms. The key novel technical ingredient is a new sharp anti-concentration inequality for linear forms which may be of independent interest. Specifically, we extend the normal approximation-based lower bound for Beta distributions by Alfers and Dinges [1984] to Dirichlet distributions. Our bound matches the lower bound of order $Ω(\sqrt{H^3SAT})$, thereby answering the open problems raised by Agrawal and Jia [2017b] for the episodic setting.

stat.ML↗

arXiv2022

From Dirichlet to Rubin: Optimistic Exploration in RL without Bonuses

We propose the Bayes-UCBVI algorithm for reinforcement learning in tabular, stage-dependent, episodic Markov decision process: a natural extension of the Bayes-UCB algorithm by Kaufmann et al. (2012) for multi-armed bandits. Our method uses the quantile of a Q-value function posterior as upper confidence bound on the optimal Q-value function. For Bayes-UCBVI, we prove a regret bound of order $\widetilde{O}(\sqrt{H^3SAT})$ where $H$ is the length of one episode, $S$ is the number of states, $A$ the number of actions, $T$ the number of episodes, that matches the lower-bound of $Ω(\sqrt{H^3SAT})$ up to poly-$\log$ terms in $H,S,A,T$ for a large enough $T$. To the best of our knowledge, this is the first algorithm that obtains an optimal dependence on the horizon $H$ (and $S$) without the need for an involved Bernstein-like bonus or noise. Crucial to our analysis is a new fine-grained anti-concentration bound for a weighted Dirichlet sum that can be of independent interest. We then explain how Bayes-UCBVI can be easily extended beyond the tabular setting, exhibiting a strong link between our algorithm and Bayesian bootstrap (Rubin, 1981).

stat.ML↗

arXiv2021

Tight High Probability Bounds for Linear Stochastic Approximation with Fixed Stepsize

This paper provides a non-asymptotic analysis of linear stochastic approximation (LSA) algorithms with fixed stepsize. This family of methods arises in many machine learning tasks and is used to obtain approximate solutions of a linear system $\bar{A}θ= \bar{b}$ for which $\bar{A}$ and $\bar{b}$ can only be accessed through random estimates $\{({\bf A}_n, {\bf b}_n): n \in \mathbb{N}^*\}$. Our analysis is based on new results regarding moments and high probability bounds for products of matrices which are shown to be tight. We derive high probability bounds on the performance of LSA under weaker conditions on the sequence $\{({\bf A}_n, {\bf b}_n): n \in \mathbb{N}^*\}$ than previous works. However, in contrast, we establish polynomial concentration bounds with order depending on the stepsize. We show that our conclusions cannot be improved without additional assumptions on the sequence of random matrices $\{{\bf A}_n: n \in \mathbb{N}^*\}$, and in particular that no Gaussian or exponential high probability bounds can hold. Finally, we pay a particular attention to establishing bounds with sharp order with respect to the number of iterations and the stepsize and whose leading terms contain the covariance matrices appearing in the central limit theorems.

stat.ML↗

arXiv2021

On the Stability of Random Matrix Product with Markovian Noise: Application to Linear Stochastic Approximation and TD Learning

This paper studies the exponential stability of random matrix products driven by a general (possibly unbounded) state space Markov chain. It is a cornerstone in the analysis of stochastic algorithms in machine learning (e.g. for parameter tracking in online learning or reinforcement learning). The existing results impose strong conditions such as uniform boundedness of the matrix-valued functions and uniform ergodicity of the Markov chains. Our main contribution is an exponential stability result for the $p$-th moment of random matrix product, provided that (i) the underlying Markov chain satisfies a super-Lyapunov drift condition, (ii) the growth of the matrix-valued functions is controlled by an appropriately defined function (related to the drift condition). Using this result, we give finite-time $p$-th moment bounds for constant and decreasing stepsize linear stochastic approximation schemes with Markovian noise on general state space. We illustrate these findings for linear value-function estimation in reinforcement learning. We provide finite-time $p$-th moment bound for various members of temporal difference (TD) family of algorithms.

stat.ML↗

arXiv2020

Finite Time Analysis of Linear Two-timescale Stochastic Approximation with Markovian Noise

Linear two-timescale stochastic approximation (SA) scheme is an important class of algorithms which has become popular in reinforcement learning (RL), particularly for the policy evaluation problem. Recently, a number of works have been devoted to establishing the finite time analysis of the scheme, especially under the Markovian (non-i.i.d.) noise settings that are ubiquitous in practice. In this paper, we provide a finite-time analysis for linear two timescale SA. Our bounds show that there is no discrepancy in the convergence rate between Markovian and martingale noise, only the constants are affected by the mixing time of the Markov chain. With an appropriate step size schedule, the transient term in the expected error bound is $o(1/k^c)$ and the steady-state term is ${\cal O}(1/k)$, where $c>1$ and $k$ is the iteration number. Furthermore, we present an asymptotic expansion of the expected error with a matching lower bound of $Ω(1/k)$. A simple numerical experiment is presented to support our theory.

stat.ML↗

arXiv2019

Local semicircle law under fourth moment condition

We consider a random symmetric matrix ${\bf X} = [X_{jk}]_{j,k=1}^n$ with upper triangular entries being independent random variables with mean zero and unit variance. Assuming that $\max_{jk} {\mathbb E} |X_{jk}|^{4+δ} < \infty, δ> 0$, it was proved in [Götze, Naumov and Tikhomirov, Bernoulli, 2018] that with high probability the typical distance between the Stieltjes transforms $m_n(z), z = u + i v$, of the empirical spectral distribution (ESD) and the Stieltjes transforms $m_{sc}(z)$ of the semicircle law is of order $(nv)^{-1} \log n$. The aim of this paper is to remove $δ>0$ and show that this result still holds if we assume that $\max_{jk} {\mathbb E} |X_{jk}|^{4} < \infty$. We also discuss applications to the rate of convergence of the ESD to the semicircle law in the Kolmogorov distance, rates of localization of the eigenvalues around the classical positions and rates of delocalization of eigenvectors.

math.PR↗

arXiv2018

On Local laws for non-Hermitian random matrices and their products

The aim of this paper is to prove a local version of the circular law for non-Hermitian random matrices and its generalization to the product of non-Hermitian random matrices under weak moment conditions. More precisely we assume that the entries $X_{jk}^{(q)}$ of non-Hermitian random matrices ${\bf X}^{(q)}, 1 \le j,k \le n, q = 1, \ldots, m, m \geq 1$ are i.i.d. r.v. with $\mathbb E X_{jk} =0, \mathbb E X_{jk}^2 = 1$ and $\mathbb E |X_{jk}|^{4+δ} < \infty$ for some $δ> 0$. It is shown that the local law holds on the optimal scale $n^{-1+2a}, a > 0$, up to some logarithmic factor. We further develop a Stein type method to estimate the perturbation of the equations for the Stieltjes transform of the limiting distribution. We also generalize the recent results [Bourgade--Yau-Yin, 2014], [Tao--Vu, 2015] and [Nemish, 2017]. An extension to the case of non-i.i.d. entries is discussed.

math.PR↗

arXiv2018

Large ball probability, Gaussian comparison and anti-concentration

We derive tight non-asymptotic bounds for the Kolmogorov distance between the probabilities of two Gaussian elements to hit a ball in a Hilbert space. The key property of these bounds is that they are dimension-free and depend on the nuclear (Schatten-one) norm of the difference between the covariance operators of the elements and on the norm of the mean shift. The obtained bounds significantly improve the bound based on Pinsker's inequality via the Kullback-Leibler divergence. We also establish an anti-concentration bound for a squared norm of a non-centered Gaussian element in Hilbert space. The paper presents a number of examples motivating our results and applications of the obtained bounds to statistical inference and to high-dimensional CLT.

math.PR↗

arXiv2017

Bootstrap confidence sets for spectral projectors of sample covariance

Let $X_{1},\ldots,X_{n}$ be i.i.d. sample in $\mathbb{R}^{p}$ with zero mean and the covariance matrix $\mathbfΣ$. The problem of recovering the projector onto an eigenspace of $\mathbfΣ$ from these observations naturally arises in many applications. Recent technique from [Koltchinskii, Lounici, 2015] helps to study the asymptotic distribution of the distance in the Frobenius norm $\| \mathbf{P}_r - \widehat{\mathbf{P}}_r \|_{2}$ between the true projector $\mathbf{P}_r$ on the subspace of the $r$-th eigenvalue and its empirical counterpart $\widehat{\mathbf{P}}_r$ in terms of the effective rank of $\mathbfΣ$. This paper offers a bootstrap procedure for building sharp confidence sets for the true projector $\mathbf{P}_r$ from the given data. This procedure does not rely on the asymptotic distribution of $\| \mathbf{P}_r - \widehat{\mathbf{P}}_r \|_{2}$ and its moments. It could be applied for small or moderate sample size $n$ and large dimension $p$. The main result states the validity of the proposed procedure for finite samples with an explicit error bound for the error of bootstrap approximation. This bound involves some new sharp results on Gaussian comparison and Gaussian anti-concentration in high-dimensional spaces. Numeric results confirm a good performance of the method in realistic examples.

math.ST↗

arXiv2016

Local semicircle law under moment conditions. Part I: The Stieltjes transform

We consider a random symmetric matrix ${\bf X} = [X_{jk}]_{j,k=1}^n$ in which the upper triangular entries are independent identically distributed random variables with mean zero and unit variance. We additionally suppose that $\mathbb E |X_{11}|^{4 + δ} =: μ_4 < \infty$ for some $δ> 0$. Under these conditions we show that the typical distance between the Stieltjes transform of the empirical spectral distribution (ESD) of the matrix $n^{-\frac{1}{2}} {\bf X}$ and Wigner's semicircle law is of order $(nv)^{-1}$, where $v$ is the distance in the complex plane to the real line. Furthermore we outline applications which are deferred to a subsequent paper, such as the rate of convergence in probability of the ESD to the distribution function of the semicircle law, rigidity of the eigenvalues and eigenvector delocalization.

math.PR↗

arXiv2016

Local semicircle law under moment conditions. Part II: Localization and delocalization

We consider a random symmetric matrix ${\bf X} = [X_{jk}]_{j,k=1}^n$ with upper triangular entries being independent identically distributed random variables with mean zero and unit variance. We additionally suppose that $\mathbb E |X_{11}|^{4 + δ} =: μ_{4+δ} < C$ for some $δ> 0$ and some absolute constant $C$. Under these conditions we show that the typical Kolmogorov distance between the empirical spectral distribution function of eigenvalues of $n^{-1/2} {\bf X}$ and Wigner's semicircle law is of order $1/n$ up to some logarithmic correction factor. As a direct consequence of this result we establish that the semicircle law holds on a short scale. Furthermore, we show for this finite moment ensemble rigidity of eigenvalues and delocalization properties of the eigenvectors. Some numerical experiments are included illustrating the influence of the tail behavior of the matrix entries when only a small number of moments exist.

math.PR↗

arXiv2016

On the Local Semicircular Law for Wigner Ensembles

We consider a random symmetric matrix ${\bf X} = [X_{jk}]_{j,k=1}^n$ with upper triangular entries being i.i.d. random variables with mean zero and unit variance. We additionally suppose that $\mathbb E |X_{11}|^{4 + δ} =: μ_{4+δ} < \infty$ for some $δ> 0$. The aim of this paper is to significantly extend recent result of the authors [18] and show that with high probability the typical distance between the Stieltjes transform of the empirical spectral distribution (ESD) of the matrix $n^{-\frac{1}{2}} {\bf X}$ and Wigner's semicircle law is of order $(nv)^{-1} \log n$, where $v$ denotes the distance to the real line in the complex plane. We apply this result to the rate of convergence of the ESD to the distribution function of the semicircle law as well as to rigidity of eigenvalues and eigenvector delocalization significantly extending a recent result by Götze, Naumov and Tikhomirov [19]. The result on delocalization is optimal by comparison with GOE ensembles. Furthermore the techniques of this paper provide a new shorter proof for the optimal $O(n^{-1})$ rate of convergence of the expected ESD to the semicircle law.

math.PR↗

arXiv2016

Asymptotic analysis of symmetric functions

In this paper we consider asymptotic expansions for a class of sequences of symmetric functions of many variables. Applications to classical and free probability theory are discussed.

math.PR↗

arXiv2015

Distribution of Linear Statistics of Singular Values of the Product of Random Matrices

In this paper we consider the product of two independent random matrices $\mathbb X^{(1)}$ and $\mathbb X^{(2)}$. Assume that $X_{jk}^{(q)}, 1 \le j,k \le n, q = 1, 2,$ are i.i.d. random variables with $\mathbb E X_{jk}^{(q)} = 0, \mathbb E (X_{jk}^{(q)})^2 = 1$. Denote by $s_1, ..., s_n$ the singular values of $\mathbb W: = \frac{1}{n} \mathbb X^{(1)} \mathbb X^{(2)}$. We prove the central limit theorem for linear statistics of the squared singular values $s_1^2, ..., s_n^2$ showing that the limiting variance depends on $κ_4: = \mathbb E (X_{11}^{1})^4 - 3$.

math.PR↗

arXiv2014

On one generalization of the elliptic law for random matrices

We consider the products of $m\ge 2$ independent large real random matrices with independent vectors $(X_{jk}^{(q)},X_{kj}^{(q)})$ of entries. The entries $X_{jk}^{(q)},X_{kj}^{(q)}$ are correlated with $ρ=\mathbb E X_{jk}^{(q)}X_{kj}^{(q)}$. The limit distribution of the empirical spectral distribution of the eigenvalues of such products doesn't depend on $ρ$ and equals to the distribution of $m$th power of the random variable uniformly distributed on the unit disc.

math.PR↗

arXiv2013

On minimal singular values of random matrices with correlated entries

Let $\mathbf X$ be a random matrix whose pairs of entries $X_{jk}$ and $X_{kj}$ are correlated and vectors $ (X_{jk},X_{kj})$, for $1\le j 0$ and $Q\ge 0$. Let $s_n(\mathbf X+\mathbf M_n)$ denote the least singular value of the matrix $\mathbf X+\mathbf M_n$. It is shown that there exist positive constants $A$ and $B$ depending on $K,Q,ρ$ only such that $$ \mathbb{P}(s_n(\mathbf X+\mathbf M_n)\le n^{-A})\le n^{-B}. $$ As an application of this result we prove the elliptic law for this class of matrices with non identically distributed correlated entries.

math.PR↗