arXiv subjects

Elliot Paquette

Publications and source records attributed to Elliot Paquette.

At least 55 records · Page 3Linked to original sources

arXiv2021

SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticality

We propose a new framework, inspired by random matrix theory, for analyzing the dynamics of stochastic gradient descent (SGD) when both number of samples and dimensions are large. This framework applies to any fixed stepsize and the finite sum setting. Using this new framework, we show that the dynamics of SGD on a least squares problem with random data become deterministic in the large sample and dimensional limit. Furthermore, the limiting dynamics are governed by a Volterra integral equation. This model predicts that SGD undergoes a phase transition at an explicitly given critical stepsize that ultimately affects its convergence rate, which we also verify experimentally. Finally, when input data is isotropic, we provide explicit expressions for the dynamics and average-case convergence rates (i.e., the complexity of an algorithm averaged over all possible inputs). These rates show significant improvement over the worst-case complexities.

math.OC↗

arXiv2020

Secular Coefficients and the Holomorphic Multiplicative Chaos

We study the secular coefficients of $N \times N$ random unitary matrices $U_{N}$ drawn from the Circular $β$-Ensemble, which are defined as the coefficients of $\{z^n\}$ in the characteristic polynomial $\det(1-zU_{N}^{*})$. When $β> 4$ we obtain a new class of limiting distributions that arise when both $n$ and $N$ tend to infinity simultaneously. We solve an open problem of Diaconis and Gamburd by showing that for $β=2$, the middle coefficient tends to zero as $N \to \infty$. We show how the theory of Gaussian multiplicative chaos (GMC) plays a prominent role in these problems and in the explicit description of the obtained limiting distributions. We extend the remarkable magic square formula of Diaconis and Gamburd for the moments of secular coefficients to all $β>0$ and analyse the asymptotic behaviour of the moments. We obtain estimates on the order of magnitude of the secular coefficients for all $β> 0,$ and these estimates are sharp when $β\geq 2$. These insights motivated us to introduce a new stochastic object associated with the secular coefficients, which we call Holomorphic Multiplicative Chaos (HMC). Viewing the HMC as a random distribution, we prove a sharp result about its regularity in an appropriate Sobolev space. Our proofs expose and exploit several novel connections with other areas, including random permutations, Tauberian theorems and combinatorics.

math.PR↗

arXiv2020

Topology of random 2-dimensional cubical complexes

We study a natural model of random 2-dimensional cubical complex which is a subcomplex of an n-dimensional cube, and where every possible square $2$-face is included independently with probability p. Our main result is to exhibit a sharp threshold p=1/2 for homology vanishing as $n \to \infty$. This is a 2-dimensional analogue of the Burtin and Erdős-Spencer theorems characterizing the connectivity threshold for random cubical graphs. Our main result can also be seen as a cubical counterpart to the Linial--Meshulam theorem for random 2-dimensional simplicial complexes. However, the models exhibit strikingly different behaviors. We show that if $p > 1 - \sqrt{1/2} \approx 0.2929$, then with high probability the fundamental group is a free group with one generator for every maximal $1$-dimensional face. As a corollary, homology vanishing and simple connectivity have the same threshold, even in the strong "hitting time" sense. This is in contrast with the simplicial case, where the thresholds are far apart. The proof depends on an iterative algorithm for contracting cycles -- we show that with high probability the algorithm rapidly and dramatically simplifies the fundamental group, converging after only a few steps.

math.CO↗

arXiv2020

Universality for the conjugate gradient and MINRES algorithms on sample covariance matrices

We present a probabilistic analysis of two Krylov subspace methods for solving linear systems. We prove a central limit theorem for norms of the residual vectors that are produced by the conjugate gradient and MINRES algorithms when applied to a wide class of sample covariance matrices satisfying some standard moment conditions. The proof involves establishing a four moment theorem for the so-called spectral measure, implying, in particular, universality for the matrix produced by the Lanczos iteration. The central limit theorem then implies an almost-deterministic iteration count for the iterative methods in question.

math.NA↗

arXiv2020

Interval fragmentations with choice: equidistribution and the evolution of tagged fragments

We consider a Markovian evolution on point processes, the $Ψ$--process, on the unit interval in which points are added according to a rule that depends only on the spacings of the existing point configuration. Having chosen a spacing, a new point is added uniformly within it. Building on previous work of the authors and of Junge, we show that the empirical distribution of points in such a process is always equidistributed under mild assumptions on the rule, generalizing work of Junge. A major portion of this article is devoted to the study of a particular growth--fragmentation process, or cell process, which is a type of piecewise--deterministic Markov process (PDMP). This process represents a linearized version of a size--biased sampling from the $Ψ$--process. We show that this PDMP is ergodic and develop the semigroup theory of it, to show that it describes a linearized version of the $Ψ$--process. This PDMP has appeared in other contexts, and in some sense we develop its theory under minimal assumptions.

math.PR↗

arXiv2020

Gaussian analytic functions of bounded mean oscillation

We consider random analytic functions given by a Taylor series with independent, centered complex Gaussian coefficients. We give a new sufficient condition for such a function to have bounded mean oscillations. Under a mild regularity assumption this condition is optimal. Using a theorem of Holland and Walsh, we give as a corollary a new bound for the norm of a random Gaussian Hankel matrix. Finally, we construct some exceptional Gaussian analytic functions which in particular disprove the conjecture that a random analytic function with bounded mean oscillations always has vanishing mean oscillations.

math.CV↗

arXiv2019

On the speed of distance stationary sequences

We prove a formula for the speed of distance stationary random sequences generalizing the law of large numbers of Karlsson and Ledrappier. A particular case is the classical formula for the largest Lyapunov exponent of i.i.d.\ matrix products, but our result has applications in various different contexts. In many situations it gives a method to estimate the speed, and in others it allows to obtain results of dimension drop for escape measures related to random walks. We show applications to stationary reversible random trees with conductances, Bernoulli bond percolation of Cayley graphs, and random walks on cocompact Fuchsian groups.

math.PR↗

arXiv2019

Spectrum of random perturbations of Toeplitz matrices with finite symbols

Let $T_N$ denote an $N\times N$ Toeplitz matrix with finite, $N$ independent symbol ${\bf a}$. For $E_N$ a noise matrix satisfying mild assumptions (ensuring, in particular, that $N^{-1/2}\|E_N\|_{\rm HS}\to_{N\to\infty} 0$ at a polynomial rate), we prove that the empirical measure of eigenvalues of $T_N+E_N$ converges to the law of ${\bf a}(U)$, where $U$ is uniformly distributed on the unit circle in the complex plane. This extends results from arXiv:1712.00042 to the non-triangular setup and non complex Gaussian noise, and confirms predictions obtained in Reichel and Trefethen (1992) using the notion of pseudo-spectrum.

math.PR↗

arXiv2019

Spectral gaps of random graphs and applications

We study the spectral gap of the Erdős--Rényi random graph through the connectivity threshold. In particular, we show that for any fixed $δ> 0$ if $$p \ge \frac{(1/2 + δ) \log n}{n},$$ then the normalized graph Laplacian of an Erdős--Rényi graph has all of its nonzero eigenvalues tightly concentrated around $1$. We estimate both the decay rate of the spectral gap to $1$ and the failure probability, up to a constant factor. We also show that the $1/2$ in the above is optimal, and that if $p = \frac{c \log n}{n}$ for $c < 1/2,$ then there are eigenvalues of the Laplacian restricted to the giant component that are separated from $1.$ We then describe several applications of our spectral gap results to stochastic topology and geometric group theory. These all depend on Garland's "p-adic curvature" method, a kind of spectral geometry for simplicial complexes. These can all be considered to be high-dimensional expander properties.

math.CO↗

arXiv2019

The maximum of the CUE field

Let $U_N$ denote a Haar Unitary matrix of dimension N, and consider the field \[ {\bf U}(z) = \log |\det(1-zU_N)| \] for z in the unit disk. Then, \[ \frac{\max_{|z|=1} {\bf U}(z) -\log N + \frac{3}{4} \log\log N} {\log\log N} \to 0 \] in probability. This provides a verification up to second order of a conjecture of Fyodorov, Hiary and Keating, improving on the recent first order verification of Arguin, Belius and Bourgade.

math.PR↗

arXiv2018

Regularization of non-normal matrices by Gaussian noise - the banded Toeplitz and twisted Toeplitz cases

We consider the spectrum of additive, polynomially vanishing random perturbations of deterministic matrices, as follows. Let $M_N$ be a deterministic $N\times N$ matrix, and let $G_N$ be a complex Ginibre matrix. We consider the matrix $\mathcal{M}_N=M_N+N^{-γ}G_N$, where $γ>1/2$. With $L_N$ the empirical measure of eigenvalues of $\mathcal{M}_N$, we provide a general deterministic equivalence theorem that ties $L_N$ to the singular values of $z-M_N$, with $z\in \mathbb{C}$. We then compute the limit of $L_N$ when $M_N$ is an upper triangular Toeplitz matrix of finite symbol: if $M_N=\sum_{i=0}^{\mathfrak{d}} a_i J^i$ where $\mathfrak{d}$ is fixed, $a_i\in\mathcal{C}$ are deterministic scalars and $J$ is the nilpotent matrix $J(i,j)={\bf 1}_{j=i+1}$, then $L_N$ converges, as $N\to\infty$, to the law of $\sum_{i=0}^{\mathfrak{d}} a_i U^i$ where $U$ is a uniform random variable on the unit circle in the complex plane. We also consider the case of slowly varying diagonals (twisted Toeplitz matrices), and, when $\mathfrak{d}=1$, also of i.i.d.~entries on the diagonals in $M_N$.

math.PR↗

arXiv2018

The integer homology threshold in $Y_d(n, p)$

We prove that in the $d$-dimensional Linial--Meshulam stochastic process the $(d - 1)$st homology group with integer coefficients vanishes exactly when the final isolated $(d - 1)$-dimensional face is covered by a top-dimensional face. This generalizes the $d = 2$ case proved recently by Łuczak and Peled and establishes that $p = \frac{d \log n}{n}$ is the sharp threshold for homology with integer coefficients to vanish in $Y_d(n, p),$ answering a 2003 question of Linial and Meshulam.

math.CO↗

arXiv2018

The maximum deviation of the Sine$_β$ counting process

In this paper, we consider the maximum of the $\text{Sine}_β$ counting process from its expectation. We show the leading order behavior is consistent with the predictions of log-correlated Gaussian fields, also consistent with work on the imaginary part of the log-characteristic polynomial of random matrices. We do this by a direct analysis of the stochastic sine equation, which gives a description of the continuum limit of the Prüfer phases of a Gaussian $β$-ensemble matrix.

math.PR↗

arXiv2018

Choices and intervals

We consider a random interval splitting process, in which the splitting rule depends on the empirical distribution of interval lengths. We show that this empirical distribution converges to a limit almost surely as the number of intervals goes to infinity. We give a characterization of this limit as a solution of an ODE and use this to derive precise tail estimates. The convergence is established by showing that the size-biased empirical distribution evolves in the limit according to a certain deterministic evolution equation. Although this equation involves a non-local, non-linear operator, it can be studied thanks to a carefully chosen norm with respect to which this operator is contractive. In finite-dimensional settings, convergence results like this usually go under the name of stochastic approximation and can be approached by a general method of Kushner and Clark. An important technical contribution of this article is the extension of this method to an infinite-dimensional setting.

math.PR↗

arXiv2018

Surjectivity of near square random matrices

We show that a nearly square iid random integral matrix is surjective over the integral lattice with very high probability. This answers a question by Koplewitz. Our result extends to sparse matrices as well as to matrices of dependent entries.

math.ST↗

arXiv2017

A Furstenberg type formula for the speed of distance stationary sequences

We prove a formula for the speed of distance stationary random sequences. A particular case is the classical formula for the largest Lyapunov exponent of an i.i.d. product of two by two matrices in terms of a stationary measure on projective space. We apply this result to Poisson-Delaunay random walks on Riemannian symmetric spaces. In particular, we obtain sharp estimates for the asymptotic behavior of the speed of hyperbolic Poisson-Delaunay random walks when the intensity of the Poisson point process goes to zero. This allows us to prove that a dimension drop phenomena occurs for the harmonic measure associated to these random walks. With the same technique we give examples of co-compact Fuchsian groups for which the harmonic measure of the simple random walk has dimension less than one.

math.PR↗

arXiv2017

Tridiagonal Models for Dyson Brownian Motion

In this paper, we consider tridiagonal matrices the eigenvalues of which evolve according to $β$-Dyson Brownian motion. This is the stochastic gradient flow on $\mathbb{R}^n$ given by, for all $1 \leq i \leq n,$ \[ dλ_{i,t} = \sqrt{\frac{2}β}dZ_{i,t} - \biggl( \frac{V'(λ_i)}{2} - \sum_{j: j \neq i} \frac{1}{λ_i - λ_j} \biggr)\,dt \] where $V$ is a constraining potential and $\left\{ Z_{i,t} \right\}_1^n$ are independent standard Brownian motions. This flow is stationary with respect to the distribution \[ ρ^β_N(λ) = \frac{1}{Z^β_N} e^{-\fracβ{2} \left( -\sum_{1 \leq i \neq j \leq N} \log|λ_i - λ_j| + \sum_{i=1}^N V(λ_i) \right) }. \] The particular choice of $V(t)=2t^2$ leads to an eigenvalue distribution constrained to lie roughly in $(-\sqrt{n},\sqrt{n}).$ We study evolution of the entries of one choice of tridiagonal flow for this $V$ in the $n\to \infty$ limit. On the way to describing the evolution of the tridiagonal matrices we give the derivative of the Lanczos tridiagonalization algorithm under perturbation.

math.PR↗

arXiv2017

Distributional Lattices on Riemannian symmetric spaces

A Riemannian symmetric space is a Riemannian manifold in which it is possible to reflect all geodesics through a point by an isometry of the space. On such spaces, we introduce the notion of a distributional lattice, generalizing the notion of lattice. Distributional lattices exist in any Riemannian symmetric space: the Voronoi tessellation of a stationary Poisson point process is an example. We show that for an appropriate notion of amenability, the amenability of a distributional lattice is equivalent to the amenability of the ambient space. Using this equivalence, we show that the simple random walk on any distributional lattice in a nonamenable space has positive embedded speed. For nonpositively curved, simply connected spaces, we show that the simple random walk on a Poisson--Voronoi tessellation has positive graph speed by developing some additional structure for Poisson--Voronoi tessellations.

math.PR↗