Skip to content

arXiv subjects

Joseph Najnudel

Publications and source records attributed to Joseph Najnudel.

At least 37 records · Page 2Linked to original sources

arXiv2022

Universal characteristics of deep neural network loss surfaces from random matrix theory

This paper considers several aspects of random matrix universality in deep neural networks. Motivated by recent experimental work, we use universal properties of random matrices related to local statistics to derive practical implications for deep neural networks based on a realistic model of their Hessians. In particular we derive universal aspects of outliers in the spectra of deep neural networks and demonstrate the important role of random matrix local laws in popular pre-conditioning gradient descent algorithms. We also present insights into deep neural network loss surfaces from quite general arguments based on tools from statistical physics and random matrix theory.

math-ph↗

arXiv2021

Emergence of jumps in quantum trajectories via homogeneization

In the strong noise regime, we study the homogeneization of quantum trajectories i.e. stochastic processes appearing in the context of quantum measurement. When the generator of the average semi-group can be separated into three distinct time scales, we start by describing a homogenized limiting semi-group. This result is of independent interest and is formulated outside of the scope of quantum trajectories. Going back to the quantum context, we show that, in the Meyer-Zheng topology, the time-continuous quantum trajectories converge weakly to the discontinuous trajectories of a pure jump Markov process. Notably, this convergence cannot hold in the usual Skorokhod topology.

math.PR↗

arXiv2021

The Loss Surfaces of Neural Networks with General Activation Functions

The loss surfaces of deep neural networks have been the subject of several studies, theoretical and experimental, over the last few years. One strand of work considers the complexity, in the sense of local optima, of high dimensional random functions with the aim of informing how local optimisation methods may perform in such complicated settings. Prior work of Choromanska et al (2015) established a direct link between the training loss surfaces of deep multi-layer perceptron networks and spherical multi-spin glass models under some very strong assumptions on the network and its data. In this work, we test the validity of this approach by removing the undesirable restriction to ReLU activation functions. In doing so, we chart a new path through the spin glass complexity calculations using supersymmetric methods in Random Matrix Theory which may prove useful in other contexts. Our results shed new light on both the strengths and the weaknesses of spin glass models in this context.

math.PR↗

arXiv2021

A spin-glass model for the loss surfaces of generative adversarial networks

We present a novel mathematical model that seeks to capture the key design feature of generative adversarial networks (GANs). Our model consists of two interacting spin glasses, and we conduct an extensive theoretical analysis of the complexity of the model's critical points using techniques from Random Matrix Theory. The result is insights into the loss surfaces of large GANs that build upon prior insights for simpler networks, but also reveal new structure unique to this setting.

math-ph↗

arXiv2020

Feller coupling of cycles and Poisson spacings

Feller (1945) provided a coupling between the counts of cycles of various sizes in a uniform random permutation of $[n]$ and the spacings between successes in a sequence of $n$ independent Bernoulli trials with success probability $1/n$ at the $n$th trial. Arratia, Barbour and Tavaré (1992) extended Feller's coupling, to associate cycles of random permutations governed by the Ewens $(θ)$ distribution with spacings derived from independent Bernoulli trials with success probability $θ/(n-1+θ)$ at the $n$th trial, and to conclude that in an infinite sequence of such trials, the numbers of spacings of length $\ell$ are independent Poisson variables with means $θ/\ell$. Ignatov (1978) first discovered this remarkable result in the uniform case $θ= 1$, by constructing Bernoulli $(1/n)$ trials as the indicators of record values in a sequence of i.i.d. uniform $[0,1]$ variables. In the present article, the Poisson property of inhomogeneous Bernoulli spacings is explained by a variation of Ignatov's approach for a general $θ>0$. Moreover, our approach naturally provides random permutations of infinite sets whose cycle counts are exactly given by independent Poisson random variables.

math.PR↗

arXiv2020

Uniform point variance bounds in classical beta ensembles

In this paper, we give bounds on the variance of the number of points of the circular and the Gaussian $β$ ensemble in arcs of the unit circle or intervals of the real line. These bounds are logarithmic with respect to the renormalized length of these sets, which is expected to be optimal up to a multiplicative constant depending only on $β$.

math.PR↗

arXiv2020

Secular Coefficients and the Holomorphic Multiplicative Chaos

We study the secular coefficients of $N \times N$ random unitary matrices $U_{N}$ drawn from the Circular $β$-Ensemble, which are defined as the coefficients of $\{z^n\}$ in the characteristic polynomial $\det(1-zU_{N}^{*})$. When $β> 4$ we obtain a new class of limiting distributions that arise when both $n$ and $N$ tend to infinity simultaneously. We solve an open problem of Diaconis and Gamburd by showing that for $β=2$, the middle coefficient tends to zero as $N \to \infty$. We show how the theory of Gaussian multiplicative chaos (GMC) plays a prominent role in these problems and in the explicit description of the obtained limiting distributions. We extend the remarkable magic square formula of Diaconis and Gamburd for the moments of secular coefficients to all $β>0$ and analyse the asymptotic behaviour of the moments. We obtain estimates on the order of magnitude of the secular coefficients for all $β> 0,$ and these estimates are sharp when $β\geq 2$. These insights motivated us to introduce a new stochastic object associated with the secular coefficients, which we call Holomorphic Multiplicative Chaos (HMC). Viewing the HMC as a random distribution, we prove a sharp result about its regularity in an appropriate Sobolev space. Our proofs expose and exploit several novel connections with other areas, including random permutations, Tauberian theorems and combinatorics.

math.PR↗

arXiv2020

The boundary of the orbital beta process

The unitarily invariant probability measures on infinite Hermitian matrices have been classified by Pickrell, and by Olshanski and Vershik. This classification is equivalent to determining the boundary of a certain inhomogeneous Markov chain with given transition probabilities. This formulation of the problem makes sense for general $β$-ensembles when one takes as the transition probabilities the Dixon-Anderson conditional probability distribution. In this paper we determine the boundary of this Markov chain for any $β\in (0,\infty]$, also giving in this way a new proof of the classical $β=2$ case. Finally, as a by-product of our results we obtain alternative proofs of the almost sure convergence of the rescaled Hua-Pickrell and Laguerre $β$-ensembles to the general $β$ Hua-Pickrell and $β$ Bessel point processes respectively.

math.PR↗

arXiv2020

Spiking and collapsing in large noise limits of SDEs

We analyze the strong noise limit of one-dimensional stochastic differential equations (SDEs). Our initial motivation comes from continuous measurements of open quantum systems. In this context, Bauer, Bernard and Tilloy pointed out an intriguing behavior. As the noise grows larger, the solutions exhibit locally a collapsing, that is to say, converge to pure jump processes very reminiscent of a metastability phenomenon. But surprisingly the limiting jump process is decorated by a spike process. We give a precise meaning to the convergence and completely prove these statements for a large class of one-dimensional diffusions, thanks to a robust strategy of proof.

math.PR↗

arXiv2020

A characterization of limiting functions arising in mod-* convergence

In this note, we characterize the limiting functions in mod-Gausssian convergence; our approach sheds a new light on the nature of mod-Gaussian convergence as well. Our results in fact more generally apply to mod-* convergence, where * stands for any family of probability distributions whose Fourier transforms do not vanish. We moreover provide new examples, including two new examples of (restricted) mod-Cauchy convergence from arithmetics related to Dedekind sums and the linking number of modular geodesics.

math.PR↗

arXiv2020

On consecutive values of random completely multiplicative functions

In this article, we study the behavior of consecutive values of random completely multiplicative functions $(X_n)_{n \geq 1}$ whose values are i.i.d. at primes. We prove that for $X_2$ uniform on the unit circle, or uniform on the set of roots of unity of a given order, and for fixed $k \geq 1$, $X_{n+1}, \dots, X_{n+k}$ are independent if $n$ is large enough. Moreover, with the same assumption, we prove the almost sure convergence of the empirical measure $N^{-1} \sum_{n=1}^N δ_{(X_{n+1}, \dots, X_{n+k})}$ when $N$ goes to infinity, with an estimate of the rate of convergence. At the end of the paper, we also show that for any probability distribution on the unit circle followed by $X_2$, the empirical measure converges almost surely when $k=1$.

math.PR↗

arXiv2020

Exponential moments of the argument of the Riemann zeta function on the critical line

In this article, we give, under the Riemann hypothesis, an upper bound for the exponential moments of the imaginary part of the logarithm of the Riemann zeta function on the critical line. Our result, which gives information on the fluctuations of the distribution of the zeros of $ζ$, has the same accuracy as the result obtained by Soundararajan in his paper entitled "Moments of the Riemann zeta function".

math.NT↗

arXiv2020

Eigenvector convergence for minors of unitarily invariant infinite random matrices

Pickrell has fully characterized the unitarily invariant probability measures on infinite Hermitian matrices, and an alternative proof of this classification has been found by Olshanski and Vershik. Borodin and Olshanski deduced from this proof that under any of these invariant measures, the extreme eigenvalues of the minors, divided by the dimension, converge almost surely. In this paper, we prove that one also has a weak convergence for the eigenvectors, in a sense which is made precise. After mapping Hermitian to unitary matrices via the Cayley transform, our result extends a convergence proven in our paper with Maples and Nikeghbali, for which a coupling of the Circular Unitary Ensemble of all dimensions is considered.

math.PR↗

arXiv2019

On smooth mesoscopic linear statistics of the eigenvalues of random permutation matrices

We study the limiting behavior of smooth linear statistics of the spectrum of random permutation matrices in the mesoscopic regime, when the permutation follows one of the Ewens measures on the symmetric group. If we apply a smooth enough test function $f$ to all the determinations of the eigenangles of the permutations, we get a convergence in distribution when the order of the permutation tends to infinity. Two distinct kinds of limit appear: if $f(0)\neq 0$, we have a central limit theorem with a logarithmic variance, and if $f(0) = 0$, the convergence holds without normalization and the limit involves a scale-invariant Poisson point process.

math.PR↗

arXiv2019

The bead process for beta ensembles

The bead process introduced by Boutillier is a countable interlacing of the determinantal sine-kernel point processes. We construct the bead process for general sine beta processes as an infinite dimensional Markov chain whose transition mechanism is explicitly described. We show that this process is the microscopic scaling limit in the bulk of the Hermite beta corner process introduced by Gorin and Shkolnikov, generalizing the process of the minors of the Gaussian unitary and orthogonal ensembles. In order to prove our results, we use bounds on the variance of the point counting of the circular and the Gaussian beta ensembles, proven in a companion paper.

math.PR↗

arXiv2019

The limiting characteristic polynomial of classical random matrix ensembles

We demonstrate the convergence of the characteristic polynomial of several random matrix ensembles to a limiting universal function, at the microscopic scale. The random matrix ensembles we treat are classical compact groups and the Gaussian Unitary Ensemble. In fact, the result is the by-product of a general limit theorem for the convergence of random entire functions whose zeros present a simple regularity property.

math.PR↗

arXiv2018

Rigidity of the $\operatorname{Sine}_β$ process

We show that the $\operatorname{Sine}_β$ point process, defined as the scaling limit of the Circular Beta Ensemble when the dimension goes to infinity, and generalizing the determinantal sine-kernel process, is rigid in the sense of Ghosh and Peres: the number of points in a given bounded Borel set $B$ is almost surely equal to a measurable function of the position of the points outside $B$.

math.PR↗

arXiv2018

On the extreme values of the Riemann zeta function on random intervals of the critical line

In the present paper, we show that under the Riemann hypothesis, and for fixed $h, ε> 0$, the supremum of the real and the imaginary parts of $\log ζ(1/2 + it)$ for $t \in [UT -h, UT + h]$ are in the interval $[(1-ε) \log \log T, (1+ ε) \log \log T]$ with probability tending to $1$ when $T$ goes to infinity, if $U$ is uniformly distributed in $[0,1]$. This proves a weak version of a conjecture by Fyodorov, Hiary and Keating, which has recently been intensively studied in the setting of random matrices. We also unconditionally show that the supremum of $\Re \log ζ(1/2 + it)$ is at most $\log \log T + g(T)$ with probability tending to $1$, $g$ being any function tending to infinity at infinity.

math.NT↗