arXiv subjects

Gérard Ben Arous

Publications and source records attributed to Gérard Ben Arous.

At least 19 recordsLinked to original sources

arXiv2026

Langevin dynamics for high-dimensional optimization: the case of multi-spiked tensor PCA

We study nonconvex optimization in high dimensions through Langevin dynamics, focusing on the multi-spiked tensor PCA problem. In this tensor estimation model, the goal is to recover a finite number of hidden signal vectors, or spikes, from noisy Gaussian tensor observations using maximum likelihood estimation. We characterize the number of samples required for Langevin dynamics to efficiently recover the spikes and identify the separation conditions on the signal-to-noise ratios (SNRs) needed for exact recovery. In particular, we show that the sample complexity required to recover the spike associated with the largest SNR matches the well-known algorithmic threshold for the single-spike case, whereas the threshold degrades when recovering all spikes. A key ingredient is a precise low-dimensional description of the Langevin trajectory through its correlations with the spikes, which captures both the high-dimensional dynamics and the interactions among competing signal directions.

stat.ML↗

arXiv2025

Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws

We study the optimization and sample complexity of gradient-based training of a two-layer neural network with quadratic activation function in the high-dimensional regime, where the data is generated as $f_*(\boldsymbol{x}) \propto \sum_{j=1}^{r}λ_j σ\left(\langle \boldsymbol{θ_j}, \boldsymbol{x}\rangle\right), \boldsymbol{x} \sim N(0,\boldsymbol{I}_d)$, $σ$ is the 2nd Hermite polynomial, and $\lbrace\boldsymbolθ_j \rbrace_{j=1}^{r} \subset \mathbb{R}^d$ are orthonormal signal directions. We consider the extensive-width regime $r \asymp d^β$ for $β\in [0, 1)$, and assume a power-law decay on the (non-negative) second-layer coefficients $λ_j\asymp j^{-α}$ for $α\geq 0$. We present a sharp analysis of the SGD dynamics in the feature learning regime, for both the population limit and the finite-sample (online) discretization, and derive scaling laws for the prediction risk that highlight the power-law dependencies on the optimization time, sample size, and model width. Our analysis combines a precise characterization of the associated matrix Riccati differential equation with novel matrix monotonicity arguments to establish convergence guarantees for the infinite-dimensional effective dynamics.

stat.ML↗

arXiv2025

Stochastic gradient descent in high dimensions for multi-spiked tensor PCA

We study the high-dimensional dynamics of online stochastic gradient descent (SGD) for the multi-spiked tensor model. This multi-index model arises from the tensor principal component analysis (PCA) problem with multiple spikes, where the goal is to estimate $r$ unknown signal vectors within the $N$-dimensional unit sphere through maximum likelihood estimation from noisy observations of a $p$-tensor. We determine the number of samples and the conditions on the signal-to-noise ratios (SNRs) required to efficiently recover the unknown spikes from natural random initializations. We show that full recovery of all spikes is possible provided a number of sample scaling as $N^{p-2}$, matching the algorithmic threshold identified in the rank-one case [Ben Arous, Gheissari, Jagannath 2020, 2021]. Our results are obtained through a detailed analysis of a low-dimensional system that describes the evolution of the correlations between the estimators and the spikes, while controlling the noise in the dynamics. We find that the spikes are recovered sequentially in a process we term "sequential elimination": once a correlation exceeds a critical threshold, all correlations sharing a row or column index become sufficiently small, allowing the next correlation to grow and become macroscopic. The order in which correlations become macroscopic depends on their initial values and the corresponding SNRs, leading to either exact recovery or recovery of a permutation of the spikes. In the matrix case, when $p=2$, if the SNRs are sufficiently separated, we achieve exact recovery of the spikes, whereas equal SNRs lead to recovery of the subspace spanned by them.

stat.ML↗

arXiv2025

Permutation recovery of spikes in noisy high-dimensional tensor estimation

We study the dynamics of gradient flow in high dimensions for the multi-spiked tensor problem, where the goal is to estimate $r$ unknown signal vectors (spikes) from noisy Gaussian tensor observations. Specifically, we analyze the maximum likelihood estimation procedure, which involves optimizing a highly nonconvex random function. We determine the sample complexity required for gradient flow to efficiently recover all spikes, without imposing any assumptions on the separation of the signal-to-noise ratios (SNRs). More precisely, our results provide the sample complexity required to guarantee recovery of the spikes up to a permutation. Our work builds on our companion paper [Ben Arous, Gerbelot, Piccolo 2024], which studies Langevin dynamics and determines the sample complexity and separation conditions for the SNRs necessary for ensuring exact recovery of the spikes (where the recovered permutation matches the identity). During the recovery process, the correlations between the estimators and the hidden vectors increase in a sequential manner. The order in which these correlations become significant depends on their initial values and the corresponding SNRs, which ultimately determines the permutation of the recovered spikes.

math.PR↗

arXiv2025

Scaling limit for the random walk on critical lattice trees

We prove a scaling limit theorem for the simple random walk on critical lattice trees in $\mathbb{Z}^d$, for $d\geq 8$. The scaling limit is the Brownian motion on the Integrated Super-Brownian Excursion (BISE) which is the same one that we have identified earlier for other simpler models of anomalous diffusion on critical graphs in large enough dimension. The proof of this theorem is based on a combination of the tools of lace-expansion (contained in the articles \cite{CFHP} and \cite{CFHP2}), and a new and general convergence theorem.

math.PR↗

arXiv2023

Landscape Complexity for the Empirical Risk of Generalized Linear Models

We present a method to obtain the average and the typical value of the number of critical points of the empirical risk landscape for generalized linear estimation problems and variants. This represents a substantial extension of previous applications of the Kac-Rice method since it allows to analyze the critical points of high dimensional non-Gaussian random functions. Under a technical hypothesis, we obtain a rigorous explicit variational formula for the annealed complexity, which is the logarithm of the average number of critical points at fixed value of the empirical risk. This result is simplified, and extended, using the non-rigorous Kac-Rice replicated method from theoretical physics. In this way we find an explicit variational formula for the quenched complexity, which is generally different from its annealed counterpart, and allows to obtain the number of critical points for typical instances up to exponential accuracy.

stat.ML↗

arXiv2022

Exponential growth of random determinants beyond invariance

We give simple criteria to identify the exponential order of magnitude of the absolute value of the determinant for wide classes of random matrix models, not requiring the assumption of invariance. These include Gaussian matrices with covariance profiles, Wigner matrices and covariance matrices with subexponential tails, Erdős-Rényi and $d$-regular graphs for any polynomial sparsity parameter, and non-mean-field random matrix models, such as random band matrices for any polynomial bandwidth. The proof builds on recent tools, including the theory of the Matrix Dyson Equation as developed in [Ajanki, Erdős, Krüger 2019]. We use these asymptotics as an important input to identify the complexity of classes of Gaussian random landscapes in our companion papers [Ben Arous, Bourgade, McKenna 2021; McKenna 2021].

math.PR↗

arXiv2021

Long Random Matrices and Tensor Unfolding

In this paper, we consider the singular values and singular vectors of low rank perturbations of large rectangular random matrices, in the regime the matrix is "long": we allow the number of rows (columns) to grow polynomially in the number of columns (rows). We prove there exists a critical signal-to-noise ratio (depending on the dimensions of the matrix), and the extreme singular values and singular vectors exhibit a BBP type phase transition. As a main application, we investigate the tensor unfolding algorithm for the asymmetric rank-one spiked tensor model, and obtain an exact threshold, which is independent of the procedure of tensor unfolding. If the signal-to-noise ratio is above the threshold, tensor unfolding detects the signals; otherwise, it fails to capture the signals.

math.PR↗

arXiv2021

Backbone scaling limits for random walks on random critical trees

We prove the existence of scaling limits for the projection on the backbone of the random walks on the Incipient Infinite Cluster and the Invasion Percolation Cluster on a regular tree. We treat these projected random walks as randomly trapped random walks (as defined in [BCČR15]) and thus describe these scaling limits as spatially subordinated Brownian motions

math.PR↗

arXiv2021

Counting equilibria of large complex systems by instability index

We consider a nonlinear autonomous system of $N\gg 1$ degrees of freedom randomly coupled by both relaxational ('gradient') and non-relaxational ('solenoidal') random interactions. We show that with increased interaction strength such systems generically undergo an abrupt transition from a trivial phase portrait with a single stable equilibrium into a topologically non-trivial regime of 'absolute instability' where equilibria are on average exponentially abundant, but typically all of them are unstable, unless the dynamics is purely gradient. When interactions increase even further the stable equilibria eventually become on average exponentially abundant unless the interaction is purely solenoidal. We further calculate the mean proportion of equilibria which have a fixed fraction of unstable directions.

math-ph↗

arXiv2021

Landscape complexity beyond invariance and the elastic manifold

This paper characterizes the annealed, topological complexity (both of total critical points and of local minima) of the elastic manifold. This classical model of a disordered elastic system captures point configurations with self-interactions in a random medium. We establish the simple-vs.-glassy phase diagram in the model parameters, with these phases separated by a physical boundary known as the Larkin mass, confirming formulas of Fyodorov and Le Doussal. One essential, dynamical, step of the proof also applies to a general signal-to-noise model of soft spins in an anisotropic well, for which we prove a negative-second-moment threshold distinguishing positive from zero complexity. A universal near-critical behavior appears within this phase portrait, namely quadratic near-critical vanishing of the complexity of total critical points, and cubic near-critical vanishing of the complexity of local minima. These two models serve as a paradigm of complexity calculations for Gaussian landscapes exhibiting few distributional symmetries, i.e. beyond the invariant setting. The two main inputs for the proof are determinant asymptotics for non-invariant random matrices from our companion paper [Ben Arous, Bourgade, McKenna 2021], and the atypical convexity and integrability of the limiting variational problems.

math.PR↗

arXiv2021

Shattering Versus Metastability in Spin Glasses

Our goal in this work is to better understand the relationship between replica symmetry breaking, shattering, and metastability. To this end, we study the static and dynamic behaviour of spherical pure $p$-spin glasses above the replica symmetry breaking temperature $T_{s}$. In this regime, we find that there are at least two distinct temperatures related to non-trivial behaviour. First we prove that there is a regime of temperatures in which the spherical $p$-spin model exhibits a shattering phase. Our results holds in a regime above but near $T_s$. We then find that metastable states exist up to an even higher temperature $T_{BBM}$ as predicted by Barrat--Burioni--Mézard which is expected to be higher than the phase boundary for the shattering phase $T_d <T_{BBM}$. We develop this work by first developing a Thouless--Anderson--Palmer decomposition which builds on the work of Subag. We then present a series of questions and conjectures regarding the sharp phase boundaries for shattering and slow mixing.

math.PR↗

arXiv2020

Free Energy Wells and Overlap Gap Property in Sparse PCA

We study a variant of the sparse PCA (principal component analysis) problem in the "hard" regime, where the inference task is possible yet no polynomial-time algorithm is known to exist. Prior work, based on the low-degree likelihood ratio, has conjectured a precise expression for the best possible (sub-exponential) runtime throughout the hard regime. Following instead a statistical physics inspired point of view, we show bounds on the depth of free energy wells for various Gibbs measures naturally associated to the problem. These free energy wells imply hitting time lower bounds that corroborate the low-degree conjecture: we show that a class of natural MCMC (Markov chain Monte Carlo) methods (with worst-case initialization) cannot solve sparse PCA with less than the conjectured runtime. These lower bounds apply to a wide range of values for two tuning parameters: temperature and sparsity misparametrization. Finally, we prove that the Overlap Gap Property (OGP), a structural property that implies failure of certain local search algorithms, holds in a significant part of the hard regime.

math.PR↗

arXiv2019

Very rare events for diffusion processes in short time

We study the large deviation estimates for the short time asymptotic behavior of a strongly degenerate diffusion process. Assuming a nilpotent structure of the Lie algebra generated by the driving vector fields, we obtain a graded large deviation principle and prove the existence of those "very rare events". In particular the first grade coincides with the classical Large Deviation Principle.

math.PR↗

arXiv2018

Geometry and temperature chaos in mixed spherical spin glasses at low temperature - the perturbative regime

We study the Gibbs measure of mixed spherical $p$-spin glass models at low temperature, in (part of) the 1-RSB regime, including, in particular, models close to pure in an appropriate sense. We show that the Gibbs measure concentrates on spherical bands around deep critical points of the (extended) Hamiltonian restricted to the sphere of radius $\sqrt N q_\star$, where $q_\star^2$ is the rightmost point in the support of the overlap distribution. We also show that the relevant critical points are pairwise orthogonal for two different low temperatures. This allows us to explain why temperature chaos occurs for those models, in contrast to the pure spherical models.

math.PR↗

arXiv2018

Spectral gap estimates in mean field spin glasses

We show that mixing for local, reversible dynamics of mean field spin glasses is exponentially slow in the low temperature regime. We introduce a notion of free energy barriers for the overlap, and prove that their existence imply that the spectral gap is exponentially small, and thus that mixing is exponentially slow. We then exhibit sufficient conditions on the equilibrium Gibbs measure which guarantee the existence of these barriers, using the notion of replicon eigenvalue and 2D Guerra Talagrand bounds. We show how these sufficient conditions cover large classes of Ising spin models for reversible nearest-neighbor dynamics and spherical models for Langevin dynamics. Finally, in the case of Ising spins, Panchenko's recent rigorous calculation [79] of the free energy for a system of "two real replica" enables us to prove a quenched LDP for the overlap distribution, which gives us a wider criterion for slow mixing directly related to the Franz-Parisi-Virasoro approach [43,60]. This condition holds in a wider range of temperatures.

math.PR↗

arXiv2017

Stable limit laws and structure of the scaling function for reaction-diffusion in random environment

We prove the emergence of stable fluctuations for reaction-diffusion in random environment with Weibull tails. This completes our work around the quenched to annealed transition phenomenon in this context of reaction diffusion. In [9], we had already considered the model treated here and had studied fully the regimes where the law of large numbers is satisfied and where the fluctuations are Gaussian, but we had left open the regime of stable fluctuations. Our work is based on a spectral approach centered on the classical theory of rank-one perturbations. It illustrates the gradual emergence of the role of the higher peaks of the environments. This approach also allows us to give the delicate exact asymptotics of the normalizing constants needed in the stable limit law.

math.PR↗

arXiv2016

Scaling limit for the ant in high-dimensional labyrinths

We study here a detailed conjecture regarding one of the most important cases of anomalous diffusion, i.e the behavior of the "ant in the labyrinth". It is natural to conjecture (see [16] and [8]) that the scaling limit for random walks on large critical random graphs exists in high dimensions, and is universal. This scaling limit is simply the natural Brownian Motion on the Integrated Super-Brownian Excursion. We give here a set of four natural sufficient conditions on the critical graphs and prove that this set of assumptions ensures the validity of this conjecture. The remaining future task is to prove that these sufficient conditions hold for the various classical cases of critical random structures, like the usual Bernoulli bond percolation, oriented percolation, spread-out percolation in high enough dimension. In the companion paper [10], we do precisely that in a first case, the random walk on the trace of a large critical branching random walk. We verify the validity of these sufficient conditions and thus obtain the scaling limit mentioned above, in dimensions larger than 14.

math.PR↗