arXiv subjects

Walid Hachem

Publications and source records attributed to Walid Hachem.

At least 19 recordsLinked to original sources

arXiv2026

The SK model with a sparse variance profile: free energy and AMP algorithm for TAP equations at high temperature

A generalization of the Sherrington-Kirkpatrick (SK) model for spin glasses is considered, in which the interaction matrix is endowed with a variance profile that has no particular structure an may be sparse. In the first part of this paper, an asymptotic equivalent of the free energy is derived at sufficiently high temperatures, regardless of the signature of the variance profile matrix. In the second part, the mean of the spin vector under the Gibbs measure is estimated using an Approximate Message Passing algorithm based on the Thouless-Anderson-Palmer equations. The dynamical approach of Adhikari et.al. (J. Stat. Phys., 2021), originally developed for the classical SK model, is adapted to the present setting to obtain these results.

math-ph↗

arXiv2026

Extreme eigenvalues and eigenvectors for finite rank additive deformations of non-hermitian sparse random matrices

Consider a $n\times n$ sparse non-Hermitian random matrix $X_n$ defined as the Hadamard product between a random matrix with centered independent and identically distributed entries and a sparse Bernoulli matrix with success probability $K_n/n$ where $K_n\le n$ (and possibly $K_n\ll n$) and $K_n\to \infty$ as $n\to \infty$. Let $E_n$ be a deterministic $n\times n$ finite-rank matrix. We prove that the outlier eigenvalues of $Y_n= X_n +E_n$ asymptotically match those of $E_n$. In the special case of a rank-one deformation, assuming further that the sparsity parameter satisfies $K_n \gg \log^9(n)$ and that the entries of the random matrix are sub-Gaussian, we describe the limiting behavior of the projection of the right eigenvector associated with the leading eigenvalue onto the right eigenvector of the rank-one deformation. In particular, we prove that the projection behaves as in the Hermitian case. To that end, we rely on the recent universality results of Brailovskaya and van Handel (2024) relating the singular value spectra of deformations of $X_n$ to Gaussian analogues of these matrices. Our analysis builds upon a recent framework introduced by Bordenave et.al. (2022), and amounts to showing the asymptotic equivalence between the reverse characteristic polynomial of the random matrix and a random analytic function on the unit disc with explicit dependence on the finite-rank deformation.

math.PR↗

arXiv2025

Spin glass analysis of the invariant distribution of a Lotka-Volterra SDE with a large random interaction matrix

The generalized Lotka-Volterra stochastic differential equation with a symmetric food interaction matrix is frequently used to model the dynamics of the abundances of the species living within an ecosystem when these interactions are mutualistic or competitive. In the relevant cases of interest, the Markov process described by this equation has an unique invariant distribution which has a Hamiltonian structure. Following an important trend in theoretical ecology, the interaction matrix is considered in this paper as a large random matrix. In this situation, the (conditional) invariant distribution takes the form of a random Gibbs measure that can be studied rigorously with the help of spin glass techniques issued from the field of physics of disordered systems. Considering that the interaction matrix is an additively deformed GOE matrix, which is a well-known model for this matrix in theoretical ecology, the free energy of the model is derived in the limit of the large number $n$ of species, making rigorous some recent results from the literature. The free energy analysis made in this paper could be adapted to other situations where the Gibbs measure is non compactly supported.

math.PR↗

arXiv2025

Approximate Message Passing for general non-Symmetric random matrices

Approximate Message Passing (AMP) algorithms are a family of iterative algorithms based on large random matrices with the special property of tracking the statistical properties of their iterates. They are used in various fields such as Statistical Physics, Machine learning, Communication systems, Theoretical ecology, etc. In this article we consider AMP algorithms based on non-Symmetric random matrices with a general variance profile, possibly sparse, a general covariance profile, and non-Gaussian entries. We hence substantially extend the results on Elliptic random matrices that we developed in [Gueddari et al., 2024]. From a technical point of view, we enhance the combinatorial techniques developed in [Bayati et al., 2015] and in [Hachem, 2024]. Our main motivation is the understanding of equilibria of large food-webs described by Lotka-Volterra systems of ODE, in the continuation of the works of [Hachem, 2024], [Akjouj et al., 2024] and [Gueddari et al., 2024], but the versatility of the model studied might be of interest beyond these particular applications.

math.PR↗

arXiv2025

On the spectral radius and the characteristic polynomial of a random matrix with independent elements and a variance profile

In this paper, it is shown that with large probability, the spectral radius of a large non-Hermitian random matrix with a general variance profile does not exceed the square root of the spectral radius of the variance profile matrix. A minimal moment assumption is considered and sparse variance profiles are covered. Following an approach developed recently by Bordenave, Chafa{\"i} and Garc{\'i}a-Zelada, the key theorem states the asymptotic equivalence between the reverse characteristic polynomial of the random matrix at hand and a random analytic function which depends on the variance profile matrix. The result is applied to the case of a non-Hermitian random matrix with a variance profile given by a piecewise constant or a continuous non-negative function, the inhomogeneous (centered) directed Erd\H{o}s-R{\'e}nyi model, and more.

math.PR↗

arXiv2024

Long run convergence of discrete-time interacting particle systems of the McKean-Vlasov type

We consider a discrete-time system of n coupled random vectors, a.k.a. interacting particles. The dynamics involve a vanishing step size, some random centered perturbations, and a mean vector field which induces the coupling between the particles. We study the doubly asymptotic regime where both the number of iterations and the number n of particles tend to infinity, without any constraint on the relative rates of convergence of these two parameters. We establish that the empirical measure of the interpolated trajectories of the particles converges in probability, in an ergodic sense, to the set of recurrent Mc-Kean-Vlasov distributions. A first application example is the granular media equation, where the particles are shown to converge to a critical point of the Helmholtz energy. A second example is the convergence of stochastic gradient descent to the global minimizer of the risk, in a wide two-layer neural networks using random features.

math.PR↗

arXiv2024

Elliptic Approximate Message Passing and an application to theoretical ecology

Approximate Message Passing (AMP) algorithmshave recently gathered significant attention across disciplines such as statistical physics, machine learning, and communication systems. This study aims to extend AMP algorithms to non-symmetric (elliptic) matrices, motivated by analyzing equilibrium properties in ecological systems featuring elliptic interaction matrices.In this article, we provide the general form of an AMP algorithm associated to a random elliptic matrix, the main change lying in a modification of the corrective (Onsager) term. In order to establish the statistical properties of this algorithm, we use and prove a generalized form of Bolthausen conditioning argument, pivotal to proceed by a Gaussian-based induction.We finally address the initial motivating question from theoretical ecology. Large foodwebs are often described by Lotka-Volterra systems of coupled differential equations, where the interaction matrix is elliptic random. In this context, we design an AMP algorithm to analyze the statistical properties of the equilibrium point in a high-dimensional regime. We rigorously recover the results established by [Bunin, 2017] and [Galla,2018] who used techniques from theoretical physics, and extend them with the help of propagation of chaos type arguments.

math.PR↗

arXiv2023

Approximate Message Passing for sparse matrices with application to the equilibria of large ecological Lotka-Volterra systems

This paper is divided into two parts. The first part is devoted to the study of a class of Approximate Message Passing (AMP) algorithms which are widely used in the fields of statistical physics, machine learning, or communication theory. The AMP algorithms studied in this part are those where the measurement matrix has independent elements, up to the symmetry constraint when this matrix is symmetric, with a variance profile that can be sparse. The AMP problem is solved by adapting the approach of Bayati, Lelarge, and Montanari (2015) to this matrix model. \\ The Lotka-Volterra (LV) model is the standard model for studying the dynamical behavior of large dimensional ecological food chains. The second part of this paper is focused on the study of the statistical distribution of the globally stable equilibrium vector of a LV system in the situation where the random symmetric interaction matrix among the living species is sparse, and in the regime of large dimensions. This equilibrium vector is the solution of a Linear Complementarity Problem, which distribution is shown to be characterized through the AMP approach developed in the first part. In the large dimensional regime, this distribution is close to a mixture of a large number of truncated Gaussians.

math.PR↗

arXiv2023

Equilibria of large random Lotka-Volterra systems with vanishing species: a mathematical approach

Ecosystems with a large number of species are often modelled as Lotka-Volterra dynamical systems built around a large random interaction matrix. Under some known conditions, a global equilibrium exists and is unique. In this article, we rigorously study its statistical properties in the large dimensional regime. Such an equilibrium vector is known to be the solution of a so-called Linear Complementarity Problem (LCP). We describe its statistical properties by designing an Approximate Message Passing (AMP) algorithm, a technique that has recently aroused an intense research effort in the fields of statistical physics, Machine Learning, or communication theory. Interaction matrices taken from the Gaussian Orthogonal Ensemble, or following a Wishart distribution are considered. Beyond these models, the AMP approach developed in this article has the potential to describe the statistical properties of equilibria associated to more involved interaction matrix models.

q-bio.PE↗

arXiv2022

Complex systems in Ecology: a guided tour with large Lotka-Volterra models and random matrices

Ecosystems represent archetypal complex dynamical systems, often modelled by coupled differential equations of the form $$ \frac{d x_i}{d t} = x_i \varphi_i(x_1,\cdots, x_N)\ , $$ where $N$ represents the number of species and $x_i$, the abundance of species $i$. Among these families of coupled diffential equations, Lotka-Volterra (LV) equations $$ \frac{d x_i}{d t} = x_i ( r_i - x_i +(\Gamma \mathbf{x})_i)\ , $$ play a privileged role, as the LV model represents an acceptable trade-off between complexity and tractability. Here, $r_i$ represents the intrinsic growth of species $i$ and $\Gamma$ stands for the interaction matrix: $\Gamma_{ij}$ represents the effect of species $j$ over species $i$. For large $N$, estimating matrix $\Gamma$ is often an overwhelming task and an alternative is to draw $\Gamma$ at random, parametrizing its statistical distribution by a limited number of model features. Dealing with large random matrices, we naturally rely on Random Matrix Theory (RMT). The aim of this review article is to present an overview of the work at the junction of theoretical ecology and large random matrix theory. It is intended to an interdisciplinary audience spanning theoretical ecology, complex systems, statistical physics and mathematical biology.

q-bio.PE↗

arXiv2022

Spectral measure of empirical autocovariance matrices of high dimensional Gaussian stationary processes

Consider the empirical autocovariance matrix at a given non-zero time lag based on observations from a multivariate complex Gaussian stationary time series. The spectral analysis of these autocovariance matrices can be useful in certain statistical problems, such as those related to testing for white noise. We study the behavior of their spectral measures in the asymptotic regime where the time series dimension and the observation window length both grow to infinity, and at the same rate. Following a general framework in the field of the spectral analysis of large random non-Hermitian matrices, at first the probabilistic behavior of the small singular values of the shifted versions of the autocovariance matrix are obtained. This is then used to infer about the large sample behaviour of the empirical spectral measure of the autocovariance matrices at any lag. Matrix orthogonal polynomials on the unit circle play a crucial role in our study.

math.ST↗

arXiv2022

Convergence of constant step stochastic gradient descent for non-smooth non-convex functions

This paper studies the asymptotic behavior of the constant step Stochastic Gradient Descent for the minimization of an unknown function F , defined as the expectation of a non convex, non smooth, locally Lipschitz random function. As the gradient may not exist, it is replaced by a certain operator: a reasonable choice is to use an element of the Clarke subdifferential of the random function; an other choice is the output of the celebrated backpropagation algorithm, which is popular amongst practionners, and whose properties have recently been studied by Bolte and Pauwels [7]. Since the expectation of the chosen operator is not in general an element of the Clarke subdifferential BF of the mean function, it has been assumed in the literature that an oracle of BF is available. As a first result, it is shown in this paper that such an oracle is not needed for almost all initialization points of the algorithm. Next, in the small step size regime, it is shown that the interpolated trajectory of the algorithm converges in probability (in the compact convergence sense) towards the set of solutions of the differential inclusion. Finally, viewing the iterates as a Markov chain whose transition kernel is indexed by the step size, it is shown that the invariant distribution of the kernel converge weakly to the set of invariant distribution of this differential inclusion as the step size tends to zero. These results show that when the step size is small, with large probability, the iterates eventually lie in a neighborhood of the critical points of the mean function F .

math.NA↗

arXiv2021

Stochastic Subgradient Descent Escapes Active Strict Saddles on Weakly Convex Functions

In non-smooth stochastic optimization, we establish the non-convergence of the stochastic subgradient descent (SGD) to the critical points recently called active strict saddles by Davis and Drusvyatskiy. Such points lie on a manifold $M$ where the function $f$ has a direction of second-order negative curvature. Off this manifold, the norm of the Clarke subdifferential of $f$ is lower-bounded. We require two conditions on $f$. The first assumption is a Verdier stratification condition, which is a refinement of the popular Whitney stratification. It allows us to establish a reinforced version of the projection formula of Bolte \emph{et.al.} for Whitney stratifiable functions, and which is of independent interest. The second assumption, termed the angle condition, allows to control the distance of the iterates to $M$. When $f$ is weakly convex, our assumptions are generic. Consequently, generically in the class of definable weakly convex functions, the SGD converges to a local minimizer.

math.OC↗

arXiv2020

Non-Hermitian random matrices with a variance profile (I): Deterministic equivalents and limiting ESDs

For each $n$, let $A_n=(σ_{ij})$ be an $n\times n$ deterministic matrix and let $X_n=(X_{ij})$ be an $n\times n$ random matrix with i.i.d. centered entries of unit variance. We study the asymptotic behavior of the empirical spectral distribution $μ_n^Y$ of the rescaled entry-wise product \[ Y_n = \left(\frac1{\sqrt{n}} σ_{ij}X_{ij}\right). \] For our main result we provide a deterministic sequence of probability measures $μ_n$, each described by a family of Master Equations, such that the difference $μ^Y_n - μ_n$ converges weakly in probability to the zero measure. A key feature of our results is to allow some of the entries $σ_{ij}$ to vanish, provided that the standard deviation profiles $A_n$ satisfy a certain quantitative irreducibility property. An important step is to obtain quantitative bounds on the solutions to an associate system of Schwinger--Dyson equations, which we accomplish in the general sparse setting using a novel graphical bootstrap argument.

math.PR↗

arXiv2020

Non-Hermitian random matrices with a variance profile (II): properties and examples

For each $n$, let $A_n=(σ_{ij})$ be an $n\times n$ deterministic matrix and let $X_n=(X_{ij})$ be an $n\times n$ random matrix with i.i.d. centered entries of unit variance. In the companion article Cook et al., we considered the empirical spectral distribution $μ_n^Y$ of the rescaled entry-wise product \[ Y_n = \frac 1{\sqrt{n}} A_n\odot X_n = \left(\frac1{\sqrt{n}} σ_{ij}X_{ij}\right) \] and provided a deterministic sequence of probability measures $μ_n$ such that the difference $μ^Y_n - μ_n$ converges weakly in probability to the zero measure. A key feature in Cook et al. was to allow some of the entries $σ_{ij}$ to vanish, provided that the standard deviation profiles $A_n$ satisfy a certain quantitative irreducibility property. In the present article, we provide more information on the sequence $(μ_n)$, described by a family of Master Equations. We consider these equations in important special cases such as separable variance profiles $σ^2_{ij}=d_i \widetilde d_j$ and sampled variance profiles $σ^2_{ij} = σ^2\left(\frac in, \frac jn \right)$ where $(x,y)\mapsto σ^2(x,y)$ is a given function on $[0,1]^2$. Associate examples are provided where $μ_n^Y$ converges to a genuine limit. We study $μ_n$'s behavior at zero and provide examples where $μ_n$'s density is bounded, blows up, or vanishes while an atom appears. As a consequence, we identify the profiles that yield the circular law. Finally, building upon recent results from Alt et al., we prove that except maybe in zero, $μ_n$ admits a positive density on the centered disc of radius $\sqrt{ρ(V_n)}$, where $V_n=(\frac 1n σ_{ij}^2)$ and $ρ(V_n)$ is its spectral radius.

math.PR↗

arXiv2020

A Fully Stochastic Primal-Dual Algorithm

A new stochastic primal--dual algorithm for solving a composite optimization problem is proposed. It is assumed that all the functions/operators that enter the optimization problem are given as statistical expectations. These expectations are unknown but revealed across time through i.i.d. realizations. The proposed algorithm is proven to converge to a saddle point of the Lagrangian function. In the framework of the monotone operator theory, the convergence proof relies on recent results on the stochastic Forward Backward algorithm involving random monotone operators. An example of convex optimization under stochastic linear constraints is considered.

math.OC↗

arXiv2019

Smallest singular value and limit eigenvalue distribution of a class of non-Hermitian random matrices with statistical application

Suppose $X$ is an $N \times n$ complex matrix whose entries are centered, independent, and identically distributed random variables with variance $1/n$ and whose fourth moment is of order ${\mathcal O}(n^{-2})$. In the first part of the paper, we consider the non-Hermitian matrix $X A X^* - z$, where $A$ is a deterministic matrix whose smallest and largest singular values are bounded below and above respectively, and $z\neq 0$ is a complex number. Asymptotic probability bounds for the smallest singular value of this model are obtained in the large dimensional regime where $N$ and $n$ diverge to infinity at the same rate. In the second part of the paper, we consider the special case where $A = J = [1_{i-j = 1\mod n} ]$ is a circulant matrix. Using the result of the first part, it is shown that the limit eigenvalue distribution of $X J X^*$ exists in the large dimensional regime, and we determine this limit explicitly. A statistical application of this result devoted towards testing the presence of correlations within a multivariate time series is considered. Assuming that $X$ represents a ${\mathbb C}^N$-valued time series which is observed over a time window of length $n$, the matrix $X J X^*$ represents the one-step sample autocovariance matrix of this time series. Guided by the result on the limit spectral measure of this matrix, a whiteness test against an MA correlation model on the time series is introduced. Numerical simulations show the excellent performance of this test.

math.PR↗

arXiv2019

Mutual Information of Wireless Channels and Block-Jacobi Ergodic Operators

Shannon's mutual information of a random multiple antenna and multipath channel is studied in the general case where the channel impulse response is an ergodic and stationary process which is assumed to be available at the receiver. From this viewpoint, the channel is represented by an ergodic self-adjoint block-Jacobi operator, which is close in many aspects to a block version of a random Schrödinger operator. The mutual information is then related to the so-called density of states of this operator. In this paper, it is shown that under the weakest assumptions on the channel, the mutual information can be expressed in terms of a matrix-valued stochastic process coupled with the channel process. This allows numerical approximations of the mutual information in this general setting. Moreover, assuming further that the channel impulse response is a Markov process, a representation for the mutual information offset in the large Signal to Noise Ratio regime is obtained in terms of another related Markov process. This generalizes previous results from Levy~\emph{et.al.}. It is also illustrated how the mutual information expressions that are closely related to those predicted by the random matrix theory can be recovered in the large dimensional regime.

cs.IT↗