arXiv subjects

Antoine Maillard

Publications and source records attributed to Antoine Maillard.

30 records · Page 2Linked to original sources

arXiv2022

Phase Retrieval: From Computational Imaging to Machine Learning

Phase retrieval consists in the recovery of a complex-valued signal from intensity-only measurements. As it pervades a broad variety of applications, many researchers have striven to develop phase-retrieval algorithms. Classical approaches involve techniques as varied as generic gradient-descent routines or specialized spectral methods, to name a few. Yet, the phase-recovery problem remains a challenge to this day. Recently, however, advances in machine learning have revitalized the study of phase retrieval in two ways: significant theoretical advances have emerged from the analogy between phase retrieval and single-layer neural networks; practical breakthroughs have been obtained thanks to deep-learning regularization. In this tutorial, we review phase retrieval under a unifying framework that encompasses classical and machine-learning methods. We focus on three key elements: applications, overview of recent reconstruction algorithms, and the latest theoretical results.

physics.optics↗

arXiv2022

A remark on Kashin's discrepancy argument and partial coloring in the Komlós conjecture

In this expository note, we discuss an early partial coloring result of B. Kashin [C. R. Acad. Bulgare Sci., 1985]. Although this result only implies Spencer's six standard deviations [Trans. Amer. Math. Soc., 1985] up to a $\log\log n$ factor, Kashin's argument gives a simple proof of the existence of a constant discrepancy partial coloring in the setup of Komlós conjecture.

math.CO↗

arXiv2022

Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising

Factorization of matrices where the rank of the two factors diverges linearly with their sizes has many applications in diverse areas such as unsupervised representation learning, dictionary learning or sparse coding. We consider a setting where the two factors are generated from known component-wise independent prior distributions, and the statistician observes a (possibly noisy) component-wise function of their matrix product. In the limit where the dimensions of the matrices tend to infinity, but their ratios remain fixed, we expect to be able to derive closed form expressions for the optimal mean squared error on the estimation of the two factors. However, this remains a very involved mathematical and algorithmic problem. A related, but simpler, problem is extensive-rank matrix denoising, where one aims to reconstruct a matrix with extensive but usually small rank from noisy measurements. In this paper, we approach both these problems using high-temperature expansions at fixed order parameters. This allows to clarify how previous attempts at solving these problems failed at finding an asymptotically exact solution. We provide a systematic way to derive the corrections to these existing approximations, taking into account the structure of correlations particular to the problem. Finally, we illustrate our approach in detail on the case of extensive-rank matrix denoising. We compare our results with known optimal rotationally-invariant estimators, and show how exact asymptotic calculations of the minimal error can be performed using extensive-rank matrix integrals.

cond-mat.dis-nn↗

arXiv2022

Optimal denoising of rotationally invariant rectangular matrices

In this manuscript we consider denoising of large rectangular matrices: given a noisy observation of a signal matrix, what is the best way of recovering the signal matrix itself? For Gaussian noise and rotationally-invariant signal priors, we completely characterize the optimal denoiser and its performance in the high-dimensional limit, in which the size of the signal matrix goes to infinity with fixed aspects ratio, and under the Bayes optimal setting, that is when the statistician knows how the signal and the observations were generated. Our results generalise previous works that considered only symmetric matrices to the more general case of non-symmetric and rectangular ones. We explore analytically and numerically a particular choice of factorized signal prior that models cross-covariance matrices and the matrix factorization problem. As a byproduct of our analysis, we provide an explicit asymptotic evaluation of the rectangular Harish-Chandra-Itzykson-Zuber integral in a special case.

cond-mat.dis-nn↗

arXiv2021

Construction of optimal spectral methods in phase retrieval

We consider the phase retrieval problem, in which the observer wishes to recover a $n$-dimensional real or complex signal $\mathbf{X}^\star$ from the (possibly noisy) observation of $|\mathbfΦ \mathbf{X}^\star|$, in which $\mathbfΦ$ is a matrix of size $m \times n$. We consider a \emph{high-dimensional} setting where $n,m \to \infty$ with $m/n = \mathcal{O}(1)$, and a large class of (possibly correlated) random matrices $\mathbfΦ$ and observation channels. Spectral methods are a powerful tool to obtain approximate observations of the signal $\mathbf{X}^\star$ which can be then used as initialization for a subsequent algorithm, at a low computational cost. In this paper, we extend and unify previous results and approaches on spectral methods for the phase retrieval problem. More precisely, we combine the linearization of message-passing algorithms and the analysis of the \emph{Bethe Hessian}, a classical tool of statistical physics. Using this toolbox, we show how to derive optimal spectral methods for arbitrary channel noise and right-unitarily invariant matrix $\mathbfΦ$, in an automated manner (i.e. with no optimization over any hyperparameter or preprocessing function).

cs.IT↗

arXiv2021

Large deviations of extreme eigenvalues of generalized sample covariance matrices

We present an analytical technique to compute the probability of rare events in which the largest eigenvalue of a random matrix is atypically large (i.e.\ the right tail of its large deviations). The results also transfer to the left tail of the large deviations of the smallest eigenvalue. The technique improves upon past methods by not requiring the explicit law of the eigenvalues, and we apply it to a large class of random matrices that were previously out of reach. In particular, we solve an open problem related to the performance of principal components analysis on highly correlated data, and open the way towards analyzing the high-dimensional landscapes of complex inference models. We probe our results using an importance sampling approach, effectively simulating events with probability as small as $10^{-100}$.

cond-mat.stat-mech↗

arXiv2020

Phase retrieval in high dimensions: Statistical and computational phase transitions

We consider the phase retrieval problem of reconstructing a $n$-dimensional real or complex signal $\mathbf{X}^{\star}$ from $m$ (possibly noisy) observations $Y_μ= | \sum_{i=1}^n Φ_{μi} X^{\star}_i/\sqrt{n}|$, for a large class of correlated real and complex random sensing matrices $\mathbfΦ$, in a high-dimensional setting where $m,n\to\infty$ while $α= m/n=Θ(1)$. First, we derive sharp asymptotics for the lowest possible estimation error achievable statistically and we unveil the existence of sharp phase transitions for the weak- and full-recovery thresholds as a function of the singular values of the matrix $\mathbfΦ$. This is achieved by providing a rigorous proof of a result first obtained by the replica method from statistical mechanics. In particular, the information-theoretic transition to perfect recovery for full-rank matrices appears at $α=1$ (real case) and $α=2$ (complex case). Secondly, we analyze the performance of the best-known polynomial time algorithm for this problem -- approximate message-passing -- establishing the existence of a statistical-to-algorithmic gap depending, again, on the spectral properties of $\mathbfΦ$. Our work provides an extensive classification of the statistical and algorithmic thresholds in high-dimensional phase retrieval for a broad class of random matrices.

math.ST↗

arXiv2020

High-temperature Expansions and Message Passing Algorithms

Improved mean-field technics are a central theme of statistical physics methods applied to inference and learning. We revisit here some of these methods using high-temperature expansions for disordered systems initiated by Plefka, Georges and Yedidia. We derive the Gibbs free entropy and the subsequent self-consistent equations for a generic class of statistical models with correlated matrices and show in particular that many classical approximation schemes, such as adaptive TAP, Expectation-Consistency, or the approximations behind the Vector Approximate Message Passing algorithm all rely on the same assumptions, that are also at the heart of high-temperature expansions. We focus on the case of rotationally invariant random coupling matrices in the `high-dimensional' limit in which the number of samples and the dimension are both large, but with a fixed ratio. This encapsulates many widely studied models, such as Restricted Boltzmann Machines or Generalized Linear Models with correlated data matrices. In this general setting, we show that all the approximation schemes described before are equivalent, and we conjecture that they are exact in the thermodynamic limit in the replica symmetric phases. We achieve this conclusion by resummation of the infinite perturbation series, which generalizes a seminal result of Parisi and Potters. A rigorous derivation of this conjecture is an interesting mathematical challenge. On the way to these conclusions, we uncover several diagrammatical results in connection with free probability and random matrix theory, that are interesting independently of the rest of our work.

cond-mat.dis-nn↗

arXiv2019

A Quantum Monte Carlo algorithm for out-of-equilibrium Green's functions at long times

We present a quantum Monte-Carlo algorithm for computing the perturbative expansion in power of the coupling constant $U$ of the out-of-equilibrium Green's functions of interacting Hamiltonians of fermions. The algorithm extends the one presented in Phys. Rev. B 91 245154 (2015), and inherits its main property: it can reach the infinite time (steady state) limit since the computational cost to compute order $U^n$ is uniform versus time; the computing time increases as $2^n$. The algorithm is based on the Schwinger-Keldysh formalism and can be used for both equilibrium and out-of-equilibrium calculations. It is stable at both small and long real times including in the stationary regime, because of its automatic cancellation of the disconnected Feynman diagrams. We apply this technique to the Anderson quantum impurity model in the quantum dot geometry to obtain the Green's function and self-energy expansion up to order $U^{10}$ at very low temperature. We benchmark our results at weak and intermediate coupling with high precision Numerical Renormalization Group (NRG) computations as well as analytical results.

cond-mat.str-el↗

arXiv2019

The spiked matrix model with generative priors

Using a low-dimensional parametrization of signals is a generic and powerful way to enhance performance in signal processing and statistical inference. A very popular and widely explored type of dimensionality reduction is sparsity; another type is generative modelling of signal distributions. Generative models based on neural networks, such as GANs or variational auto-encoders, are particularly performant and are gaining on applicability. In this paper we study spiked matrix models, where a low-rank matrix is observed through a noisy channel. This problem with sparse structure of the spikes has attracted broad attention in the past literature. Here, we replace the sparsity assumption by generative modelling, and investigate the consequences on statistical and algorithmic properties. We analyze the Bayes-optimal performance under specific generative models for the spike. In contrast with the sparsity assumption, we do not observe regions of parameters where statistical performance is superior to the best known algorithmic performance. We show that in the analyzed cases the approximate message passing algorithm is able to reach optimal performance. We also design enhanced spectral algorithms and analyze their performance and thresholds using random matrix theory, showing their superiority to the classical principal component analysis. We complement our theoretical results by illustrating the performance of the spectral algorithms when the spikes come from real datasets.

math.ST↗

arXiv2018

The Mutual Information in Random Linear Estimation Beyond i.i.d. Matrices

There has been definite progress recently in proving the variational single-letter formula given by the heuristic replica method for various estimation problems. In particular, the replica formula for the mutual information in the case of noisy linear estimation with random i.i.d. matrices, a problem with applications ranging from compressed sensing to statistics, has been proven rigorously. In this contribution we go beyond the restrictive i.i.d. matrix assumption and discuss the formula proposed by Takeda, Uda, Kabashima and later by Tulino, Verdu, Caire and Shamai who used the replica method. Using the recently introduced adaptive interpolation method and random matrix theory, we prove this formula for a relevant large sub-class of rotationally invariant matrices.

cs.IT↗

arXiv2015

Islands of stability and recurrence times in AdS

We study the stability of anti-de Sitter (AdS) spacetime to spherically symmetric perturbations of a real scalar field in general relativity. Further, we work within the context of the "two time framework" (TTF) approximation, which describes the leading nonlinear effects for small amplitude perturbations, and is therefore suitable for studying the weakly turbulent instability of AdS---including both collapsing and non-collapsing solutions. We have previously identified a class of quasi-periodic (QP) solutions to the TTF equations, and in this work we analyze their stability. We show that there exist several families of QP solutions that are stable to linear order, and we argue that these solutions represent islands of stability in TTF. We extract the eigenmodes of small oscillations about QP solutions, and we use them to predict approximate recurrence times for generic non-collapsing initial data in the full (non-TTF) system. Alternatively, when sufficient energy is driven to high-frequency modes, as occurs for initial data far from a QP solution, the TTF description breaks down as an approximation to the full system. Depending on the higher order dynamics of the full system, this often signals an imminent collapse to a black hole.

gr-qc↗