arXiv subjects

Romain Couillet

Publications and source records attributed to Romain Couillet.

At least 55 records · Page 3Linked to original sources

arXiv2018

Large-dimensional behavior of regularized Maronna's M-estimators of covariance matrices

Robust estimators of large covariance matrices are considered, comprising regularized (linear shrinkage) modifications of Maronna's classical M-estimators. These estimators provide robustness to outliers, while simultaneously being well-defined when the number of samples does not exceed the number of variables. By applying tools from random matrix theory, we characterize the asymptotic performance of such estimators when the numbers of samples and variables grow large together. In particular, our results show that, when outliers are absent, many estimators of the regularized-Maronna type share the same asymptotic performance, and for these estimators we present a data-driven method for choosing the asymptotically optimal regularization parameter with respect to a quadratic loss. Robustness in the presence of outliers is then studied: in the non-regularized case, a large-dimensional robustness metric is proposed, and explicitly computed for two particular types of estimators, exhibiting interesting differences depending on the underlying contamination model. The impact of outliers in regularized estimators is then studied, with interesting differences with respect to the non-regularized case, leading to new practical insights on the choice of particular estimators.

math.ST↗

arXiv2017

Gallager Bound for MIMO Channels: Large-N Asymptotics

The use of multiple antenna arrays in transmission and reception has become an integral part of modern wireless communications. To quantify the performance of such systems, the evaluation of bounds on the error probability of realistic finite length codewords is important. In this paper, we analyze the standard Gallager error bound for both constraints of maximum average power and maximum instantaneous power. Applying techniques from random matrix theory, we obtain analytic expressions of the error exponent when the length of the codeword increases to infinity at a fixed ratio with the antenna array dimensions. Analyzing its behavior at rates close to the ergodic rate, we find that the Gallager error bound becomes asymptotically close to an upper error bound obtained recently by Hoydis et al. 2015. We also obtain an expression for the Gallager exponent in the case when the codelength spans several Rayleigh fading blocks, hence taking into account the situation when the channel varies during each transmission.

cs.IT↗

arXiv2017

A random matrix analysis and improvement of semi-supervised learning for large dimensional data

This article provides an original understanding of the behavior of a class of graph-oriented semi-supervised learning algorithms in the limit of large and numerous data. It is demonstrated that the intuition at the root of these methods collapses in this limit and that, as a result, most of them become inconsistent. Corrective measures and a new data-driven parametrization scheme are proposed along with a theoretical analysis of the asymptotic performances of the resulting approach. A surprisingly close behavior between theoretical performances on Gaussian mixture models and on real datasets is also illustrated throughout the article, thereby suggesting the importance of the proposed analysis for dealing with practical data. As a result, significant performance gains are observed on practical data classification using the proposed parametrization.

cs.LG↗

arXiv2017

A Random Matrix Approach to Neural Networks

This article studies the Gram random matrix model $G=\frac1TΣ^{\rm T}Σ$, $Σ=σ(WX)$, classically found in the analysis of random feature maps and random neural networks, where $X=[x_1,\ldots,x_T]\in{\mathbb R}^{p\times T}$ is a (data) matrix of bounded norm, $W\in{\mathbb R}^{n\times p}$ is a matrix of independent zero-mean unit variance entries, and $σ:{\mathbb R}\to{\mathbb R}$ is a Lipschitz continuous (activation) function --- $σ(WX)$ being understood entry-wise. By means of a key concentration of measure lemma arising from non-asymptotic random matrix arguments, we prove that, as $n,p,T$ grow large at the same rate, the resolvent $Q=(G+γI_T)^{-1}$, for $γ>0$, has a similar behavior as that met in sample covariance matrix models, involving notably the moment $Φ=\frac{T}n{\mathbb E}[G]$, which provides in passing a deterministic equivalent for the empirical spectral measure of $G$. Application-wise, this result enables the estimation of the asymptotic performance of single-layer random neural networks. This in turn provides practical insights into the underlying mechanisms into play in random neural networks, entailing several unexpected consequences, as well as a fast practical means to tune the network hyperparameters.

math.PR↗

arXiv2017

Large System Analysis of Power Normalization Techniques in Massive MIMO

Linear precoding has been widely studied in the context of Massive multiple-input-multiple-output (MIMO) together with two common power normalization techniques, namely, matrix normalization (MN) and vector normalization (VN). Despite this, their effect on the performance of Massive MIMO systems has not been thoroughly studied yet. The aim of this paper is to fulfill this gap by using large system analysis. Considering a system model that accounts for channel estimation, pilot contamination, arbitrary pathloss, and per-user channel correlation, we compute tight approximations for the signal-to-interference-plus-noise ratio and the rate of each user equipment in the system while employing maximum ratio transmission (MRT), zero forcing (ZF), and regularized ZF precoding under both MN and VN techniques. Such approximations are used to analytically reveal how the choice of power normalization affects the performance of MRT and ZF under uncorrelated fading channels. It turns out that ZF with VN resembles a sum rate maximizer while it provides a notion of fairness under MN. Numerical results are used to validate the accuracy of the asymptotic analysis and to show that in Massive MIMO, non-coherent interference and noise, rather than pilot contamination, are often the major limiting factors of the considered precoding schemes.

cs.IT↗

arXiv2017

Reducing the Computational Complexity of Multicasting in Large-Scale Antenna Systems

In this paper, we study the physical layer multicasting to multiple co-channel groups in large-scale antenna systems. The users within each group are interested in a common message and different groups have distinct messages. In particular, we aim at designing the precoding vectors solving the so-called quality of service (QoS) and weighted max-min fairness (MMF) problems, assuming that the channel state information is available at the base station (BS). To solve both problems, the baseline approach exploits the semidefinite relaxation (SDR) technique. Considering a BS with $N$ antennas, the SDR complexity is more than $\mathcal{O}(N^{6})$, which prevents its application in large-scale antenna systems. To overcome this issue, we present two new classes of algorithms that, not only have significantly lower computational complexity than existing solutions, but also largely outperform the SDR based methods. Moreover, we present a novel duality between transformed versions of the QoS and the weighted MMF problems. The duality explicitly determines the solution to the weighted MMF problem given the solution to the QoS problem, and vice versa. Numerical results are used to validate the effectiveness of the proposed solutions and to make comparisons with existing alternatives under different operating conditions.

cs.IT↗

arXiv2016

Spectral community detection in heterogeneous large networks

In this article, we study spectral methods for community detection based on $ α$-parametrized normalized modularity matrix hereafter called $ {\bf L}_α$ in heterogeneous graph models. We show, in a regime where community detection is not asymptotically trivial, that $ {\bf L}_α$ can be well approximated by a more tractable random matrix which falls in the family of spiked random matrices. The analysis of this equivalent spiked random matrix allows us to improve spectral methods for community detection and assess their performances in the regime under study. In particular, we prove the existence of an optimal value $ α_{\rm opt} $ of the parameter $ α$ for which the detection of communities is best ensured and we provide an on-line estimation of $ α_{\rm opt} $ only based on the knowledge of the graph adjacency matrix. Unlike classical spectral methods for community detection where clustering is performed on the eigenvectors associated with extreme eigenvalues, we show through our theoretical analysis that a regularization should instead be performed on those eigenvectors prior to clustering in heterogeneous graphs. Finally, through a deeper study of the regularized eigenvectors used for clustering, we assess the performances of our new algorithm for community detection. Numerical simulations in the course of the article show that our methods outperform state-of-the-art spectral methods on dense heterogeneous graphs.

stat.ML↗

arXiv2016

Random matrices meet machine learning: a large dimensional analysis of LS-SVM

This article proposes a performance analysis of kernel least squares support vector machines (LS-SVMs) based on a random matrix approach, in the regime where both the dimension of data $p$ and their number $n$ grow large at the same rate. Under a two-class Gaussian mixture model for the input data, we prove that the LS-SVM decision function is asymptotically normal with means and covariances shown to depend explicitly on the derivatives of the kernel function. This provides improved understanding along with new insights into the internal workings of SVM-type methods for large datasets.

stat.ML↗

arXiv2016

Large System Analysis of Base Station Cooperation for Power Minimization

This work focuses on a large-scale multi-cell multi-user MIMO system in which $L$ base stations (BSs) of $N$ antennas each communicate with $K$ single-antenna user equipments. We consider the design of the linear precoder that minimizes the total power consumption while ensuring target user rates. Three configurations with different degrees of cooperation among BSs are considered: the coordinated beamforming scheme (only channel state information is shared among BSs), the coordinated multipoint MIMO processing technology or network MIMO (channel state and data cooperation), and a single cell beamforming scheme (only local channel state information is used for beamforming while channel state cooperation is needed for power allocation). The analysis is conducted assuming that $N$ and $K$ grow large with a non trivial ratio $K/N$ and imperfect channel state information (modeled by the generic Gauss-Markov formulation form) is available at the BSs. Tools of random matrix theory are used to compute, in explicit form, deterministic approximations for: (i) the parameters of the optimal precoder; (ii) the powers needed to ensure target rates; and (iii) the total transmit power. These results are instrumental to get further insight into the structure of the optimal precoders and also to reduce the implementation complexity in large-scale networks. Numerical results are used to validate the asymptotic analysis in the finite system regime and to make comparisons among the different configurations.

cs.IT↗

arXiv2016

Kernel spectral clustering of large dimensional data

This article proposes a first analysis of kernel spectral clustering methods in the regime where the dimension $p$ of the data vectors to be clustered and their number $n$ grow large at the same rate. We demonstrate, under a $k$-class Gaussian mixture model, that the normalized Laplacian matrix associated with the kernel matrix asymptotically behaves similar to a so-called spiked random matrix. Some of the isolated eigenvalue-eigenvector pairs in this model are shown to carry the clustering information upon a separability condition classical in spiked matrix models. We evaluate precisely the position of these eigenvalues and the content of the eigenvectors, which unveil important (sometimes quite disruptive) aspects of kernel spectral clustering both from a theoretical and practical standpoints. Our results are then compared to the actual clustering performance of images from the MNIST database, thereby revealing an important match between theory and practice.

math.ST↗

arXiv2016

The Asymptotic Performance of Linear Echo State Neural Networks

In this article, a study of the mean-square error (MSE) performance of linear echo-state neural networks is performed, both for training and testing tasks. Considering the realistic setting of noise present at the network nodes, we derive deterministic equivalents for the aforementioned MSE in the limit where the number of input data $T$ and network size $n$ both grow large. Specializing then the network connectivity matrix to specific random settings, we further obtain simple formulas that provide new insights on the performance of such networks.

cs.LG↗

arXiv2016

Spectral analysis of the Gram matrix of mixture models

This text is devoted to the asymptotic study of some spectral properties of the Gram matrix $W^{\sf T} W$ built upon a collection $w_1, \ldots, w_n\in \mathbb{R}^p$ of random vectors (the columns of $W$), as both the number $n$ of observations and the dimension $p$ of the observations tend to infinity and are of similar order of magnitude. The random vectors $w_1, \ldots, w_n$ are independent observations, each of them belonging to one of $k$ classes $\mathcal{C}_1,\ldots, \mathcal{C}_k$. The observations of each class $\mathcal{C}_a$ ($1\le a\le k$) are characterized by their distribution $\mathcal{N}(0, p^{-1}C_a)$, where $C_1, \ldots, C_k$ are some non negative definite $p\times p$ matrices. The cardinality $n_a$ of class $\mathcal{C}_a$ and the dimension $p$ of the observations are such that $\frac{n_a}{n}$ ($1\le a\le k$) and $\frac{p}{n}$ stay bounded away from $0$ and $+\infty$. We provide deterministic equivalents to the empirical spectral distribution of $W^{\sf T}W$ and to the matrix entries of its resolvent (as well as of the resolvent of $WW^{\sf T}$). These deterministic equivalents are defined thanks to the solutions of a fixed-point system. Besides, we prove that $W^{\sf T} W$ has asymptotically no eigenvalues outside the bulk of its spectrum, defined thanks to these deterministic equivalents. These results are directly used in our companion paper "Kernel spectral clustering of large dimensional data", which is devoted to the analysis of the spectral clustering algorithm in large dimensions. They also find applications in various other fields such as wireless communications where functionals of the aforementioned resolvents allow one to assess the communication performance across multi-user multi-antenna channels.

math.PR↗

arXiv2015

The Second-Order Coding Rate of the MIMO Rayleigh Block-Fading Channel

The second-order coding rate of the multiple-input multiple-output (MIMO) quasi-static Rayleigh fading channel is studied. We tackle this problem via an information-spectrum approach and statistical bounds based on recent random matrix theory techniques. We derive a central limit theorem (CLT) to analyze the information density in the regime where the block-length n and the number of transmit and receive antennas K and N, respectively, grow simultaneously large. This result leads to the characterization of closed-form upper and lower bounds on the optimal average error probability when the coding rate is within O((nK)^-1/2) of the asymptotic capacity.

cs.IT↗

arXiv2015

Optimal Design of the Adaptive Normalized Matched Filter Detector

This article addresses improvements on the design of the adaptive normalized matched filter (ANMF) for radar detection. It is well-acknowledged that the estimation of the noise-clutter covariance matrix is a fundamental step in adaptive radar detection. In this paper, we consider regularized estimation methods which force by construction the eigenvalues of the scatter estimates to be greater than a positive regularization parameter rho. This makes them more suitable for high dimensional problems with a limited number of secondary data samples than traditional sample covariance estimates. While an increase of rho seems to improve the conditioning of the estimate, it might however cause it to significantly deviate from the true covariance matrix. The setting of the optimal regularization parameter is a difficult question for which no convincing answers have thus far been provided. This constitutes the major motivation behind our work. More specifically, we consider the design of the ANMF detector for two kinds of regularized estimators, namely the regularized sample covariance matrix (RSCM), appropriate when the clutter follows a Gaussian distribution and the regularized Tyler estimator (RTE) for non-Gaussian spherically invariant distributed clutters. Based on recent random matrix theory results studying the asymptotic fluctuations of the statistics of the ANMF detector when the number of samples and their dimension grow together to infinity, we propose a design for the regularization parameter that maximizes the detection probability under constant false alarm rates. Simulation results which support the efficiency of the proposed method are provided in order to illustrate the gain of the proposed optimal design over conventional settings of the regularization parameter.

cs.IT↗

arXiv2015

Interference-Aware RZF Precoding for Multi Cell Downlink Systems

Recently, a structure of an optimal linear precoder for multi cell downlink systems has been described in [1, Eq (3.33)]. Other references (e.g., [2,3]) have used simplified versions of the precoder to obtain promising performance gains. These gains have been hypothesized to stem from the additional degrees of freedom that allow for interference mitigation through interference relegation to orthogonal subspaces. However, no conclusive or rigorous understanding has yet been developed. In this paper, we build on an intuitive interference induction trade-off and the aforementioned precoding structure to propose an interference aware RZF (iaRZF) precoding scheme for multi cell downlink systems and we analyze its rate performance. Special emphasis is placed on the induced interference mitigation mechanism of iaRZF. For example, we will verify the intuitive expectation that the precoder structure can either completely remove induced inter-cell or intra-cell interference. We state new results from large-scale random matrix theory that make it possible to give more intuitive and insightful explanations of the precoder behavior, also for cases involving imperfect channel state information (CSI). We remark especially that the interference-aware precoder makes use of all available information about interfering channels to improve performance. Even very poor CSI allows for significant sum-rate gains. Our obtained insights are then used to propose heuristic precoder parameters for arbitrary systems, whose effectiveness are shown in more involved system scenarios. Furthermore, calculation and implementation of these parameters does not require explicit inter base station cooperation.

cs.IT↗

arXiv2015

Convergence and Fluctuations of Regularized Tyler Estimators

This article studies the behavior of regularized Tyler estimators (RTEs) of scatter matrices. The key advantages of these estimators are twofold. First, they guarantee by construction a good conditioning of the estimate and second, being a derivative of robust Tyler estimators, they inherit their robustness properties, notably their resilience to the presence of outliers. Nevertheless, one major problem that poses the use of RTEs in practice is represented by the question of setting the regularization parameter $ρ$. While a high value of $ρ$ is likely to push all the eigenvalues away from zero, it comes at the cost of a larger bias with respect to the population covariance matrix. A deep understanding of the statistics of RTEs is essential to come up with appropriate choices for the regularization parameter. This is not an easy task and might be out of reach, unless one considers asymptotic regimes wherein the number of observations $n$ and/or their size $N$ increase together. First asymptotic results have recently been obtained under the assumption that $N$ and $n$ are large and commensurable. Interestingly, no results concerning the regime of $n$ going to infinity with $N$ fixed exist, even though the investigation of this assumption has usually predated the analysis of the most difficult $N$ and $n$ large case. This motivates our work. In particular, we prove in the present paper that the RTEs converge to a deterministic matrix when $n\to\infty$ with $N$ fixed, which is expressed as a function of the theoretical covariance matrix. We also derive the fluctuations of the RTEs around this deterministic matrix and establish that these fluctuations converge in distribution to a multivariate Gaussian distribution with zero mean and a covariance depending on the population covariance and the parameter $ρ$.

cs.IT↗

arXiv2015

A Robust Statistics Approach to Minimum Variance Portfolio Optimization

We study the design of portfolios under a minimum risk criterion. The performance of the optimized portfolio relies on the accuracy of the estimated covariance matrix of the portfolio asset returns. For large portfolios, the number of available market returns is often of similar order to the number of assets, so that the sample covariance matrix performs poorly as a covariance estimator. Additionally, financial market data often contain outliers which, if not correctly handled, may further corrupt the covariance estimation. We address these shortcomings by studying the performance of a hybrid covariance matrix estimator based on Tyler's robust M-estimator and on Ledoit-Wolf's shrinkage estimator while assuming samples with heavy-tailed distribution. Employing recent results from random matrix theory, we develop a consistent estimator of (a scaled version of) the realized portfolio risk, which is minimized by optimizing online the shrinkage intensity. Our portfolio optimization method is shown via simulations to outperform existing methods both for synthetic and real market data.

q-fin.PM↗

arXiv2015

Large Dimensional Analysis of Robust M-Estimators of Covariance with Outliers

A large dimensional characterization of robust M-estimators of covariance (or scatter) is provided under the assumption that the dataset comprises independent (essentially Gaussian) legitimate samples as well as arbitrary deterministic samples, referred to as outliers. Building upon recent random matrix advances in the area of robust statistics, we specifically show that the so-called Maronna M-estimator of scatter asymptotically behaves similar to well-known random matrices when the population and sample sizes grow together to infinity. The introduction of outliers leads the robust estimator to behave asymptotically as the weighted sum of the sample outer products, with a constant weight for all legitimate samples and different weights for the outliers. A fine analysis of this structure reveals importantly that the propensity of the M-estimator to attenuate (or enhance) the impact of outliers is mostly dictated by the alignment of the outliers with the inverse population covariance matrix of the legitimate samples. Thus, robust M-estimators can bring substantial benefits over more simplistic estimators such as the per-sample normalized version of the sample covariance matrix, which is not capable of differentiating the outlying samples. The analysis shows that, within the class of Maronna's estimators of scatter, the Huber estimator is most favorable for rejecting outliers. On the contrary, estimators more similar to Tyler's scale invariant estimator (often preferred in the literature) run the risk of inadvertently enhancing some outliers.

math.ST↗