arXiv subjects

Frédéric Pascal

Publications and source records attributed to Frédéric Pascal.

At least 19 recordsLinked to original sources

arXiv2026

A Robust Framework for Model Order Selection in Correlated Large-Dimensional CES Noise

This paper addresses model order selection under large-dimensional, correlated, non-Gaussian noise. Sources are assumed to be embedded in additive Complex Elliptically Symmetric (CES) noise with an unknown Toeplitz-structured scatter matrix. We propose a two-stage robust framework: (i) a noise-whitening step based on a Toeplitz-rectified $M$-estimator of the scatter matrix, and (ii) signal subspace rank inference via large-dimensional Random Matrix Theory (RMT). Almost sure consistency of the proposed estimators is established, together with explicit RMT eigenvalue upper bounds separating signal from noise components, in the regime where the observation dimension $m$ and the sample size $N$ grow proportionally. Three estimation branches are derived, based respectively on the sample covariance matrix (SCM), Maronna's $M$-estimator, and the distribution-free Tyler $M$-estimator for whitening. The methodology is validated on synthetic data, real hyperspectral images, EEG recordings, and financial data, with significant gains over AIC and unwhitened methods.

stat.ME↗

arXiv2024

Elliptical Wishart distributions: information geometry, maximum likelihood estimator, performance analysis and statistical learning

This paper deals with Elliptical Wishart distributions - which generalize the Wishart distribution - in the context of signal processing and machine learning. Two algorithms to compute the maximum likelihood estimator (MLE) are proposed: a fixed point algorithm and a Riemannian optimization method based on the derived information geometry of Elliptical Wishart distributions. The existence and uniqueness of the MLE are characterized as well as the convergence of both estimation algorithms. Statistical properties of the MLE are also investigated such as consistency, asymptotic normality and an intrinsic version of Fisher efficiency. On the statistical learning side, novel classification and clustering methods are designed. For the $t$-Wishart distribution, the performance of the MLE and statistical learning algorithms are evaluated on both simulated and real EEG and hyperspectral data, showcasing the interest of our proposed methods.

stat.ML↗

arXiv2024

On Elliptical and Inverse Elliptical Wishart distributions

This paper deals with the Elliptical Wishart and Inverse Elliptical Wishart distributions, which play a major role when handling covariance matrices. Similarly to multivariate elliptical distributions, these form a large family of covariance distributions, encompassing, e.g., the Wishart or \textit{t}-Wishart ones. Our first major contribution is to derive a stochastic representation for Elliptical Wishart and Inverse Elliptical Wishart matrices. This later enables us to obtain various key statistical properties of Elliptical Wishart and Inverse Elliptical Wishart distributions such as expectations, variances, and Kronecker moments up to any orders. The stochastic representation also allows us to provide an efficient method to generate random matrices from Elliptical Wishart and Inverse Elliptical Wishart distributions. Finally, the practical interest of Elliptical Wishart distributions - in particular the \textit{t}-Wishart one - is demonstrated through a fitting experiment on real electroencephalographic data. This showcases their effectiveness in accurately modeling real covariance matrices.

math.ST↗

arXiv2024

Random matrix theory improved Fréchet mean of symmetric positive definite matrices

In this study, we consider the realm of covariance matrices in machine learning, particularly focusing on computing Fréchet means on the manifold of symmetric positive definite matrices, commonly referred to as Karcher or geometric means. Such means are leveraged in numerous machine-learning tasks. Relying on advanced statistical tools, we introduce a random matrix theory-based method that estimates Fréchet means, which is particularly beneficial when dealing with low sample support and a high number of matrices to average. Our experimental evaluation, involving both synthetic and real-world EEG and hyperspectral datasets, shows that we largely outperform state-of-the-art methods.

stat.ML↗

arXiv2024

Sparse PCA with False Discovery Rate Controlled Variable Selection

Sparse principal component analysis (PCA) aims at mapping large dimensional data to a linear subspace of lower dimension. By imposing loading vectors to be sparse, it performs the double duty of dimension reduction and variable selection. Sparse PCA algorithms are usually expressed as a trade-off between explained variance and sparsity of the loading vectors (i.e., number of selected variables). As a high explained variance is not necessarily synonymous with relevant information, these methods are prone to select irrelevant variables. To overcome this issue, we propose an alternative formulation of sparse PCA driven by the false discovery rate (FDR). We then leverage the Terminating-Random Experiments (T-Rex) selector to automatically determine an FDR-controlled support of the loading vectors. A major advantage of the resulting T-Rex PCA is that no sparsity parameter tuning is required. Numerical experiments and a stock market data example demonstrate a significant performance improvement.

stat.ML↗

arXiv2023

Deinterleaving RADAR emitters with optimal transport distances

Detection and identification of emitters provide vital information for defensive strategies in electronic intelligence. Based on a received signal containing pulses from an unknown number of emitters, this paper introduces an unsupervised methodology for deinterleaving RADAR signals based on a combination of clustering algorithms and optimal transport distances. The first step involves separating the pulses with a clustering algorithm under the constraint that the pulses of two different emitters cannot belong to the same cluster. Then, as the emitters exhibit complex behavior and can be represented by several clusters, we propose a hierarchical clustering algorithm based on an optimal transport distance to merge these clusters. A variant is also developed, capable of handling more complex signals. Finally, the proposed methodology is evaluated on simulated data provided through a realistic simulator. Results show that the proposed methods are capable of deinterleaving complex RADAR signals.

stat.ME↗

arXiv2023

Convex Parameter Estimation of Perturbed Multivariate Generalized Gaussian Distributions

The multivariate generalized Gaussian distribution (MGGD), also known as the multivariate exponential power (MEP) distribution, is widely used in signal and image processing. However, estimating MGGD parameters, which is required in practical applications, still faces specific theoretical challenges. In particular, establishing convergence properties for the standard fixed-point approach when both the distribution mean and the scatter (or the precision) matrix are unknown is still an open problem. In robust estimation, imposing classical constraints on the precision matrix, such as sparsity, has been limited by the non-convexity of the resulting cost function. This paper tackles these issues from an optimization viewpoint by proposing a convex formulation with well-established convergence properties. We embed our analysis in a noisy scenario where robustness is induced by modelling multiplicative perturbations. The resulting framework is flexible as it combines a variety of regularizations for the precision matrix, the mean and model perturbations. This paper presents proof of the desired theoretical properties, specifies the conditions preserving these properties for different regularization choices and designs a general proximal primal-dual optimization strategy. The experiments show a more accurate precision and covariance matrix estimation with similar performance for the mean vector parameter compared to Tyler's M-estimator. In a high-dimensional setting, the proposed method outperforms the classical GLASSO, one of its robust extensions, and the regularized Tyler's estimator.

stat.ME↗

arXiv2023

Choosing the parameter of the Fermat distance: navigating geometry and noise

The Fermat distance has been recently established as a useful tool for machine learning tasks when a natural distance is not directly available to the practitioner or to improve the results given by Euclidean distances by exploding the geometrical and statistical properties of the dataset. This distance depends on a parameter $α$ that greatly impacts the performance of subsequent tasks. Ideally, the value of $α$ should be large enough to navigate the geometric intricacies inherent to the problem. At the same, it should remain restrained enough to sidestep any deleterious ramifications stemming from noise during the process of distance estimation. We study both theoretically and through simulations how to select this parameter.

stat.ML↗

arXiv2023

A Robust and Flexible EM Algorithm for Mixtures of Elliptical Distributions with Missing Data

This paper tackles the problem of missing data imputation for noisy and non-Gaussian data. A classical imputation method, the Expectation Maximization (EM) algorithm for Gaussian mixture models, has shown interesting properties when compared to other popular approaches such as those based on k-nearest neighbors or on multiple imputations by chained equations. However, Gaussian mixture models are known to be non-robust to heterogeneous data, which can lead to poor estimation performance when the data is contaminated by outliers or follows non-Gaussian distributions. To overcome this issue, a new EM algorithm is investigated for mixtures of elliptical distributions with the property of handling potential missing data. This paper shows that this problem reduces to the estimation of a mixture of Angular Gaussian distributions under generic assumptions (i.e., each sample is drawn from a mixture of elliptical distributions, which is possibly different for one sample to another). In that case, the complete-data likelihood associated with mixtures of elliptical distributions is well adapted to the EM framework with missing data thanks to its conditional distribution, which is shown to be a multivariate $t$-distribution. Experimental results on synthetic data demonstrate that the proposed algorithm is robust to outliers and can be used with non-Gaussian data. Furthermore, experiments conducted on real-world datasets show that this algorithm is very competitive when compared to other classical imputation methods.

stat.ML↗

arXiv2022

Riemannian classification of EEG signals with missing values

This paper proposes a strategy to handle missing data for the classification of electroencephalograms using covariance matrices. It relies on the observed-data likelihood within an expectation-maximization algorithm. This approach is compared to two existing state-of-the-art methods: (i) covariance matrices computed with imputed data; (ii) Riemannian averages of partially observed covariance matrix. All approaches are combined with the minimum distance to Riemannian mean classifier and applied to a classification task of two widely known paradigms of brain-computer interfaces. In addition to be applicable for a wider range of missing data scenarios, the proposed strategy generally performs better than other methods on the considered real EEG data.

cs.HC↗

arXiv2022

Robust classification with flexible discriminant analysis in heterogeneous data

Linear and Quadratic Discriminant Analysis are well-known classical methods but can heavily suffer from non-Gaussian distributions and/or contaminated datasets, mainly because of the underlying Gaussian assumption that is not robust. To fill this gap, this paper presents a new robust discriminant analysis where each data point is drawn by its own arbitrary Elliptically Symmetrical (ES) distribution and its own arbitrary scale parameter. Such a model allows for possibly very heterogeneous, independent but non-identically distributed samples. After deriving a new decision rule, it is shown that maximum-likelihood parameter estimation and classification are very simple, fast and robust compared to state-of-the-art methods.

stat.ML↗

arXiv2021

PCA-based Multi Task Learning: a Random Matrix Approach

The article proposes and theoretically analyses a \emph{computationally efficient} multi-task learning (MTL) extension of popular principal component analysis (PCA)-based supervised learning schemes \cite{barshan2011supervised,bair2006prediction}. The analysis reveals that (i) by default learning may dramatically fail by suffering from \emph{negative transfer}, but that (ii) simple counter-measures on data labels avert negative transfer and necessarily result in improved performances. Supporting experiments on synthetic and real data benchmarks show that the proposed method achieves comparable performance with state-of-the-art MTL methods but at a \emph{significantly reduced computational cost}.

stat.ML↗

arXiv2021

Joint Estimation of Location and Scatter in Complex Elliptical Distributions: A robust semiparametric and computationally efficient $R$-estimator of the shape matrix

The joint estimation of the location vector and the shape matrix of a set of independent and identically Complex Elliptically Symmetric (CES) distributed observations is investigated from both the theoretical and computational viewpoints. This joint estimation problem is framed in the original context of semiparametric models allowing us to handle the (generally unknown) density generator as an \textit{infinite-dimensional} nuisance parameter. In the first part of the paper, a computationally efficient and memory saving implementation of the robust and semiparmaetric efficient $R$-estimator for shape matrices is derived. Building upon this result, in the second part, a joint estimator, relying on the Tyler's $M$-estimator of location and on the $R$-estimator of shape matrix, is proposed and its Mean Squared Error (MSE) performance compared with the Semiparametric Cramér-Rao Bound (CSCRB).

stat.ME↗

arXiv2020

Shrinking the eigenvalues of M-estimators of covariance matrix

A highly popular regularized (shrinkage) covariance matrix estimator is the shrinkage sample covariance matrix (SCM) which shares the same set of eigenvectors as the SCM but shrinks its eigenvalues toward the grand mean of the eigenvalues of the SCM. In this paper, a more general approach is considered in which the SCM is replaced by an M-estimator of scatter matrix and a fully automatic data adaptive method to compute the optimal shrinkage parameter with minimum mean squared error is proposed. Our approach permits the use of any weight function such as Gaussian, Huber's, Tyler's, or t-weight functions, all of which are commonly used in M-estimation framework. Our simulation examples illustrate that shrinkage M-estimators based on the proposed optimal tuning combined with robust weight function do not loose in performance to shrinkage SCM estimator when the data is Gaussian, but provide significantly improved performance when the data is sampled from an unspecified heavy-tailed elliptically symmetric distribution. Also, real-world and synthetic stock market data validate the performance of the proposed method in practical applications.

stat.ME↗

arXiv2020

A flexible EM-like clustering algorithm for noisy data

Though very popular, it is well known that the EM for GMM algorithm suffers from non-Gaussian distribution shapes, outliers and high-dimensionality. In this paper, we design a new robust clustering algorithm that can efficiently deal with noise and outliers in diverse data sets. As an EM-like algorithm, it is based on both estimations of clusters centers and covariances. In addition, using a semi-parametric paradigm, the method estimates an unknown scale parameter per data-point. This allows the algorithm to accommodate for heavier tails distributions and outliers without significantly loosing efficiency in various classical scenarios. We first derive and analyze the proposed algorithm in the context of elliptical distributions, showing in particular important insensitivity properties to the underlying data distributions. We then study the convergence and accuracy of the algorithm by considering first synthetic data. Then, we show that the proposed algorithm outperforms other classical unsupervised methods of the literature such as k-means, the EM for Gaussian mixture models and its recent modifications or spectral clustering when applied to real data sets as MNIST, NORB, and 20newsgroups.

stat.ML↗

arXiv2020

Robust Semiparametric Efficient Estimators in Elliptical Distributions

Covariance matrices play a major role in statistics, signal processing and machine learning applications. This paper focuses on the \textit{semiparametric} covariance/scatter matrix estimation problem in elliptical distributions. The class of elliptical distributions can be seen as a semiparametric model where the finite-dimensional vector of interest is given by the location vector and by the (vectorized) covariance/scatter matrix, while the density generator represents an infinite-dimensional nuisance function. The main aim of this work is then to provide possible estimators of the finite-dimensional parameter vector able to reconcile the two dichotomic concepts of \textit{robustness} and (semiparametric) \textit{efficiency}. An $R$-estimator satisfying these requirements has been recently proposed by Hallin, Oja and Paindaveine for real-valued elliptical data by exploiting the Le Cam's theory of \textit{one-step efficient estimators} and the \textit{rank-based statistics}. In this paper, we firstly recall the building blocks underlying the derivation of such real-valued $R$-estimator, then its extension to complex-valued data is proposed. Moreover, through numerical simulations, its estimation performance and robustness to outliers are investigated in a finite-sample regime.

eess.SP↗

arXiv2020

Properties of a new $R$-estimator of shape matrices

This paper aims at presenting a simulative analysis of the main properties of a new $R$-estimator of shape matrices in Complex Elliptically Symmetric (CES) distributed observations. First proposed by Hallin, Oja and Paindaveine for the real-valued case and then extended to the complex field in our recent work, this $R$-estimator has the remarkable property to be, at the same time, \textit{distributionally robust} and \textit{semiparametric efficient}. Here, the efficiency of different possible configurations of this $R$-estimator are investigated by comparing the resulting Mean Square Error (MSE) with the Constrained Semiparametric Cramér-Rao Bound (CSCRB). Moreover, its robustness to outliers is assessed and compared with the one of the celebrated Tyler's estimator.

eess.SP↗

arXiv2020

Robust Semiparametric DOA Estimation in non-Gaussian Environment

A general non-Gaussian semiparametric model is adopted to characterize the measurement vectors, i.e.\ the \textit{snapshots}, collected by a linear array. Moreover, the recently derived \textit{robust semiparametric efficient} $R$-estimator of the data covariance matrix is exploited to implement an original version of the MUSIC estimator. The efficiency of the resulting $R$-MUSIC algorithm is investigated by comparing its Mean Squared Error (MSE) in the estimation of the source spatial frequencies with the relevant Semiparametric Stochastic Cramér-Rao Bound (SSCRB).

eess.SP↗