arXiv subjects

Esa Ollila

Publications and source records attributed to Esa Ollila.

At least 37 records · Page 2Linked to original sources

arXiv2024

Sparse signal recovery and source localization via covariance learning

In the Multiple Measurements Vector (MMV) model, measurement vectors are connected to unknown, jointly sparse signal vectors through a linear regression model employing a single known measurement matrix (or dictionary). Typically, the number of atoms (columns of the dictionary) is greater than the number measurements and the sparse signal recovery problem is generally ill-posed. In this paper, we treat the signals and measurement noise as independent Gaussian random vectors with unknown signal covariance matrix and noise variance, respectively, and characterize the solution of the likelihood equation in terms of fixed point equation, thereby enabling the recovery of the sparse signal support (sources with non-zero variances) via a block coordinate descent (BCD) algorithm that leverage the FP characterization of the likelihood equation. Additionally, a greedy pursuit method, analogous to popular simultaneous orthogonal matching pursuit (OMP), is introduced. Our numerical examples demonstrate effectiveness of the proposed covariance learning (CL) algorithms both in classic sparse signal recovery as well as in direction-of-arrival (DOA) estimation problems where they perform favourably compared to the state-of-the-art algorithms under a broad variety of settings.

stat.ME↗

arXiv2023

Linear shrinkage of sample covariance matrix or matrices under elliptical distributions: a review

This chapter reviews methods for linear shrinkage of the sample covariance matrix (SCM) and matrices (SCM-s) under elliptical distributions in single and multiple populations settings, respectively. In the single sample setting a popular linear shrinkage estimator is defined as a linear combination of the sample covariance matrix (SCM) with a scaled identity matrix. The optimal shrinkage coefficients minimizing the mean squared error (MSE) under elliptical sampling are shown to be functions of few key parameters only, such as elliptical kurtosis and sphericity parameter. Similar results and estimators are derived for multiple population setting and applications of the studied shrinkage estimators are illustrated in portfolio optimization.

stat.ME↗

arXiv2023

Robust and Sparse M-Estimation of DOA

A robust and sparse Direction of Arrival (DOA) estimator is derived for array data that follows a Complex Elliptically Symmetric (CES) distribution with zero-mean and finite second-order moments. The derivation allows to choose the loss function and four loss functions are discussed in detail: the Gauss loss which is the Maximum-Likelihood (ML) loss for the circularly symmetric complex Gaussian distribution, the ML-loss for the complex multivariate $t$-distribution (MVT) with $ν$ degrees of freedom, as well as Huber and Tyler loss functions. For Gauss loss, the method reduces to Sparse Bayesian Learning (SBL). The root mean square DOA error of the derived estimators is discussed for Gaussian, MVT, and $ε$-contaminated data. The robust SBL estimators perform well for all cases and nearly identical with classical SBL for Gaussian noise.

math.ST↗

arXiv2023

Affine equivariant Tyler's M-estimator applied to tail parameter learning of elliptical distributions

We propose estimating the scale parameter (mean of the eigenvalues) of the scatter matrix of an unspecified elliptically symmetric distribution using weights obtained by solving Tyler's M-estimator of the scatter matrix. The proposed Tyler's weights-based estimate (TWE) of scale is then used to construct an affine equivariant Tyler's M-estimator as a weighted sample covariance matrix using normalized Tyler's weights. We then develop a unified framework for estimating the unknown tail parameter of the elliptical distribution (such as the degrees of freedom (d.o.f.) $ν$ of the multivariate $t$ (MVT) distribution). Using the proposed TWE of scale, a new robust estimate of the d.o.f. parameter of MVT distribution is proposed with excellent performance in heavy-tailed scenarios, outperforming other competing methods. R-package is available that implements the proposed method.

stat.ME↗

arXiv2023

Regularized EM algorithm

Expectation-Maximization (EM) algorithm is a widely used iterative algorithm for computing (local) maximum likelihood estimate (MLE). It can be used in an extensive range of problems, including the clustering of data based on the Gaussian mixture model (GMM). Numerical instability and convergence problems may arise in situations where the sample size is not much larger than the data dimensionality. In such low sample support (LSS) settings, the covariance matrix update in the EM-GMM algorithm may become singular or poorly conditioned, causing the algorithm to crash. On the other hand, in many signal processing problems, a priori information can be available indicating certain structures for different cluster covariance matrices. In this paper, we present a regularized EM algorithm for GMM-s that can make efficient use of such prior knowledge as well as cope with LSS situations. The method aims to maximize a penalized GMM likelihood where regularized estimation may be used to ensure positive definiteness of covariance matrix updates and shrink the estimators towards some structured target covariance matrices. We show that the theoretical guarantees of convergence hold, leading to better performing EM algorithm for structured covariance matrix models or with low sample settings.

stat.ML↗

arXiv2022

Linear pooling of sample covariance matrices

We consider the problem of estimating high-dimensional covariance matrices of $K$-populations or classes in the setting where the sample sizes are comparable to the data dimension. We propose estimating each class covariance matrix as a distinct linear combination of all class sample covariance matrices. This approach is shown to reduce the estimation error when the sample sizes are limited, and the true class covariance matrices share a somewhat similar structure. We develop an effective method for estimating the coefficients in the linear combination that minimize the mean squared error under the general assumption that the samples are drawn from (unspecified) elliptically symmetric distributions possessing finite fourth-order moments. To this end, we utilize the spatial sign covariance matrix, which we show (under rather general conditions) to be an asymptotically unbiased estimator of the normalized covariance matrix as the dimension grows to infinity. We also show how the proposed method can be used in choosing the regularization parameters for multiple target matrices in a single class covariance matrix estimation problem. We assess the proposed method via numerical simulation studies including an application in global minimum variance portfolio optimization using real stock data.

stat.ME↗

arXiv2021

On the variability of the sample covariance matrix under complex elliptical distributions

We derive the form of the variance-covariance matrix for any affine equivariant matrix-valued statistics when sampling from complex elliptical distributions. We then use this result to derive the variance-covariance matrix of the sample covariance matrix (SCM) as well as its theoretical mean squared error (MSE) when finite fourth-order moments exist. Finally, illustrative examples of the formulas are presented.

math.ST↗

arXiv2021

Coupled regularized sample covariance matrix estimator for multiple classes

The estimation of covariance matrices of multiple classes with limited training data is a difficult problem. The sample covariance matrix (SCM) is known to perform poorly when the number of variables is large compared to the available number of samples. In order to reduce the mean squared error (MSE) of the SCM, regularized (shrinkage) SCM estimators are often used. In this work, we consider regularized SCM (RSCM) estimators for multiclass problems that couple together two different target matrices for regularization: the pooled (average) SCM of the classes and the scaled identity matrix. Regularization toward the pooled SCM is beneficial when the population covariances are similar, whereas regularization toward the identity matrix guarantees that the estimators are positive definite. We derive the MSE optimal tuning parameters for the estimators as well as propose a method for their estimation under the assumption that the class populations follow (unspecified) elliptical distributions with finite fourth-order moments. The MSE performance of the proposed coupled RSCMs are evaluated with simulations and in a regularized discriminant analysis (RDA) classification set-up on real data. The results based on three different real data sets indicate comparable performance to cross-validation but with a significant speed-up in computation time.

stat.ME↗

arXiv2021

Regularized tapered sample covariance matrix

Covariance matrix tapers have a long history in signal processing and related fields. Examples of applications include autoregressive models (promoting a banded structure) or beamforming (widening the spectral null width associated with an interferer). In this paper, the focus is on high-dimensional setting where the dimension $p$ is high, while the data aspect ratio $n/p$ is low. We propose an estimator called Tabasco (TApered or BAnded Shrinkage COvariance matrix) that shrinks the tapered sample covariance matrix towards a scaled identity matrix. We derive optimal and estimated (data adaptive) regularization parameters that are designed to minimize the mean squared error (MSE) between the proposed shrinkage estimator and the true covariance matrix. These parameters are derived under the general assumption that the data is sampled from an unspecified elliptically symmetric distribution with finite 4th order moments (both real- and complex-valued cases are addressed). Simulation studies show that the proposed Tabasco outperforms all competing tapering covariance matrix estimators in diverse setups. A space-time adaptive processing (STAP) application also illustrates the benefit of the proposed estimator in a practical signal processing setup.

stat.ME↗

arXiv2021

Coupled Feature Learning for Multimodal Medical Image Fusion

Multimodal image fusion aims to combine relevant information from images acquired with different sensors. In medical imaging, fused images play an essential role in both standard and automated diagnosis. In this paper, we propose a novel multimodal image fusion method based on coupled dictionary learning. The proposed method is general and can be employed for different medical imaging modalities. Unlike many current medical fusion methods, the proposed approach does not suffer from intensity attenuation nor loss of critical information. Specifically, the images to be fused are decomposed into coupled and independent components estimated using sparse representations with identical supports and a Pearson correlation constraint, respectively. An alternating minimization algorithm is designed to solve the resulting optimization problem. The final fusion step uses the max-absolute-value rule. Experiments are conducted using various pairs of multimodal inputs, including real MR-CT and MR-PET images. The resulting performance and execution times show the competitiveness of the proposed method in comparison with state-of-the-art medical image fusion methods.

cs.CV↗

arXiv2020

A Compressive Classification Framework for High-Dimensional Data

We propose a compressive classification framework for settings where the data dimensionality is significantly higher than the sample size. The proposed method, referred to as compressive regularized discriminant analysis (CRDA) is based on linear discriminant analysis and has the ability to select significant features by using joint-sparsity promoting hard thresholding in the discriminant rule. Since the number of features is larger than the sample size, the method also uses state-of-the-art regularized sample covariance matrix estimators. Several analysis examples on real data sets, including image, speech signal and gene expression data illustrate the promising improvements offered by the proposed CRDA classifier in practise. Overall, the proposed method gives fewer misclassification errors than its competitors, while at the same time achieving accurate feature selection results. The open-source R package and MATLAB toolbox of the proposed method (named compressiveRDA) is freely available.

stat.ML↗

arXiv2020

Shrinking the eigenvalues of M-estimators of covariance matrix

A highly popular regularized (shrinkage) covariance matrix estimator is the shrinkage sample covariance matrix (SCM) which shares the same set of eigenvectors as the SCM but shrinks its eigenvalues toward the grand mean of the eigenvalues of the SCM. In this paper, a more general approach is considered in which the SCM is replaced by an M-estimator of scatter matrix and a fully automatic data adaptive method to compute the optimal shrinkage parameter with minimum mean squared error is proposed. Our approach permits the use of any weight function such as Gaussian, Huber's, Tyler's, or t-weight functions, all of which are commonly used in M-estimation framework. Our simulation examples illustrate that shrinkage M-estimators based on the proposed optimal tuning combined with robust weight function do not loose in performance to shrinkage SCM estimator when the data is Gaussian, but provide significantly improved performance when the data is sampled from an unspecified heavy-tailed elliptically symmetric distribution. Also, real-world and synthetic stock market data validate the performance of the proposed method in practical applications.

stat.ME↗

arXiv2020

Modelling Graph Errors: Towards Robust Graph Signal Processing

The first step for any graph signal processing (GSP) procedure is to learn the graph signal representation, i.e., to capture the dependence structure of the data into an adjacency matrix. Indeed, the adjacency matrix is typically not known a priori and has to be learned. However, it is learned with errors. A little attention has been paid to modelling such errors in the adjacency matrix, and studying their effects on GSP methods. However, modelling errors in the adjacency matrix will enable both to study the graph error effects in GSP and to develop robust GSP algorithms. In this paper, we therefore introduce practically justifiable graph error models. We also study, both analytically when possible and numerically, the graph error effect on the performance of GSP methods in different types of problems such as filtering of graph signals and independent component analysis of graph signals (graph decorrelation).

cs.IT↗

arXiv2020

Block-wise Minimization-Majorization algorithm for Huber's criterion: sparse learning and applications

Huber's criterion can be used for robust joint estimation of regression and scale parameters in the linear model. Huber's (Huber, 1981) motivation for introducing the criterion stemmed from non-convexity of the joint maximum likelihood objective function as well as non-robustness (unbounded influence function) of the associated ML-estimate of scale. In this paper, we illustrate how the original algorithm proposed by Huber can be set within the block-wise minimization majorization framework. In addition, we propose novel data-adaptive step sizes for both the location and scale, which are further improving the convergence. We then illustrate how Huber's criterion can be used for sparse learning of underdetermined linear model using the iterative hard thresholding approach. We illustrate the usefulness of the algorithms in an image denoising application and simulation studies.

stat.ML↗

arXiv2020

Graph Signal Processing Meets Blind Source Separation

In graph signal processing (GSP), prior information on the dependencies in the signal is collected in a graph which is then used when processing or analyzing the signal. Blind source separation (BSS) techniques have been developed and analyzed in different domains, but for graph signals the research on BSS is still in its infancy. In this paper, this gap is filled with two contributions. First, a nonparametric BSS method, which is relevant to the GSP framework, is refined, the Cramér-Rao bound (CRB) for mixing and unmixing matrix estimators in the case of Gaussian moving average graph signals is derived, and for studying the achievability of the CRB, a new parametric method for BSS of Gaussian moving average graph signals is introduced. Second, we also consider BSS of non-Gaussian graph signals and two methods are proposed. Identifiability conditions show that utilizing both graph structure and non-Gaussianity provides a more robust approach than methods which are based on only either graph dependencies or non-Gaussianity. It is also demonstrated by numerical study that the proposed methods are more efficient in separating non-Gaussian graph signals.

stat.ME↗

arXiv2020

M-estimators of scatter with eigenvalue shrinkage

A popular regularized (shrinkage) covariance estimator is the shrinkage sample covariance matrix (SCM) which shares the same set of eigenvectors as the SCM but shrinks its eigenvalues toward its grand mean. In this paper, a more general approach is considered in which the SCM is replaced by an M-estimator of scatter matrix and a fully automatic data adaptive method to compute the optimal shrinkage parameter with minimum mean squared error is proposed. Our approach permits the use of any weight function such as Gaussian, Huber's, or $t$ weight functions, all of which are commonly used in M-estimation framework. Our simulation examples illustrate that shrinkage M-estimators based on the proposed optimal tuning combined with robust weight function do not loose in performance to shrinkage SCM estimator when the data is Gaussian, but provide significantly improved performance when the data is sampled from a heavy-tailed distribution.

stat.ME↗

arXiv2018

Optimal shrinkage covariance matrix estimation under random sampling from elliptical distributions

This paper considers the problem of estimating a high-dimensional (HD) covariance matrix when the sample size is smaller, or not much larger, than the dimensionality of the data, which could potentially be very large. We develop a regularized sample covariance matrix (RSCM) estimator which can be applied in commonly occurring sparse data problems. The proposed RSCM estimator is based on estimators of the unknown optimal (oracle) shrinkage parameters that yield the minimum mean squared error (MMSE) between the RSCM and the true covariance matrix when the data is sampled from an unspecified elliptically symmetric distribution. We propose two variants of the RSCM estimator which differ in the approach in which they estimate the underlying sphericity parameter involved in the theoretical optimal shrinkage parameter. The performance of the proposed RSCM estimators are evaluated with numerical simulation studies. In particular when the sample sizes are low, the proposed RSCM estimators often show a significant improvement over the conventional RSCM estimator by Ledoit and Wolf (2004). We further evaluate the performance of the proposed estimators in classification and portfolio optimization problems with real data wherein the proposed methods are able to outperform the benchmark methods.

stat.ME↗

arXiv2018

Simultaneous Signal Subspace Rank and Model Selection with an Application to Single-snapshot Source Localization

This paper proposes a novel method for model selection in linear regression by utilizing the solution path of $\ell_1$ regularized least-squares (LS) approach (i.e., Lasso). This method applies the complex-valued least angle regression and shrinkage (c-LARS) algorithm coupled with a generalized information criterion (GIC) and referred to as the c-LARS-GIC method. c-LARS-GIC is a two-stage procedure, where firstly precise values of the regularization parameter, called knots, at which a new predictor variable enters (or leaves) the active set are computed in the Lasso solution path. Active sets provide a nested sequence of regression models and GIC then selects the best model. The sparsity order of the chosen model serves as an estimate of the model order and the LS fit based only on the active set of the model provides an estimate of the regression parameter vector. We then consider a source localization problem, where the aim is to detect the number of impinging source waveforms at a sensor array as well to estimate their direction-of-arrivals (DoA-s) using only a single-snapshot measurement. We illustrate via simulations that, after formulating the problem as a grid-based sparse signal reconstruction problem, the proposed c-LARS-GIC method detects the number of sources with high probability while at the same time it provides accurate estimates of source locations.

stat.ME↗