arXiv subjects

Esa Ollila

Publications and source records attributed to Esa Ollila.

48 records · Page 3Linked to original sources

arXiv2018

Sequential adaptive elastic net approach for single-snapshot source localization

This paper proposes efficient algorithms for accurate recovery of direction-of-arrival (DoA) of sources from single-snapshot measurements using compressed beamforming (CBF). In CBF, the conventional sensor array signal model is cast as an underdetermined complex-valued linear regression model and sparse signal recovery methods are used for solving the DoA finding problem. We develop a complex-valued pathwise weighted elastic net (c-PW-WEN) algorithm that finds solutions at knots of penalty parameter values over a path (or grid) of EN tuning parameter values. c-PW-WEN also computes Lasso or weighted Lasso in its path. We then propose a sequential adaptive EN (SAEN) method that is based on c-PW-WEN algorithm with adaptive weights that depend on the previous solution. Extensive simulation studies illustrate that SAEN improves the probability of exact recovery of true support compared to conventional sparse signal recovery approaches such as Lasso, elastic net or orthogonal matching pursuit in several challenging multiple target scenarios. The effectiveness of SAEN is more pronounced in the presence of high mutual coherence.

stat.ME↗

arXiv2018

Compressive Regularized Discriminant Analysis of High-Dimensional Data with Applications to Microarray Studies

We propose a modification of linear discriminant analysis, referred to as compressive regularized discriminant analysis (CRDA), for analysis of high-dimensional datasets. CRDA is specially designed for feature elimination purpose and can be used as gene selection method in microarray studies. CRDA lends ideas from $\ell_{q,1}$ norm minimization algorithms in the multiple measurement vectors (MMV) model and utilizes joint-sparsity promoting hard thresholding for feature elimination. A regularization of the sample covariance matrix is also needed as we consider the challenging scenario where the number of features (variables) is comparable or exceeding the sample size of the training dataset. A simulation study and four examples of real-life microarray datasets evaluate the performances of CRDA based classifiers. Overall, the proposed method gives fewer misclassification errors than its competitors, while at the same time achieving accurate feature selection.

stat.ME↗

arXiv2017

Optimal High-Dimensional Shrinkage Covariance Estimation for Elliptical Distributions

We derive an optimal shrinkage sample covariance matrix (SCM) estimator which is suitable for high dimensional problems and when sampling from an unspecified elliptically symmetric distribution. Specifically, we derive the optimal (oracle) shrinkage parameters that obtain the minimum mean-squared error (MMSE) between the shrinkage SCM and the true covariance matrix when sampling from an elliptical distribution. Subsequently, we show how the oracle shrinkage parameters can be consistently estimated under the random matrix theory regime. Simulations show the advantage of the proposed estimator over the conventional shrinkage SCM estimator due to Ledoit and Wolf (2004). The proposed shrinkage SCM estimator often provides significantly better performance than the Ledoit-Wolf estimator and has the advantage that consistency is guaranteed over the whole class of elliptical distributions with finite 4th order moments.

stat.ME↗

arXiv2017

Pathwise Least Angle Regression and a Significance Test for the Elastic Net

Least angle regression (LARS) by Efron et al. (2004) is a novel method for constructing the piece-wise linear path of Lasso solutions. For several years, it remained also as the de facto method for computing the Lasso solution before more sophisticated optimization algorithms preceded it. LARS method has recently again increased its popularity due to its ability to find the values of the penalty parameters, called knots, at which a new parameter enters the active set of non-zero coefficients. Significance test for the Lasso by Lockhart et al. (2014), for example, requires solving the knots via the LARS algorithm. Elastic net (EN), on the other hand, is a highly popular extension of Lasso that uses a linear combination of Lasso and ridge regression penalties. In this paper, we propose a new novel algorithm, called pathwise (PW-)LARS-EN, that is able to compute the EN knots over a grid of EN tuning parameter α values. The developed PW-LARS-EN algorithm decreases the EN tuning parameter and exploits the previously found knot values and the original LARS algorithm. A covariance test statistic for the Lasso is then generalized to the EN for testing the significance of the predictors. Our simulation studies validate the fact that the test statistic has an asymptotic Exp(1) distribution.

stat.ME↗

arXiv2016

Simultaneous penalized M-estimation of covariance matrices using geodesically convex optimization

A common assumption when sampling $p$-dimensional observations from $K$ distinct group is the equality of the covariance matrices. In this paper, we propose two penalized $M$-estimation approaches for the estimation of the covariance or scatter matrices under the broader assumption that they may simply be close to each other, and hence roughly deviate from some positive definite "center". The first approach begins by generating a pooled $M$-estimator of scatter based on all the data, followed by a penalised $M$-estimator of scatter for each group, with the penalty term chosen so that the individual scatter matrices are shrunk towards the pooled scatter matrix. In the second approach, we minimize the sum of the individual group $M$-estimation cost functions together with an additive joint penalty term which enforces some similarity between the individual scatter estimators, i.e. shrinkage towards a mutual center. In both approaches, we utilize the concept of geodesic convexity to prove the existence and uniqueness of the penalized solution under general conditions. We consider three specific penalty functions based on the Euclidean, the Riemannian, and the Kullback-Leibler distances. In the second approach, the distance based penalties are shown to lead to estimators of the mutual center that are related to the arithmetic, the Riemannian and the harmonic means of positive definite matrices, respectively. A penalty based on an ellipticity measure is also considered which is particularly useful for shape matrix estimators. Fixed point equations are derived for each penalty function and the benefits of the estimators are illustrated in regularized discriminant analysis problem.

stat.ME↗

arXiv2016

Direction of arrival estimation using robust complex Lasso

The Lasso (Least Absolute Shrinkage and Selection Operator) has been a popular technique for simultaneous linear regression estimation and variable selection. In this paper, we propose a new novel approach for robust Lasso that follows the spirit of M-estimation. We define $M$-Lasso estimates of regression and scale as solutions to generalized zero subgradient equations. Another unique feature of this paper is that we consider complex-valued measurements and regression parameters, which requires careful mathematical characterization of the problem. An explicit and efficient algorithm for computing the $M$-Lasso solution is proposed that has comparable computational complexity as state-of-the-art algorithm for computing the Lasso solution. Usefulness of the $M$-Lasso method is illustrated for direction-of-arrival (DoA) estimation with sensor arrays in a single snapshot case.

stat.ME↗

arXiv2015

Nonparametric Simultaneous Sparse Recovery: an Application to Source Localization

We consider multichannel sparse recovery problem where the objective is to find good recovery of jointly sparse unknown signal vectors from the given multiple measurement vectors which are different linear combinations of the same known elementary vectors. Many popular greedy or convex algorithms perform poorly under non-Gaussian heavy-tailed noise conditions or in the face of outliers. In this paper, we propose the usage of mixed $\ell_{p,q}$ norms on data fidelity (residual matrix) term and the conventional $\ell_{0,2}$-norm constraint on the signal matrix to promote row-sparsity. We devise a greedy pursuit algorithm based on simultaneous normalized iterative hard thresholding (SNIHT) algorithm. Simulation studies highlight the effectiveness of the proposed approaches to cope with different noise environments (i.i.d., row i.i.d, etc) and outliers. Usefulness of the methods are illustrated in source localization application with sensor arrays.

cs.IT↗

arXiv2015

Multichannel sparse recovery of complex-valued signals using Huber's criterion

In this paper, we generalize Huber's criterion to multichannel sparse recovery problem of complex-valued measurements where the objective is to find good recovery of jointly sparse unknown signal vectors from the given multiple measurement vectors which are different linear combinations of the same known elementary vectors. This requires careful characterization of robust complex-valued loss functions as well as Huber's criterion function for the multivariate sparse regression problem. We devise a greedy algorithm based on simultaneous normalized iterative hard thresholding (SNIHT) algorithm. Unlike the conventional SNIHT method, our algorithm, referred to as HUB-SNIHT, is robust under heavy-tailed non-Gaussian noise conditions, yet has a negligible performance loss compared to SNIHT under Gaussian noise. Usefulness of the method is illustrated in source localization application with sensor arrays.

cs.IT↗

arXiv2015

Robust, scalable and fast bootstrap method for analyzing large scale data

In this paper we address the problem of performing statistical inference for large scale data sets i.e., Big Data. The volume and dimensionality of the data may be so high that it cannot be processed or stored in a single computing node. We propose a scalable, statistically robust and computationally efficient bootstrap method, compatible with distributed processing and storage systems. Bootstrap resamples are constructed with smaller number of distinct data points on multiple disjoint subsets of data, similarly to the bag of little bootstrap method (BLB) [1]. Then significant savings in computation is achieved by avoiding the re-computation of the estimator for each bootstrap sample. Instead, a computationally efficient fixed-point estimation equation is analytically solved via a smart approximation following the Fast and Robust Bootstrap method (FRB) [2]. Our proposed bootstrap method facilitates the use of highly robust statistical methods in analyzing large scale data sets. The favorable statistical properties of the method are established analytically. Numerical examples demonstrate scalability, low complexity and robust statistical performance of the method in analyzing large data sets.

stat.ME↗

arXiv2014

Regularized $M$-estimators of scatter matrix

In this paper, a general class of regularized $M$-estimators of scatter matrix are proposed which are suitable also for low or insufficient sample support (small $n$ and large $p$) problems. The considered class constitutes a natural generalization of $M$-estimators of scatter matrix (Maronna, 1976) and are defined as a solution to a penalized $M$-estimation cost function that depend on a pair $(α,β)$ of regularization parameters. We derive general conditions for uniqueness of the solution using concept of geodesic convexity. Since these conditions do not include Tyler's $M$-estimator, necessary and sufficient conditions for uniqueness of the penalized Tyler's cost function are established separately. For the regularized Tyler's $M$-estimator, we also derive a simple, closed form and data dependent solution for choosing the regularization parameter based on shape matrix matching in the mean squared sense. An iterative algorithm that converges to the solution of the regularized $M$-estimating equation is also provided. Finally, some simulations studies illustrate the improved accuracy of the proposed regularized $M$-estimators of scatter compared to their non-regularized counterparts in low sample support problems. An example of radar detection using normalized matched filter (NMF) illustrate that an adaptive NMF detector based on regularized $M$-estimators are able to maintain accurately the preset CFAR level and at at the same time provide similar probability of detection as the (theoretical) NMF detector.

stat.AP↗

arXiv2014

Robust iterative hard thresholding for compressed sensing

Compressed sensing (CS) or sparse signal reconstruction (SSR) is a signal processing technique that exploits the fact that acquired data can have a sparse representation in some basis. One popular technique to reconstruct or approximate the unknown sparse signal is the iterative hard thresholding (IHT) which however performs very poorly under non-Gaussian noise conditions or in the face of outliers (gross errors). In this paper, we propose a robust IHT method based on ideas from $M$-estimation that estimates the sparse signal and the scale of the error distribution simultaneously. The method has a negligible performance loss compared to IHT under Gaussian noise, but superior performance under heavy-tailed non-Gaussian noise conditions.

cs.IT↗

arXiv2012

On asymptotics of ICA estimators and their performance indices

Independent component analysis (ICA) has become a popular multivariate analysis and signal processing technique with diverse applications. This paper is targeted at discussing theoretical large sample properties of ICA unmixing matrix functionals. We provide a formal definition of unmixing matrix functional and consider two popular estimators in detail: the family based on two scatter matrices with the independence property (e.g., FOBI estimator) and the family of deflation-based fastICA estimators. The limiting behavior of the corresponding estimates is discussed and the asymptotic normality of the deflation-based fastICA estimate is proven under general assumptions. Furthermore, properties of several performance indices commonly used for comparison of different unmixing matrix estimates are discussed and a new performance index is proposed. The proposed index fullfills three desirable features which promote its use in practice and distinguish it from others. Namely, the index possesses an easy interpretation, is fast to compute and its asymptotic properties can be inferred from asymptotics of the unmixing matrix estimate. We illustrate the derived asymptotical results and the use of the proposed index with a small simulation study.

stat.ME↗