arXiv subjects

Klaus Nordhausen

Publications and source records attributed to Klaus Nordhausen.

At least 55 records · Page 3Linked to original sources

arXiv2018

ICS for Multivariate Outlier Detection with Application to Quality Control

In high reliability standards fields such as automotive, avionics or aerospace, the detection of anomalies is crucial. An efficient methodology for automatically detecting multivariate outliers is introduced. It takes advantage of the remarkable properties of the Invariant Coordinate Selection (ICS) method. Based on the simultaneous spectral decomposition of two scatter matrices, ICS leads to an affine invariant coordinate system in which the Euclidian distance corresponds to a Mahalanobis Distance (MD) in the original coordinates. The limitations of MD are highlighted using theoretical arguments in a context where the dimension of the data is large. Unlike MD, ICS makes it possible to select relevant components which removes the limitations. Owing to the resulting dimension reduction, the method is expected to improve the power of outlier detection rules such as MD-based criteria. It also greatly simplifies outliers interpretation. The paper includes practical guidelines for using ICS in the context of a small proportion of outliers which is relevant in high reliability standards fields. The choice of scatter matrices together with the selection of relevant invariant components through parallel analysis and normality tests are addressed. The use of the regular covariance matrix and the so called matrix of fourth moments as the scatter pair is recommended. This choice combines the simplicity of implementation together with the possibility to derive theoretical results. A simulation study confirms the good properties of the proposal and compares it with other scatter pairs. This study also provides a comparison with Principal Component Analysis and MD. The performance of our proposal is also evaluated on several real data sets using a user-friendly R package accompanying the paper.

stat.ME↗

arXiv2018

On the number of signals in multivariate time series

We assume a second-order source separation model where the observed multivariate time series is a linear mixture of latent, temporally uncorrelated time series with some components pure white noise. To avoid the modelling of noise, we extract the non-noise latent components using some standard method, allowing the modelling of the extracted univariate time series individually. An important question is the determination of which of the latent components are of interest in modelling and which can be considered as noise. Bootstrap-based methods have recently been used in determining the latent dimension in various methods of unsupervised and supervised dimension reduction and we propose a set of similar estimation strategies for second-order stationary time series. Simulation studies and a sound wave example are used to show the method's effectiveness.

stat.ME↗

arXiv2017

Independent component analysis for multivariate functional data

We extend two methods of independent component analysis, fourth order blind identification and joint approximate diagonalization of eigen-matrices, to vector-valued functional data. Multivariate functional data occur naturally and frequently in modern applications, and extending independent component analysis to this setting allows us to distill important information from this type of data, going a step further than the functional principal component analysis. To allow the inversion of the covariance operator we make the assumption that the dependency between the component functions lies in a finite-dimensional subspace. In this subspace we define fourth cross-cumulant operators and use them to construct the two novel, Fisher consistent methods for solving the independent component problem for vector-valued functions. Both simulations and an application on a hand gesture data set show the usefulness and advantages of the proposed methods over functional principal component analysis.

math.ST↗

arXiv2017

Asymptotic and bootstrap tests for subspace dimension

Most linear dimension reduction methods proposed in the literature can be formulated using an appropriate pair of scatter matrices, see e.g. Ye and Weiss (2003), Tyler et al. (2009), Bura and Yang (2011), Liski et al. (2014) and Luo and Li (2016). The eigen-decomposition of one scatter matrix with respect to another is then often used to determine the dimension of the signal subspace and to separate signal and noise parts of the data. Three popular dimension reduction methods, namely principal component analysis (PCA), fourth order blind identification (FOBI) and sliced inverse regression (SIR) are considered in detail and the first two moments of subsets of the eigenvalues are used to test for the dimension of the signal space. The limiting null distributions of the test statistics are discussed and novel bootstrap strategies are suggested for the small sample cases. In all three cases, consistent test-based estimates of the signal subspace dimension are introduced as well. The asymptotic and bootstrap tests are compared in simulations and illustrated in real data examples.

stat.ME↗

arXiv2017

Independent component analysis for tensor-valued data

In preprocessing tensor-valued data, e.g. images and videos, a common procedure is to vectorize the observations and subject the resulting vectors to one of the many methods used for independent component analysis (ICA). However, the tensor structure of the original data is lost in the vectorization and, as a more suitable alternative, we propose the matrix- and tensor fourth order blind identification (MFOBI and TFOBI). In these tensorial extensions of the classic fourth order blind identification (FOBI) we assume a Kronecker structure for the mixing and perform FOBI simultaneously on each direction of the observed tensors. We discuss the theory and assumptions behind MFOBI and TFOBI and provide two different algorithms and related estimates of the unmixing matrices along with their asymptotic properties. Finally, simulations are used to compare the method's performance with that of classical FOBI for vectorized data and we end with a real data clustering example.

math.ST↗

arXiv2017

Blind source separation of tensor-valued time series

The blind source separation model for multivariate time series generally assumes that the observed series is a linear transformation of an unobserved series with temporally uncorrelated or independent components. Given the observations, the objective is to find a linear transformation that recovers the latent series. Several methods for accomplishing this exist and three particular ones are the classic SOBI and the recently proposed generalized FOBI (gFOBI) and generalized JADE (gJADE), each based on the use of joint lagged moments. In this paper we generalize the methodologies behind these algorithms for tensor-valued time series. We assume that our data consists of a tensor observed at each time point and that the observations are linear transformations of latent tensors we wish to estimate. The tensorial generalizations are shown to have particularly elegant forms and we show that each of them is Fisher consistent and orthogonal equivariant. Comparing the new methods with the original ones in various settings shows that the tensorial extensions are superior to both their vector-valued counterparts and to two existing tensorial dimension reduction methods for i.i.d. data. Finally, applications to fMRI-data and video processing show that the methods are capable of extracting relevant information from noisy high-dimensional data.

math.ST↗

arXiv2017

Asymptotic and bootstrap tests for the dimension of the non-Gaussian subspace

Dimension reduction is often a preliminary step in the analysis of large data sets. The so-called non-Gaussian component analysis searches for a projection onto the non-Gaussian part of the data, and it is then important to know the correct dimension of the non-Gaussian signal subspace. In this paper we develop asymptotic as well as bootstrap tests for the dimension based on the popular fourth order blind identification (FOBI) method.

math.ST↗

arXiv2016

REPPlab: An R package for detecting clusters and outliers using exploratory projection pursuit

The R-package REPPlab is designed to explore multivariate data sets using one-dimensional unsupervised projection pursuit. It is useful in practice as a preprocessing step to find clusters or as an outlier detection tool for multivariate numerical data. Except from the package tourr that implements smooth sequences of projection matrices and rggobi that provides an interface to a dynamic graphics package called GGobi, there is no implementation of exploratory projection pursuit tools available in R especially in the context of outlier detection. REPPlab is an R interface for the Java program EPPlab that implements four projection indices and three biologically inspired optimization algorithms. The implemented indices are either adapted to cluster or to outlier detection and the optimization algorithms have at most one parameter to tune. Following the original software EPPlab, the exploration strategy in REPPlab is divided into two steps. Many potentially interesting projections are calculated at the first step and examined at the second step. For this second step, different tools for plotting and combining the results are proposed with specific tools for outlier detection. Compared to EPPlab, some of these tools are new and their performance is illustrated through some simulations and using some real data sets in a clustering context. The functionalities of the package are also illustrated for outlier detection on a new data set that is provided with the package.

stat.CO↗

arXiv2016

Projection Pursuit for non-Gaussian Independent Components

In independent component analysis it is assumed that the observed random variables are linear combinations of latent, mutually independent random variables called the independent components. Our model further assumes that only the non-Gaussian independent components are of interest, the Gaussian components being treated as noise. In this paper projection pursuit is used to extract the non-Gaussian components and to separate the corresponding signal and noise subspaces. Our choice for the projection index is a convex combination of squared third and fourth cumulants and we estimate the non-Gaussian components either one-by-one (deflation-based approach) or simultaneously (symmetric approach). The properties of both estimates are considered in detail through the corresponding optimization problems, estimating equations, algorithms and asymptotic properties. Various comparisons of the estimates show that the two approaches separate the signal and noise subspaces equally well but the symmetric one is generally better in extracting the individual non-Gaussian components.

math.ST↗

arXiv2016

Computing the Oja Median in R: The Package OjaNP

The Oja median is one of several extensions of the univariate median to the multivariate case. It has many nice properties, but is computationally demanding. In this paper, we first review the properties of the Oja median and compare it to other multivariate medians. Afterwards we discuss four algorithms to compute the Oja median, which are implemented in our R-package OjaNP. Besides these algorithms, the package contains also functions to compute Oja signs, Oja signed ranks, Oja ranks, and the related scatter concepts. To illustrate their use, the corresponding multivariate one- and $C$-sample location tests are implemented.

stat.CO↗

arXiv2016

JADE for Tensor-Valued Observations

Independent component analysis is a standard tool in modern data analysis and numerous different techniques for applying it exist. The standard methods however quickly lose their effectiveness when the data are made up of structures of higher order than vectors, namely matrices or tensors (for example, images or videos), being unable to handle the high amounts of noise. Recently, an extension of the classic fourth order blind identification (FOBI) specifically suited for tensor-valued observations was proposed and showed to outperform its vector version for tensor data. In this paper we extend another popular independent component analysis method, the joint approximate diagonalization of eigen-matrices (JADE), for tensor observations. In addition to the theoretical background we also provide the asymptotic properties of the proposed estimator and use both simulations and real data to show its usefulness and superiority over its competitors.

math.ST↗

arXiv2015

The squared symmetric FastICA estimator

In this paper we study the theoretical properties of the deflation-based FastICA method, the original symmetric FastICA method, and a modified symmetric FastICA method, here called the squared symmetric FastICA. This modification is obtained by replacing the absolute values in the FastICA objective function by their squares. In the deflation-based case this replacement has no effect on the estimate since the maximization problem stays the same. However, in the symmetric case a novel estimate with unknown properties is obtained. In the paper we review the classic deflation-based and symmetric FastICA approaches and contrast these with the new squared symmetric version of FastICA. We find the estimating equations and derive the asymptotical properties of the squared symmetric FastICA estimator with an arbitrary choice of nonlinearity. Asymptotic variances of the unmixing matrix estimates are then used to compare their efficiencies for large sample sizes showing that the squared symmetric FastICA estimator outperforms the other two estimators in a wide variety of situations.

math.ST↗

arXiv2015

New Algorithms for $M$-Estimation of Multivariate Scatter and Location

We present new algorithms for $M$-estimators of multivariate scatter and location and for symmetrized $M$-estimators of multivariate scatter. The new algorithms are considerably faster than currently used fixed-point and related algorithms. The main idea is to utilize a second order Taylor expansion of the target functional and to devise a partial Newton-Raphson procedure. In connection with symmetrized $M$-estimators we work with incomplete $U$-statistics to accelerate our procedures initially.

stat.CO↗

arXiv2015

Fourth Moments and Independent Component Analysis

In independent component analysis it is assumed that the components of the observed random vector are linear combinations of latent independent random variables, and the aim is then to find an estimate for a transformation matrix back to these independent components. In the engineering literature, there are several traditional estimation procedures based on the use of fourth moments, such as FOBI (fourth order blind identification), JADE (joint approximate diagonalization of eigenmatrices), and FastICA, but the statistical properties of these estimates are not well known. In this paper various independent component functionals based on the fourth moments are discussed in detail, starting with the corresponding optimization problems, deriving the estimating equations and estimation algorithms, and finding asymptotic statistical properties of the estimates. Comparisons of the asymptotic variances of the estimates in wide independent component models show that in most cases JADE and the symmetric version of FastICA perform better than their competitors.

math.ST↗

arXiv2015

Joint Use of Third and Fourth Cumulants in Independent Component Analysis

The independent component model is a latent variable model where the components of the observed random vector are linear combinations of latent independent variables. The aim is to find an estimate for a transformation matrix back to independent components. In moment-based approaches third cumulants are often neglected in favor of fourth cumulants, even though both approaches have similar appealing properties. This paper considers the joint use of third and fourth cumulants in finding independent components. First, univariate cumulants are used as projection indices in search for independent components (projection pursuit). Second, multivariate cumulant matrices are jointly used to solve the problem. The properties of the estimates are considered in detail through corresponding optimization problems, estimating equations, algorithms and asymptotic statistical properties. Comparisons of the asymptotic variances of different estimates in wide independent component models show that in most cases symmetric projection pursuit approach using both third and fourth squared cumulants is a safe choice.

math.ST↗

arXiv2014

Separation of uncorrelated stationary time series using autocovariance matrices

Blind source separation (BSS) is a signal processing tool, which is widely used in various fields. Examples include biomedical signal separation, brain imaging and economic time series applications. In BSS, one assumes that the observed $p$ time series are linear combinations of $p$ latent uncorrelated weakly stationary time series. The aim is then to find an estimate for an unmixing matrix, which transforms the observed time series back to uncorrelated latent time series. In SOBI (Second Order Blind Identification) joint diagonalization of the covariance matrix and autocovariance matrices with several lags is used to estimate the unmixing matrix. The rows of an unmixing matrix can be derived either one by one (deflation-based approach) or simultaneously (symmetric approach). The latter of these approaches is well-known especially in signal processing literature, however, the rigorous analysis of its statistical properties has been missing so far. In this paper, we fill this gap and investigate the statistical properties of the symmetric SOBI estimate in detail and find its limiting distribution under general conditions. The asymptotical efficiencies of symmetric SOBI estimate are compared to those of recently introduced deflation-based SOBI estimate under general multivariate MA$(\infty)$ processes. The theory is illustrated by some finite-sample simulation studies as well as a real EEG data example.

math.ST↗

arXiv2014

A cautionary note on robust covariance plug-in methods

Many multivariate statistical methods rely heavily on the sample covariance matrix. It is well known though that the sample covariance matrix is highly non-robust. One popular alternative approach for "robustifying" the multivariate method is to simply replace the role of the covariance matrix with some robust scatter matrix. The aim of this paper is to point out that in some situations certain properties of the covariance matrix are needed for the corresponding robust "plug-in" method to be a valid approach, and that not all scatter matrices necessarily possess these important properties. In particular, the following three multivariate methods are discussed in this paper: independent components analysis, observational regression and graphical modeling. For each case, it is shown that using a symmetrized robust scatter matrix in place of the covariance matrix results in a proper robust multivariate method.

stat.ME↗

arXiv2012

On asymptotics of ICA estimators and their performance indices

Independent component analysis (ICA) has become a popular multivariate analysis and signal processing technique with diverse applications. This paper is targeted at discussing theoretical large sample properties of ICA unmixing matrix functionals. We provide a formal definition of unmixing matrix functional and consider two popular estimators in detail: the family based on two scatter matrices with the independence property (e.g., FOBI estimator) and the family of deflation-based fastICA estimators. The limiting behavior of the corresponding estimates is discussed and the asymptotic normality of the deflation-based fastICA estimate is proven under general assumptions. Furthermore, properties of several performance indices commonly used for comparison of different unmixing matrix estimates are discussed and a new performance index is proposed. The proposed index fullfills three desirable features which promote its use in practice and distinguish it from others. Namely, the index possesses an easy interpretation, is fast to compute and its asymptotic properties can be inferred from asymptotics of the unmixing matrix estimate. We illustrate the derived asymptotical results and the use of the proposed index with a small simulation study.

stat.ME↗