Stan Lipovetsky | Searcharxiv

arXiv2014

Dimension reduction for data of unknown cluster structure

For numerous reasons there raises a need for dimension reduction that preserves certain characteristics of data. In this work we focus on data coming from a mixture of Gaussian distributions and we propose a method that preserves distinctness of clustering structure, although the structure is assumed to be yet unknown. The rationale behind the method is the following: (i) had one known the clusters (classes) within the data, one could facilitate further analysis and reduce space dimension by projecting the data to the Fisher's linear subspace, which -- by definition -- preserves the structure of the given classes best (ii) under some reasonable assumptions, this can be done, albeit approximately, without the prior knowledge of the clusters (classes). In the paper, we show how this approach works. We present a method of preliminary data transformation that brings the directions of largest overall variability close to the directions of the best between-class separation. Hence, for the transformed data, simple PCA provides an approximation to the Fisher's subspace. We show that the transformation preserves distinctness of unknown structure in the data to a great extent.

math.ST↗

arXiv2014

Tractable Measure of Component Overlap for Gaussian Mixture Models

The ability to quantify distinctness of a cluster structure is fundamental for certain simulation studies, in particular for those comparing performance of different classification algorithms. The intrinsic integral measure based on the overlap of corresponding mixture components is often analytically intractable. This is also the case for Gaussian mixture models with unequal covariance matrices when space dimension $d > 1$. In this work we focus on Gaussian mixture models and at the sample level we assume the class assignments to be known. We derive a measure of component overlap based on eigenvalues of a generalized eigenproblem that represents Fisher's discriminant task. We explain rationale behind it and present simulation results that show how well it can reflect the behavior of the integral measure in its linear approximation. The analyzed coefficient possesses the advantage of being analytically tractable and numerically computable even in complex setups.

math.ST↗