arXiv subjects

Paul D. McNicholas

Publications and source records attributed to Paul D. McNicholas.

88 records · Page 5Linked to original sources

arXiv2014

Model Based Clustering of High-Dimensional Binary Data

We propose a mixture of latent trait models with common slope parameters (MCLT) for model-based clustering of high-dimensional binary data, a data type for which few established methods exist. Recent work on clustering of binary data, based on a $d$-dimensional Gaussian latent variable, is extended by incorporating common factor analyzers. Accordingly, our approach facilitates a low-dimensional visual representation of the clusters. We extend the model further by the incorporation of random block effects. The dependencies in each block are taken into account through block-specific parameters that are considered to be random variables. A variational approximation to the likelihood is exploited to derive a fast algorithm for determining the model parameters. Our approach is demonstrated on real and simulated data.

stat.ME↗

arXiv2013

Accelerated Failure Time Models for Competing Risks in a Cluster Weighted Modelling Framework

A novel approach for dealing with censored competing risks regression data is proposed. This is implemented by a mixture of accelerated failure time (AFT) models for a competing risks scenario within a cluster-weighted modelling (CWM) framework. Specifically, we make use of the log-normal AFT model here but any commonly used AFT model can be utilized. The alternating expectation conditional maximization algorithm (AECM) is used for parameter estimation and bootstrapping for standard error estimation. Finally, we present our results on some simulated and real competing risks data.

stat.ME↗

arXiv2013

Families of Parsimonious Finite Mixtures of Regression Models

Finite mixtures of regression models offer a flexible framework for investigating heterogeneity in data with functional dependencies. These models can be conveniently used for unsupervised learning on data with clear regression relationships. We extend such models by imposing an eigen-decomposition on the multivariate error covariance matrix. By constraining parts of this decomposition, we obtain families of parsimonious mixtures of regressions and mixtures of regressions with concomitant variables. These families of models account for correlations between multiple responses. An expectation-maximization algorithm is presented for parameter estimation and performance is illustrated on simulated and real data.

stat.ME↗

arXiv2013

Parsimonious Shifted Asymmetric Laplace Mixtures

A family of parsimonious shifted asymmetric Laplace mixture models is introduced. We extend the mixture of factor analyzers model to the shifted asymmetric Laplace distribution. Imposing constraints on the constitute parts of the resulting decomposed component scale matrices leads to a family of parsimonious models. An explicit two-stage parameter estimation procedure is described, and the Bayesian information criterion and the integrated completed likelihood are compared for model selection. This novel family of models is applied to real data, where it is compared to its Gaussian analogue within clustering and classification paradigms.

stat.ME↗

arXiv2013

Variational Bayes Approximations for Clustering via Mixtures of Normal Inverse Gaussian Distributions

Parameter estimation for model-based clustering using a finite mixture of normal inverse Gaussian (NIG) distributions is achieved through variational Bayes approximations. Univariate NIG mixtures and multivariate NIG mixtures are considered. The use of variational Bayes approximations here is a substantial departure from the traditional EM approach and alleviates some of the associated computational complexities and uncertainties. Our variational algorithm is applied to simulated and real data. The paper concludes with discussion and suggestions for future work.

stat.ME↗

arXiv2013

Mixtures of Common Skew-t Factor Analyzers

A mixture of common skew-t factor analyzers model is introduced for model-based clustering of high-dimensional data. By assuming common component factor loadings, this model allows clustering to be performed in the presence of a large number of mixture components or when the number of dimensions is too large to be well-modelled by the mixtures of factor analyzers model or a variant thereof. Furthermore, assuming that the component densities follow a skew-t distribution allows robust clustering of skewed data. The alternating expectation-conditional maximization algorithm is employed for parameter estimation. We demonstrate excellent clustering performance when our model is applied to real and simulated data.This paper marks the first time that skewed common factors have been used.

stat.ME↗

arXiv2013

Standardizing Interestingness Measures for Association Rules

Interestingness measures provide information that can be used to prune or select association rules. A given value of an interestingness measure is often interpreted relative to the overall range of the values that the interestingness measure can take. However, properties of individual association rules restrict the values an interestingness measure can achieve. An interesting measure can be standardized to take this into account, but this has only been done for one interestingness measure to date, i.e., the lift. Standardization provides greater insight than the raw value and may even alter researchers' perception of the data. We derive standardized analogues of three interestingness measures and use real and simulated data to compare them to their raw versions, each other, and the standardized lift.

stat.AP↗

arXiv2013

Mixtures of Skew-t Factor Analyzers

In this paper, we introduce a mixture of skew-t factor analyzers as well as a family of mixture models based thereon. The mixture of skew-t distributions model that we use arises as a limiting case of the mixture of generalized hyperbolic distributions. Like their Gaussian and t-distribution analogues, our mixture of skew-t factor analyzers are very well-suited to the model-based clustering of high-dimensional data. Imposing constraints on components of the decomposed covariance parameter results in the development of eight flexible models. The alternating expectation-conditional maximization algorithm is used for model parameter estimation and the Bayesian information criterion is used for model selection. The models are applied to both real and simulated data, giving superior clustering results compared to a well-established family of Gaussian mixture models.

stat.ME↗

arXiv2013

Variable Selection for Clustering and Classification

As data sets continue to grow in size and complexity, effective and efficient techniques are needed to target important features in the variable space. Many of the variable selection techniques that are commonly used alongside clustering algorithms are based upon determining the best variable subspace according to model fitting in a stepwise manner. These techniques are often computationally intensive and can require extended periods of time to run; in fact, some are prohibitively computationally expensive for high-dimensional data. In this paper, a novel variable selection technique is introduced for use in clustering and classification analyses that is both intuitive and computationally efficient. We focus largely on applications in mixture model-based learning, but the technique could be adapted for use with various other clustering/classification methods. Our approach is illustrated on both simulated and real data, highlighted by contrasting its performance with that of other comparable variable selection techniques on the real data sets.

stat.CO↗

arXiv2013

Capturing Patterns via Parsimonious t Mixture Models

This paper exploits a simplified version of the mixture of multivariate t-factor analyzers (MtFA) for robust mixture modelling and clustering of high-dimensional data that frequently contain a number of outliers. Two classes of eight parsimonious t mixture models are introduced and computation of maximum likelihood estimates of parameters is achieved using the alternating expectation conditional maximization (AECM) algorithm. The usefulness of the methodology is illustrated through applications of image compression and compact facial representation.

stat.ME↗

arXiv2013

A Partial EM Algorithm for Clustering White Breads

The design of new products for consumer markets has undergone a major transformation over the last 50 years. Traditionally, inventors would create a new product that they thought might address a perceived need of consumers. Such products tended to be developed to meet the inventors own perception and not necessarily that of consumers. The social consequence of a top-down approach to product development has been a large failure rate in new product introduction. By surveying potential customers, a refined target is created that guides developers and reduces the failure rate. Today, however, the proliferation of products and the emergence of consumer choice has resulted in the identification of segments within the market. Understanding your target market typically involves conducting a product category assessment, where 12 to 30 commercial products are tested with consumers to create a preference map. Every consumer gets to test every product in a complete-block design; however, many classes of products do not lend themselves to such approaches because only a few samples can be evaluated before `fatigue' sets in. We consider an analysis of incomplete balanced-incomplete-block data on 12 different types of white bread. A latent Gaussian mixture model is used for this analysis, with a partial expectation-maximization (PEM) algorithm developed for parameter estimation. This PEM algorithm circumvents the need for a traditional E-step, by performing a partial E-step that reduces the Kullback-Leibler divergence between the conditional distribution of the missing data and the distribution of the missing data given the observed data. The results of the white bread analysis are discussed and some mathematical details are given in an appendix.

stat.AP↗

arXiv2013

Parsimonious Skew Mixture Models for Model-Based Clustering and Classification

In recent work, robust mixture modelling approaches using skewed distributions have been explored to accommodate asymmetric data. We introduce parsimony by developing skew-t and skew-normal analogues of the popular GPCM family that employ an eigenvalue decomposition of a positive-semidefinite matrix. The methods developed in this paper are compared to existing models in both an unsupervised and semi-supervised classification framework. Parameter estimation is carried out using the expectation-maximization algorithm and models are selected using the Bayesian information criterion. The efficacy of these extensions is illustrated on simulated and benchmark clustering data sets.

stat.ME↗

arXiv2013

Estimating Common Principal Components in High Dimensions

We consider the problem of minimizing an objective function that depends on an orthonormal matrix. This situation is encountered when looking for common principal components, for example, and the Flury method is a popular approach. However, the Flury method is not effective for higher dimensional problems. We obtain several simple majorization-minizmation (MM) algorithms that provide solutions to this problem and are effective in higher dimensions. We then use simulated data to compare them with other approaches in terms of convergence and computational time.

stat.CO↗

arXiv2012

Mixtures of Shifted Asymmetric Laplace Distributions

A mixture of shifted asymmetric Laplace distributions is introduced and used for clustering and classification. A variant of the EM algorithm is developed for parameter estimation by exploiting the relationship with the general inverse Gaussian distribution. This approach is mathematically elegant and relatively computationally straightforward. Our novel mixture modelling approach is demonstrated on both simulated and real data to illustrate clustering and classification applications. In these analyses, our mixture of shifted asymmetric Laplace distributions performs favourably when compared to the popular Gaussian approach. This work, which marks an important step in the non-Gaussian model-based clustering and classification direction, concludes with discussion as well as suggestions for future work.

stat.ME↗

arXiv2012

A LASSO-Penalized BIC for Mixture Model Selection

The efficacy of family-based approaches to mixture model-based clustering and classification depends on the selection of parsimonious models. Current wisdom suggests the Bayesian information criterion (BIC) for mixture model selection. However, the BIC has well-known limitations, including a tendency to overestimate the number of components as well as a proclivity for, often drastically, underestimating the number of components in higher dimensions. While the former problem might be soluble through merging components, the latter is impossible to mitigate in clustering and classification applications. In this paper, a LASSO-penalized BIC (LPBIC) is introduced to overcome this problem. This approach is illustrated based on applications of extensions of mixtures of factor analyzers, where the LPBIC is used to select both the number of components and the number of latent factors. The LPBIC is shown to match or outperform the BIC in several situations.

stat.ME↗

arXiv2012

Clustering and Classification via Cluster-Weighted Factor Analyzers

In model-based clustering and classification, the cluster-weighted model constitutes a convenient approach when the random vector of interest constitutes a response variable Y and a set p of explanatory variables X. However, its applicability may be limited when p is high. To overcome this problem, this paper assumes a latent factor structure for X in each mixture component. This leads to the cluster-weighted factor analyzers (CWFA) model. By imposing constraints on the variance of Y and the covariance matrix of X, a novel family of sixteen CWFA models is introduced for model-based clustering and classification. The alternating expectation-conditional maximization algorithm, for maximum likelihood estimation of the parameters of all the models in the family, is described; to initialize the algorithm, a 5-step hierarchical procedure is proposed, which uses the nested structures of the models within the family and thus guarantees the natural ranking among the sixteen likelihoods. Artificial and real data show that these models have very good clustering and classification performance and that the algorithm is able to recover the parameters very well.

stat.ME↗