arXiv subjects

Alfred O. Hero

Publications and source records attributed to Alfred O. Hero.

At least 55 records · Page 3Linked to original sources

arXiv2017

Binary Hypothesis Testing via Measure Transformed Quasi Likelihood Ratio Test

In this paper, the Gaussian quasi likelihood ratio test (GQLRT) for non-Bayesian binary hypothesis testing is generalized by applying a transform to the probability distribution of the data. The proposed generalization, called measure-transformed GQLRT (MT-GQLRT), selects a Gaussian probability model that best empirically fits a transformed probability measure of the data. By judicious choice of the transform we show that, unlike the GQLRT, the proposed test is resilient to outliers and involves higher-order statistical moments leading to significant mitigation of the model mismatch effect on the decision performance. Under some mild regularity conditions we show that the MT-GQLRT is consistent and its corresponding test statistic is asymptotically normal. A data driven procedure for optimal selection of the measure transformation parameters is developed that maximizes an empirical estimate of the asymptotic power given a fixed empirical asymptotic size. A Bayesian extension of the proposed MT-GQLRT is also developed that is based on selection of a Gaussian probability model that best empirically fits a transformed conditional probability distribution of the data. In the Bayesian MT-GQLRT the threshold and the measure transformation parameters are selected via joint minimization of the empirical asymptotic Bayes risk. The non-Bayesian and Bayesian MT-GQLRTs are applied to signal detection and classification, in simulation examples that illustrate their advantages over the standard GQLRT and other robust alternatives.

stat.ME↗

arXiv2017

Bounds on Variance for Unimodal Distributions

We show a direct relationship between the variance and the differential entropy for subclasses of symmetric and asymmetric unimodal distributions by providing an upper bound on variance in terms of entropy power. Combining this bound with the well-known entropy power lower bound on variance, we prove that the variance of the appropriate subclasses of unimodal distributions can be bounded below and above by the scaled entropy power. As differential entropy decreases, the variance is sandwiched between two exponentially decreasing functions in the differential entropy. This establishes that for the subclasses of unimodal distributions, the differential entropy can be used as a surrogate for concentration of the distribution.

cs.IT↗

arXiv2017

Multilayer Spectral Graph Clustering via Convex Layer Aggregation: Theory and Algorithms

Multilayer graphs are commonly used for representing different relations between entities and handling heterogeneous data processing tasks. Non-standard multilayer graph clustering methods are needed for assigning clusters to a common multilayer node set and for combining information from each layer. This paper presents a multilayer spectral graph clustering (SGC) framework that performs convex layer aggregation. Under a multilayer signal plus noise model, we provide a phase transition analysis of clustering reliability. Moreover, we use the phase transition criterion to propose a multilayer iterative model order selection algorithm (MIMOSA) for multilayer SGC, which features automated cluster assignment and layer weight adaptation, and provides statistical clustering reliability guarantees. Numerical simulations on synthetic multilayer graphs verify the phase transition analysis, and experiments on real-world multilayer graphs show that MIMOSA is competitive or better than other clustering methods.

stat.ML↗

arXiv2016

Robust training on approximated minimal-entropy set

In this paper, we propose a general framework to learn a robust large-margin binary classifier when corrupt measurements, called anomalies, caused by sensor failure might be present in the training set. The goal is to minimize the generalization error of the classifier on non-corrupted measurements while controlling the false alarm rate associated with anomalous samples. By incorporating a non-parametric regularizer based on an empirical entropy estimator, we propose a Geometric-Entropy-Minimization regularized Maximum Entropy Discrimination (GEM-MED) method to learn to classify and detect anomalies in a joint manner. We demonstrate using simulated data and a real multimodal data set. Our GEM-MED method can yield improved performance over previous robust classification methods in terms of both classification accuracy and anomaly detection rate.

cs.LG↗

arXiv2016

Measure-Transformed Quasi Maximum Likelihood Estimation

In this paper the Gaussian quasi maximum likelihood estimator (GQMLE) is generalized by applying a transform to the probability distribution of the data. The proposed estimator, called measure-transformed GQMLE (MT-GQMLE), minimizes the empirical Kullback-Leibler divergence between a transformed probability distribution of the data and a hypothesized Gaussian probability measure. By judicious choice of the transform we show that, unlike the GQMLE, the proposed estimator can gain sensitivity to higher-order statistical moments and resilience to outliers leading to significant mitigation of the model mismatch effect on the estimates. Under some mild regularity conditions we show that the MT-GQMLE is consistent, asymptotically normal and unbiased. Furthermore, we derive a necessary and sufficient condition for asymptotic efficiency. A data driven procedure for optimal selection of the measure transformation parameters is developed that minimizes the trace of an empirical estimate of the asymptotic mean-squared-error matrix. The MT-GQMLE is applied to linear regression and source localization and numerical comparisons illustrate its robustness and resilience to outliers.

stat.ME↗

arXiv2016

Incremental Method for Spectral Clustering of Increasing Orders

The smallest eigenvalues and the associated eigenvectors (i.e., eigenpairs) of a graph Laplacian matrix have been widely used for spectral clustering and community detection. However, in real-life applications the number of clusters or communities (say, $K$) is generally unknown a-priori. Consequently, the majority of the existing methods either choose $K$ heuristically or they repeat the clustering method with different choices of $K$ and accept the best clustering result. The first option, more often, yields suboptimal result, while the second option is computationally expensive. In this work, we propose an incremental method for constructing the eigenspectrum of the graph Laplacian matrix. This method leverages the eigenstructure of graph Laplacian matrix to obtain the $K$-th eigenpairs of the Laplacian matrix given a collection of all the $K-1$ smallest eigenpairs. Our proposed method adapts the Laplacian matrix such that the batch eigenvalue decomposition problem transforms into an efficient sequential leading eigenpair computation problem. As a practical application, we consider user-guided spectral clustering. Specifically, we demonstrate that users can utilize the proposed incremental method for effective eigenpair computation and determining the desired number of clusters based on multiple clustering metrics.

cs.SI↗

arXiv2016

Diversion Detection in Partially Observed Nuclear Fuel Cycle Networks

A nuclear fuel cycle contains several facilities with different purposes such as mining, conversion, enrichment, and fuel rod fabrication. These facilities form a network, which is naturally sparse in the number of connections (i.e., edges) since not every facility directly interacts with all the others. Given the knowledge of a network baseline, we are interested in detecting anomalous activities in this network, which may signal the diversion of nuclear materials. Anomalies can take the form of a new or missing edge or abnormal rates of interaction. However, often it is not possible to observe the entire network traffic directly due to some constraints such as cost, physical limitations, or laws. By treating the unobserved network traffic as latent variables, we propose estimators for the true network traffic, including the anomalous activity, to use in testing for significant deviations from the baseline. We provide simulation results of a simple network of facilities and show that our estimators have superior performance over existing alternatives. Additionally, we establish that while a good estimate of the network traffic is necessary, perfect reconstruction is not required to effectively detect anomalous network activity. Instead it suffices to detect perturbations within the network at an aggregate or global scale.

stat.AP↗

arXiv2016

Online Diversion Detection in Nuclear Fuel Cycles via Multimodal Observations

In nuclear fuel cycles, an enrichment facility typically provides low enriched uranium (LEU) to a number of customers. We consider monitoring an enrichment facility to timely detect a possible diversion of highly enriched uranium (HEU). To increase the the detection accuracy it is important to efficiently use the available information diversity. In this work, it is assumed that the shipment times and the power consumption of the enrichment facility are observed for each shipment of enriched uranium. We propose to initially learn the statistical patterns of the enrichment facility through the bimodal observations in a training period, that is known to be free of diversions. Then, for the goal of timely diversion detection, we propose to use an online detection algorithm which sequentially compares each set of new observations in the test period, which possibly includes diversions, to the learned patterns, and raises a diversion alarm when a significant statistical deviation is detected. The efficacy of the proposed method is shown by comparing its detection performance to those of the traditional detection methods in the Statistics literature.

stat.AP↗

arXiv2016

Multi-centrality Graph Spectral Decompositions and their Application to Cyber Intrusion Detection

Many modern datasets can be represented as graphs and hence spectral decompositions such as graph principal component analysis (PCA) can be useful. Distinct from previous graph decomposition approaches based on subspace projection of a single topological feature, e.g., the Fiedler vector of centered graph adjacency matrix (graph Laplacian), we propose spectral decomposition approaches to graph PCA and graph dictionary learning that integrate multiple features, including graph walk statistics, centrality measures and graph distances to reference nodes. In this paper we propose a new PCA method for single graph analysis, called multi-centrality graph PCA (MC-GPCA), and a new dictionary learning method for ensembles of graphs, called multi-centrality graph dictionary learning (MC-GDL), both based on spectral decomposition of multi-centrality matrices. As an application to cyber intrusion detection, MC-GPCA can be an effective indicator of anomalous connectivity pattern and MC-GDL can provide discriminative basis for attack classification.

cs.SI↗

arXiv2016

Learning to classify with possible sensor failures

cs.LG↗

arXiv2015

Robust Multiple Signal Classification via Probability Measure Transformation

In this paper, we introduce a new framework for robust multiple signal classification (MUSIC). The proposed framework, called robust measure-transformed (MT) MUSIC, is based on applying a transform to the probability distribution of the received signals, i.e., transformation of the probability measure defined on the observation space. In robust MT-MUSIC, the sample covariance is replaced by the empirical MT-covariance. By judicious choice of the transform we show that: 1) the resulting empirical MT-covariance is B-robust, with bounded influence function that takes negligible values for large norm outliers, and 2) under the assumption of spherically contoured noise distribution, the noise subspace can be determined from the eigendecomposition of the MT-covariance. Furthermore, we derive a new robust measure-transformed minimum description length (MDL) criterion for estimating the number of signals, and extend the MT-MUSIC framework to the case of coherent signals. The proposed approach is illustrated in simulation examples that show its advantages as compared to other robust MUSIC and MDL generalizations.

stat.ME↗

arXiv2015

Multimodal Factor Analysis

A multimodal system with Poisson, Gaussian, and multinomial observations is considered. A generative graphical model that combines multiple modalities through common factor loadings is proposed. In this model, latent factors are like summary objects that has latent factor scores in each modality, and the observed objects are represented in terms of such summary objects. This potentially brings about a significant dimensionality reduction. It also naturally enables a powerful means of clustering based on a diverse set of observations. An expectation-maximization (EM) algorithm to find the model parameters is provided. The algorithm is tested on a Twitter dataset which consists of the counts and geographical coordinates of hashtag occurrences, together with the bag of words for each hashtag. The resultant factors successfully localizes the hashtags in all dimensions: counts, coordinates, topics. The algorithm is also extended to accommodate von Mises-Fisher distribution, which is used to model the spherical coordinates.

stat.AP↗

arXiv2015

Deep Community Detection

A deep community in a graph is a connected component that can only be seen after removal of nodes or edges from the rest of the graph. This paper formulates the problem of detecting deep communities as multi-stage node removal that maximizes a new centrality measure, called the local Fiedler vector centrality (LFVC), at each stage. The LFVC is associated with the sensitivity of algebraic connectivity to node or edge removals. We prove that a greedy node/edge removal strategy, based on successive maximization of LFVC, has bounded performance loss relative to the optimal, but intractable, combinatorial batch removal strategy. Under a stochastic block model framework, we show that the greedy LFVC strategy can extract deep communities with probability one as the number of observations becomes large. We apply the greedy LFVC strategy to real-world social network datasets. Compared with conventional community detection methods we demonstrate improved ability to identify important communities and key members in the network.

cs.SI↗

arXiv2015

Foundational principles for large scale inference: Illustrations through correlation mining

When can reliable inference be drawn in the "Big Data" context? This paper presents a framework for answering this fundamental question in the context of correlation mining, with implications for general large scale inference. In large scale data applications like genomics, connectomics, and eco-informatics the dataset is often variable-rich but sample-starved: a regime where the number $n$ of acquired samples (statistical replicates) is far fewer than the number $p$ of observed variables (genes, neurons, voxels, or chemical constituents). Much of recent work has focused on understanding the computational complexity of proposed methods for "Big Data." Sample complexity however has received relatively less attention, especially in the setting when the sample size $n$ is fixed, and the dimension $p$ grows without bound. To address this gap, we develop a unified statistical framework that explicitly quantifies the sample complexity of various inferential tasks. Sampling regimes can be divided into several categories: 1) the classical asymptotic regime where the variable dimension is fixed and the sample size goes to infinity; 2) the mixed asymptotic regime where both variable dimension and sample size go to infinity at comparable rates; 3) the purely high dimensional asymptotic regime where the variable dimension goes to infinity and the sample size is fixed. Each regime has its niche but only the latter regime applies to exa-scale data dimension. We illustrate this high dimensional framework for the problem of correlation mining, where it is the matrix of pairwise and partial correlations among the variables that are of interest. We demonstrate various regimes of correlation mining based on the unifying perspective of high dimensional learning rates and sample complexity for different structured covariance models and different inference tasks.

math.ST↗

arXiv2015

A Dictionary Approach to EBSD Indexing

We propose a framework for indexing of grain and sub-grain structures in electron backscatter diffraction (EBSD) images of polycrystalline materials. The framework is based on a previously introduced physics-based forward model by Callahan and De Graef (2013) relating measured patterns to grain orientations (Euler angle). The forward model is tuned to the microscope and the sample symmetry group. We discretize the domain of the forward model onto a dense grid of Euler angles and for each measured pattern we identify the most similar patterns in the dictionary. These patterns are used to identify boundaries, detect anomalies, and index crystal orientations. The statistical distribution of these closest matches is used in an unsupervised binary decision tree (DT) classifier to identify grain boundaries and anomalous regions. The DT classifies a pattern as an anomaly if it has an abnormally low similarity to any pattern in the dictionary. It classifies a pixel as being near a grain boundary if the highly ranked patterns in the dictionary differ significantly over the pixels 3x3 neighborhood. Indexing is accomplished by computing the mean orientation of the closest dictionary matches to each pattern. The mean orientation is estimated using a maximum likelihood approach that models the orientation distribution as a mixture of Von Mises-Fisher distributions over the quaternionic 3-sphere. The proposed dictionary matching approach permits segmentation, anomaly detection, and indexing to be performed in a unified manner with the additional benefit of uncertainty quantification. We demonstrate the proposed dictionary-based approach on a Ni-base IN100 alloy.

cs.CV↗

arXiv2015

Empirically Estimable Classification Bounds Based on a New Divergence Measure

Information divergence functions play a critical role in statistics and information theory. In this paper we show that a non-parametric f-divergence measure can be used to provide improved bounds on the minimum binary classification probability of error for the case when the training and test data are drawn from the same distribution and for the case where there exists some mismatch between training and test distributions. We confirm the theoretical results by designing feature selection algorithms using the criteria from these bounds and by evaluating the algorithms on a series of pathological speech classification tasks.

cs.IT↗

arXiv2014

Empirical non-parametric estimation of the Fisher Information

The Fisher information matrix (FIM) is a foundational concept in statistical signal processing. The FIM depends on the probability distribution, assumed to belong to a smooth parametric family. Traditional approaches to estimating the FIM require estimating the probability distribution function (PDF), or its parameters, along with its gradient or Hessian. However, in many practical situations the PDF of the data is not known but the statistician has access to an observation sample for any parameter value. Here we propose a method of estimating the FIM directly from sampled data that does not require knowledge of the underlying PDF. The method is based on non-parametric estimation of an $f$-divergence over a local neighborhood of the parameter space and a relation between curvature of the $f$-divergence and the FIM. Thus we obtain an empirical estimator of the FIM that does not require density estimation and is asymptotically consistent. We empirically evaluate the validity of our approach using two experiments.

stat.CO↗

arXiv2014

Jointly Sparse Global SIMPLS Regression

Partial least squares (PLS) regression combines dimensionality reduction and prediction using a latent variable model. Since partial least squares regression (PLS-R) does not require matrix inversion or diagonalization, it can be applied to problems with large numbers of variables. As predictor dimension increases, variable selection becomes essential to avoid over-fitting, to provide more accurate predictors and to yield more interpretable parameters. We propose a global variable selection approach that penalizes the total number of variables across all PLS components. Put another way, the proposed global penalty encourages the selected variables to be shared among the PLS components. We formulate PLS-R with joint sparsity as a variational optimization problem with objective function equal to a novel global SIMPLS criterion plus a mixed norm sparsity penalty on the weight matrix. The mixed norm sparsity penalty is the $\ell_1$ norm of the $\ell_2$ norm on the weights corresponding to the same variable used over all the PLS components. A novel augmented Lagrangian method is proposed to solve the optimization problem and soft thresholding for sparsity occurs naturally as part of the iterative solution. Experiments show that the modified PLS-R attains better or as good performance with many fewer selected predictor variables.

stat.ME↗