Searcharxiv⌕ Search

arXiv subjects

Bala Rajaratnam

Publications and source records attributed to Bala Rajaratnam.

At least 55 records · Page 3Linked to original sources
arXiv2014

G-AMA: Sparse Gaussian graphical model estimation via alternating minimization

Several methods have been recently proposed for estimating sparse Gaussian graphical models using $\ell_{1}$ regularization on the inverse covariance matrix. Despite recent advances, contemporary applications require methods that are even faster in order to handle ill-conditioned high dimensional modern day datasets. In this paper, we propose a new method, G-AMA, to solve the sparse inverse covariance estimation problem using Alternating Minimization Algorithm (AMA), that effectively works as a proximal gradient algorithm on the dual problem. Our approach has several novel advantages over existing methods. First, we demonstrate that G-AMA is faster than the previous best algorithms by many orders of magnitude and is thus an ideal approach for modern high throughput applications. Second, global linear convergence of G-AMA is demonstrated rigorously, underscoring its good theoretical properties. Third, the dual algorithm operates on the covariance matrix, and thus easily facilitates incorporating additional constraints on pairwise/marginal relationships between feature pairs based on domain specific knowledge. Over and above estimating a sparse inverse covariance matrix, we also illustrate how to (1) incorporate constraints on the (bivariate) correlations and, (2) incorporate equality (equisparsity) or linear constraints between individual inverse covariance elements. Fourth, we also show that G-AMA is better adept at handling extremely ill-conditioned problems, as is often the case with real data. The methodology is demonstrated on both simulated and real datasets to illustrate its superior performance over recently proposed methods.

stat.CO↗
arXiv2014

Loewner positive entrywise functions, and classification of measurable solutions of Cauchy's functional equations

Entrywise functions preserving Loewner positivity have been studied by many authors, most notably Schoenberg and Rudin. Following their work, it is known that functions preserving positivity when applied entrywise to positive semidefinite matrices of all dimensions are necessarily analytic with nonnegative Taylor coefficients. When the dimension is fixed, it has been shown by Vasudeva and Horn that such functions are automatically continuous and sufficiently differentiable. A natural refinement of the aforementioned problem consists of characterizing functions preserving positivity under rank constraints. In this paper, we begin this study by characterizing entrywise functions which preserve the cone of positive semidefinite real matrices of rank $1$ with entries in a general interval. Classifying such functions is intimately connected to the classical problem of solving Cauchy's functional equations, which have non-measurable solutions. We demonstrate that under mild local measurability assumptions, such functions are automatically smooth and can be completely characterized. We then extend our results by classifying functions preserving positivity on rank $1$ Hermitian complex matrices.

math.FA↗
arXiv2014

On fractional Hadamard powers of positive block matrices

Entrywise powers of matrices have been well-studied in the literature, and have recently received renewed attention in the regularization of high-dimensional correlation matrices. In this paper, we study powers of positive semidefinite block matrices $(H_{st})_{s,t=1}^n$ with complex entries. We first characterize the powers $α\in\mathbb{R}$ such that the blockwise power map $(H_{st}) \mapsto (H_{st}^α)$ preserves Loewner positivity. The characterization is obtained by exploiting connections with the theory of matrix monotone functions developed by Loewner. Second, we revisit previous work by Choudhury [Proc. AMS 108] who had provided a lower bound on $α$ for preserving positivity when the blocks $H_{st}$ pairwise commute. We completely settle this problem by characterizing the full set of powers preserving positivity in this setting. Our characterizations generalize previous work by FitzGerald-Horn, Bhatia-Elsner, and Hiai from scalars to arbitrary block size, and in particular, generalize the Schur Product Theorem. Finally, a natural and unifying framework for studying the case of diagonalizable blocks consists of replacing real powers by general characters of the complex plane. We thus classify such characters, and generalize our results to this more general setting. In the course of our work, given $β\in\mathbb{Z}$, we provide lower and upper bounds for the threshold power $α>0$ above which the complex characters $re^{iθ}\mapsto r^αe^{iβθ}$ preserve positivity when applied entrywise to positive semidefinite matrices. In particular, we completely resolve the $n=3$ case of a question raised in 2001 by Xingzhi Zhan. As an application, we extend previous work by de Pillis [Duke Math. J. 36] by classifying the characters $K$ of the complex plane for which the map $(H_{st})_{s,t=1}^n \mapsto (K({\rm tr}(H_{st})))_{s,t=1}^n$ preserves positivity.

math.FA↗
arXiv2013

Successive normalization of rectangular arrays

Standard statistical techniques often require transforming data to have mean $0$ and standard deviation $1$. Typically, this process of "standardization" or "normalization" is applied across subjects when each subject produces a single number. High throughput genomic and financial data often come as rectangular arrays where each coordinate in one direction concerns subjects who might have different status (case or control, say), and each coordinate in the other designates "outcome" for a specific feature, for example, "gene," "polymorphic site" or some aspect of financial profile. It may happen, when analyzing data that arrive as a rectangular array, that one requires BOTH the subjects and the features to be "on the same footing." Thus there may be a need to standardize across rows and columns of the rectangular matrix. There arises the question as to how to achieve this double normalization. We propose and investigate the convergence of what seems to us a natural approach to successive normalization which we learned from our colleague Bradley Efron. We also study the implementation of the method on simulated data and also on data that arose from scientific experimentation.

math.ST↗
arXiv2013

Duality in Graphical Models

Graphical models have proven to be powerful tools for representing high-dimensional systems of random variables. One example of such a model is the undirected graph, in which lack of an edge represents conditional independence between two random variables given the rest. Another example is the bidirected graph, in which absence of edges encodes pairwise marginal independence. Both of these classes of graphical models have been extensively studied, and while they are considered to be dual to one another, except in a few instances this duality has not been thoroughly investigated. In this paper, we demonstrate how duality between undirected and bidirected models can be used to transport results for one class of graphical models to the dual model in a transparent manner. We proceed to apply this technique to extend previously existing results as well as to prove new ones, in three important domains. First, we discuss the pairwise and global Markov properties for undirected and bidirected models, using the pseudographoid and reverse-pseudographoid rules which are weaker conditions than the typically used intersection and composition rules. Second, we investigate these pseudographoid and reverse pseudographoid rules in the context of probability distributions, using the concept of duality in the process. Duality allows us to quickly relate them to the more familiar intersection and composition properties. Third and finally, we apply the dualization method to understand the implications of faithfulness, which in turn leads to a more general form of an existing result.

math.PR↗
arXiv2013

A Methodology for Robust Multiproxy Paleoclimate Reconstructions and Modeling of Temperature Conditional Quantiles

Great strides have been made in the field of reconstructing past temperatures based on models relating temperature to temperature-sensitive paleoclimate proxies. One of the goals of such reconstructions is to assess if current climate is anomalous in a millennial context. These regression based approaches model the conditional mean of the temperature distribution as a function of paleoclimate proxies (or vice versa). Some of the recent focus in the area has considered methods which help reduce the uncertainty inherent in such statistical paleoclimate reconstructions, with the ultimate goal of improving the confidence that can be attached to such endeavors. A second important scientific focus in the subject area is the area of forward models for proxies, the goal of which is to understand the way paleoclimate proxies are driven by temperature and other environmental variables. In this paper we introduce novel statistical methodology for (1) quantile regression with autoregressive residual structure, (2) estimation of corresponding model parameters, (3) development of a rigorous framework for specifying uncertainty estimates of quantities of interest, yielding (4) statistical byproducts that address the two scientific foci discussed above. Our statistical methodology demonstrably produces a more robust reconstruction than is possible by using conditional-mean-fitting methods. Our reconstruction shares some of the common features of past reconstructions, but also gains useful insights. More importantly, we are able to demonstrate a significantly smaller uncertainty than that from previous regression methods. In addition, the quantile regression component allows us to model, in a more complete and flexible way than least squares, the conditional distribution of temperature given proxies. This relationship can be used to inform forward models relating how proxies are driven by temperature.

stat.ME↗
arXiv2013

The critical exponent conjecture for powers of doubly nonnegative matrices

Doubly non-negative matrices arise naturally in many setting including Markov random fields (positively banded graphical models) and in the convergence analysis of Markov chains. In this short note, we settle a recent conjecture by C.R. Johnson et al. [Linear Algebra Appl. 435 (2011)] by proving that the critical exponent beyond which all continuous conventional powers of $n$-by-$n$ doubly nonnegative matrices are doubly nonnegative is exactly $n-2$. We show that the conjecture follows immediately by applying a general characterization from the literature. We prove a stronger form of the conjecture by classifying all powers preserving doubly nonnegative matrices, and proceed to generalize the conjecture for broad classes of functions. We also provide different approaches for settling the original conjecture.

math.CA↗
arXiv2013

Functions preserving positive definiteness for sparse matrices

We consider the problem of characterizing entrywise functions that preserve the cone of positive definite matrices when applied to every off-diagonal element. Our results extend theorems of Schoenberg [Duke Math. J. 9], Rudin [Duke Math. J. 26], Christensen and Ressel [Trans. Amer. Math. Soc., 243], and others, where similar problems were studied when the function is applied to all elements, including the diagonal ones. It is shown that functions that are guaranteed to preserve positive definiteness cannot at the same time induce sparsity, i.e., set elements to zero. These results have important implications for the regularization of positive definite matrices, where functions are often applied to only the off-diagonal elements to obtain sparse matrices with better properties (e.g., Markov random field/graphical model structure, better condition number). As a particular case, it is shown that \emph{soft-thresholding}, a commonly used operation in modern high-dimensional probability and statistics, is not guaranteed to maintain positive definiteness, even if the original matrix is sparse. This result has a deep connection to graphs, and in particular, to the class of trees. We then proceed to fully characterize functions which do preserve positive definiteness. This characterization is in terms of absolutely monotonic functions and turns out to be quite different from the case when the function is also applied to diagonal elements. We conclude by giving bounds on the condition number of a matrix which guarantee that the regularized matrix is positive definite.

math.CA↗
arXiv2013

Predictive Correlation Screening: Application to Two-stage Predictor Design in High Dimension

We introduce a new approach to variable selection, called Predictive Correlation Screening, for predictor design. Predictive Correlation Screening (PCS) implements false positive control on the selected variables, is well suited to small sample sizes, and is scalable to high dimensions. We establish asymptotic bounds for Familywise Error Rate (FWER), and resultant mean square error of a linear predictor on the selected variables. We apply Predictive Correlation Screening to the following two-stage predictor design problem. An experimenter wants to learn a multivariate predictor of gene expressions based on successive biological samples assayed on mRNA arrays. She assays the whole genome on a few samples and from these assays she selects a small number of variables using Predictive Correlation Screening. To reduce assay cost, she subsequently assays only the selected variables on the remaining samples, to learn the predictor coefficients. We show superiority of Predictive Correlation Screening relative to LASSO and correlation learning (sometimes popularly referred to in the literature as marginal regression or simple thresholding) in terms of performance and computational complexity.

stat.ML↗
arXiv2012

Iterative Thresholding Algorithm for Sparse Inverse Covariance Estimation

The L1-regularized maximum likelihood estimation problem has recently become a topic of great interest within the machine learning, statistics, and optimization communities as a method for producing sparse inverse covariance estimators. In this paper, a proximal gradient method (G-ISTA) for performing L1-regularized covariance matrix estimation is presented. Although numerous algorithms have been proposed for solving this problem, this simple proximal gradient method is found to have attractive theoretical and numerical properties. G-ISTA has a linear rate of convergence, resulting in an O(log e) iteration complexity to reach a tolerance of e. This paper gives eigenvalue bounds for the G-ISTA iterates, providing a closed-form linear convergence rate. The rate is shown to be closely related to the condition number of the optimal point. Numerical convergence results and timing comparisons for the proposed method are presented. G-ISTA is shown to perform very well, especially when the optimal point is well-conditioned.

stat.CO↗
arXiv2012

A note on the lack of symmetry in the graphical lasso

The graphical lasso (glasso) is a widely-used fast algorithm for estimating sparse inverse covariance matrices. The glasso solves an L1 penalized maximum likelihood problem and is available as an R library on CRAN. The output from the glasso, a regularized covariance matrix estimate a sparse inverse covariance matrix estimate, not only identify a graphical model but can also serve as intermediate inputs into multivariate procedures such as PCA, LDA, MANOVA, and others. The glasso indeed produces a covariance matrix estimate which solves the L1 penalized optimization problem in a dual sense; however, the method for producing the inverse covariance matrix estimator after this optimization is inexact and may produce asymmetric estimates. This problem is exacerbated when the amount of L1 regularization that is applied is small, which in turn is more likely to occur if the true underlying inverse covariance matrix is not sparse. The lack of symmetry can potentially have consequences. First, it implies that the covariance and inverse covariance estimates are not numerical inverses of one another, and second, asymmetry can possibly lead to negative or complex eigenvalues,rendering many multivariate procedures which may depend on the inverse covariance estimator unusable. We demonstrate this problem, explain its causes, and propose possible remedies.

stat.ML↗
arXiv2012

Positive definite completion problems for directed acyclic graphs

A positive definite completion problem pertains to determining whether the unspecified positions of a partial (or incomplete) matrix can be completed in a desired subclass of positive definite matrices. In this paper we study an important and new class of positive definite completion problems where the desired subclasses are the spaces of covariance and inverse-covariance matrices of probabilistic models corresponding to directed acyclic graph models (also known as Bayesian networks). We provide fast procedures that determine whether a partial matrix can be completed in either of these spaces and thereafter proceed to construct the completed matrices. We prove an analog of the positive definite completion result for undirected graphs in the context of directed acyclic graphs, and thus proceed to characterize the class of DAGs which can always be completed. We also proceed to give closed form expressions for the inverse and the determinant of a completed matrix as a function of only the elements of the corresponding partial matrix.

math.RA↗
arXiv2012

Sparse Matrix Decompositions and Graph Characterizations

The question of when zeros (i.e., sparsity) in a positive definite matrix $A$ are preserved in its Cholesky decomposition, and vice versa, was addressed by Paulsen et al. in the Journal of Functional Analysis (85, pp151-178). In particular, they prove that for the pattern of zeros in $A$ to be retained in the Cholesky decomposition of $A$, the pattern of zeros in $A$ has to necessarily correspond to a chordal (or decomposable) graph associated with a specific type of vertex ordering. This result therefore yields a characterization of chordal graphs in terms of sparse positive definite matrices. It has also proved to be extremely useful in probabilistic and statistical analysis of Markov random fields where zeros in positive definite correlation matrices are intimately related to the notion of stochastic independence. Now, consider a positive definite matrix $A$ and its Cholesky decomposition given by $A = LDL^T$, where $L$ is lower triangular with unit diagonal entries, and $D$ a diagonal matrix with positive entries. In this paper, we prove that a necessary and sufficient condition for zeros (i.e., sparsity) in a positive definite matrix $A$ to be preserved in its associated Cholesky matrix $L$, \, and in addition also preserved in the inverse of the Cholesky matrix $L^{-1}$, is that the pattern of zeros corresponds to a co-chordal or homogeneous graph associated with a specific type of vertex ordering. We proceed to provide a second characterization of this class of graphs in terms of determinants of submatrices that correspond to cliques in the graph. These results add to the growing body of literature in the field of sparse matrix decompositions, and also prove to be critical ingredients in the probabilistic analysis of an important class of Markov random fields.

math.CO↗
arXiv2012

Successive Standardization of Rectangular Arrays

In this note we illustrate and develop further with mathematics and examples, the work on successive standardization (or normalization) that is studied earlier by the same authors in Olshen and Rajaratnam (2010) and Olshen and Rajaratnam (2011). Thus, we deal with successive iterations applied to rectangular arrays of numbers, where to avoid technical difficulties an array has at least three rows and at least three columns. Without loss, an iteration begins with operations on columns: first subtract the mean of each column; then divide by its standard deviation. The iteration continues with the same two operations done successively for rows. These four operations applied in sequence completes one iteration. One then iterates again, and again, and again,.... In Olshen and Rajaratnam (2010) it was argued that if arrays are made up of real numbers, then the set for which convergence of these successive iterations fails has Lebesgue measure 0. The limiting array has row and column means 0, row and column standard deviations 1. A basic result on convergence given in Olshen and Rajaratnam (2010) is true, though the argument in Olshen and Rajaratnam (2010) is faulty. The result is stated in the form of a theorem here, and the argument for the theorem is correct. Moreover, many graphics given in Olshen and Rajaratnam (2010) suggest that but for a set of entries of any array with Lebesgue measure 0, convergence is very rapid, eventually exponentially fast in the number of iterations. Because we learned this set of rules from Bradley Efron, we call it "Efron's algorithm". More importantly, the rapidity of convergence is illustrated by numerical examples.

math.PR↗
arXiv2011

Hub discovery in partial correlation graphical models

This paper treats the problem of screening a p-variate sample for strongly and multiply connected vertices in the partial correlation graph associated with the the partial correlation matrix of the sample. This problem, called hub screening, is important in many applications ranging from network security to computational biology to finance to social networks. In the area of network security, a node that becomes a hub of high correlation with neighboring nodes might signal anomalous activity such as a coordinated flooding attack. In the area of computational biology the set of hubs of a gene expression correlation graph can serve as potential targets for drug treatment to block a pathway or modulate host response. In the area of finance a hub might indicate a vulnerable financial instrument or sector whose collapse might have major repercussions on the market. In the area of social networks a hub of observed interactions between criminal suspects could be an influential ringleader. The techniques and theory presented in this paper permit scalable and reliable screening for such hubs. This paper extends our previous work on correlation screening [arXiv:1102.1204] to the more challenging problem of partial correlation screening for variables with a high degree of connectivity. In particular we consider 1) extension to the more difficult problem of screening for partial correlations exceeding a specified magnitude; 2) extension to screening variables whose vertex degree in the associated partial correlation graph, often called the concentration graph, exceeds a specified degree.

math.ST↗
arXiv2011

Retaining positive definiteness in thresholded matrices

Positive definite (p.d.) matrices arise naturally in many areas within mathematics and also feature extensively in scientific applications. In modern high-dimensional applications, a common approach to finding sparse positive definite matrices is to threshold their small off-diagonal elements. This thresholding, sometimes referred to as hard-thresholding, sets small elements to zero. Thresholding has the attractive property that the resulting matrices are sparse, and are thus easier to interpret and work with. In many applications, it is often required, and thus implicitly assumed, that thresholded matrices retain positive definiteness. In this paper we formally investigate the algebraic properties of p.d. matrices which are thresholded. We demonstrate that for positive definiteness to be preserved, the pattern of elements to be set to zero has to necessarily correspond to a graph which is a union of disconnected complete components. This result rigorously demonstrates that, except in special cases, positive definiteness can be easily lost. We then proceed to demonstrate that the class of diagonally dominant matrices is not maximal in terms of retaining positive definiteness when thresholded. Consequently, we derive characterizations of matrices which retain positive definiteness when thresholded with respect to important classes of graphs. In particular, we demonstrate that retaining positive definiteness upon thresholding is governed by complex algebraic conditions.

math.ST↗
arXiv2011

Large Scale Correlation Screening

This paper treats the problem of screening for variables with high correlations in high dimensional data in which there can be many fewer samples than variables. We focus on threshold-based correlation screening methods for three related applications: screening for variables with large correlations within a single treatment (autocorrelation screening); screening for variables with large cross-correlations over two treatments (cross-correlation screening); screening for variables that have persistently large auto-correlations over two treatments (persistent-correlation screening). The novelty of correlation screening is that it identifies a smaller number of variables which are highly correlated with others, as compared to identifying a number of correlation parameters. Correlation screening suffers from a phase transition phenomenon: as the correlation threshold decreases the number of discoveries increases abruptly. We obtain asymptotic expressions for the mean number of discoveries and the phase transition thresholds as a function of the number of samples, the number of variables, and the joint sample distribution. We also show that under a weak dependency condition the number of discoveries is dominated by a Poisson random variable giving an asymptotic expression for the false positive rate. The correlation screening approach bears tremendous dividends in terms of the type and strength of the asymptotic results that can be obtained. It also overcomes some of the major hurdles faced by existing methods in the literature as correlation screening is naturally scalable to high dimension. Numerical results strongly validate the theory that is presented in this paper. We illustrate the application of the correlation screening methodology on a large scale gene-expression dataset, revealing a few influential variables that exhibit a significant amount of correlation over multiple treatments.

stat.ML↗