arXiv subjects

Caroline Uhler

Publications and source records attributed to Caroline Uhler.

At least 91 records · Page 5Linked to original sources

arXiv2018

Patchnet: Interpretable Neural Networks for Image Classification

Understanding how a complex machine learning model makes a classification decision is essential for its acceptance in sensitive areas such as health care. Towards this end, we present PatchNet, a method that provides the features indicative of each class in an image using a tradeoff between restricting global image context and classification error. We mathematically analyze this tradeoff, demonstrate Patchnet's ability to construct sharp visual heatmap representations of the learned features, and quantitatively compare these features with features selected by domain experts by applying PatchNet to the classification of benign/malignant skin lesions from the ISBI-ISIC 2017 melanoma classification challenge.

cs.CV↗

arXiv2018

Direct Estimation of Differences in Causal Graphs

We consider the problem of estimating the differences between two causal directed acyclic graph (DAG) models with a shared topological order given i.i.d. samples from each model. This is of interest for example in genomics, where changes in the structure or edge weights of the underlying causal graphs reflect alterations in the gene regulatory networks. We here provide the first provably consistent method for directly estimating the differences in a pair of causal DAGs without separately learning two possibly large and dense DAG models and computing their difference. Our two-step algorithm first uses invariance tests between regression coefficients of the two data sets to estimate the skeleton of the difference graph and then orients some of the edges using invariance tests between regression residual variances. We demonstrate the properties of our method through a simulation study and apply it to the analysis of gene expression data from ovarian cancer and during T-cell activation.

stat.ME↗

arXiv2018

Minimal I-MAP MCMC for Scalable Structure Discovery in Causal DAG Models

Learning a Bayesian network (BN) from data can be useful for decision-making or discovering causal relationships. However, traditional methods often fail in modern applications, which exhibit a larger number of observed variables than data points. The resulting uncertainty about the underlying network as well as the desire to incorporate prior information recommend a Bayesian approach to learning the BN, but the highly combinatorial structure of BNs poses a striking challenge for inference. The current state-of-the-art methods such as order MCMC are faster than previous methods but prevent the use of many natural structural priors and still have running time exponential in the maximum indegree of the true directed acyclic graph (DAG) of the BN. We here propose an alternative posterior approximation based on the observation that, if we incorporate empirical conditional independence tests, we can focus on a high-probability DAG associated with each order of the vertices. We show that our method allows the desired flexibility in prior specification, removes timing dependence on the maximum indegree and yields provably good posterior approximations; in addition, we show that it achieves superior accuracy, scalability, and sampler mixing on several datasets.

stat.CO↗

arXiv2018

Geometry of Discrete Copulas

Multivariate distributions are fundamental to modeling. Discrete copulas can be used to construct diverse multivariate joint distributions over random variables from estimated univariate marginals. The space of discrete copulas admits a representation as a convex polytope which can be exploited in entropy-copula methods relevant to hydrology and climatology. To allow for an extensive use of such methods in a wide range of applied fields, it is important to have a geometric representation of discrete copulas with desirable stochastic properties. In this paper, we show that the families of ultramodular discrete copulas and their generalization to convex discrete quasi-copulas admit representations as polytopes. We draw connections to the prominent Birkhoff polytope, alternating sign matrix polytope, and their most extensive generalizations in the discrete geometry literature. In doing so, we generalize some well-known results on these polytopes from both the statistics literature and the discrete geometry literature.

math.ST↗

arXiv2018

Maximum likelihood estimation in Gaussian models under total positivity

We analyze the problem of maximum likelihood estimation for Gaussian distributions that are multivariate totally positive of order two (MTP2). By exploiting connections to phylogenetics and single-linkage clustering, we give a simple proof that the maximum likelihood estimator (MLE) for such distributions exists based on at least 2 observations, irrespective of the underlying dimension. Slawski and Hein, who first proved this result, also provided empirical evidence showing that the MTP2 constraint serves as an implicit regularizer and leads to sparsity in the estimated inverse covariance matrix, determining what we name the ML graph. We show that we can find an upper bound for the ML graph by adding edges corresponding to correlations in excess of those explained by the maximum weight spanning forest of the correlation matrix. Moreover, we provide globally convergent coordinate descent algorithms for calculating the MLE under the MTP2 constraint which are structurally similar to iterative proportional scaling. We conclude the paper with a discussion of signed MTP2 distributions.

stat.ME↗

arXiv2017

Generalized Permutohedra from Probabilistic Graphical Models

A graphical model encodes conditional independence relations via the Markov properties. For an undirected graph these conditional independence relations can be represented by a simple polytope known as the graph associahedron, which can be constructed as a Minkowski sum of standard simplices. There is an analogous polytope for conditional independence relations coming from a regular Gaussian model, and it can be defined using multiinformation or relative entropy. For directed acyclic graphical models and also for mixed graphical models containing undirected, directed and bidirected edges, we give a construction of this polytope, up to equivalence of normal fans, as a Minkowski sum of matroid polytopes. Finally, we apply this geometric insight to construct a new ordering-based search algorithm for causal inference via directed acyclic graphical models.

math.ST↗

arXiv2017

Generalized Fréchet Bounds for Cell Entries in Multidimensional Contingency Tables

We consider the lattice, $\mathcal{L}$, of all subsets of a multidimensional contingency table and establish the properties of monotonicity and supermodularity for the marginalization function, $n(\cdot)$, on $\mathcal{L}$. We derive from the supermodularity of $n(\cdot)$ some generalized Fréchet inequalities complementing and extending inequalities of Dobra and Fienberg. Further, we construct new monotonic and supermodular functions from $n(\cdot)$, and we remark on the connection between supermodularity and some correlation inequalities for probability distributions on lattices. We also apply an inequality of Ky Fan to derive a new approach to Fréchet inequalities for multidimensional contingency tables.

math.ST↗

arXiv2017

Permutation-based Causal Inference Algorithms with Interventions

Learning directed acyclic graphs using both observational and interventional data is now a fundamentally important problem due to recent technological developments in genomics that generate such single-cell gene expression data at a very large scale. In order to utilize this data for learning gene regulatory networks, efficient and reliable causal inference algorithms are needed that can make use of both observational and interventional data. In this paper, we present two algorithms of this type and prove that both are consistent under the faithfulness assumption. These algorithms are interventional adaptations of the Greedy SP algorithm and are the first algorithms using both observational and interventional data with consistency guarantees. Moreover, these algorithms have the advantage that they are nonparametric, which makes them useful also for analyzing non-Gaussian data. In this paper, we present these two algorithms and their consistency guarantees, and we analyze their performance on simulated data, protein signaling data, and single-cell gene expression data.

stat.ME↗

arXiv2017

Gaussian Graphical Models: An Algebraic and Geometric Perspective

Gaussian graphical models are used throughout the natural sciences, social sciences, and economics to model the statistical relationships between variables of interest in the form of a graph. We here provide a pedagogic introduction to Gaussian graphical models and review recent results on maximum likelihood estimation for such models. Throughout, we highlight the rich algebraic and geometric properties of Gaussian graphical models and explain how these properties relate to convex optimization and ultimately result in insights on the existence of the maximum likelihood estimator (MLE) and algorithms for computing the MLE.

math.ST↗

arXiv2017

Counting Markov Equivalence Classes by Number of Immoralities

Two directed acyclic graphs (DAGs) are called Markov equivalent if and only if they have the same underlying undirected graph (i.e. skeleton) and the same set of immoralities. Using observational data, a DAG model can only be determined up to Markov equivalence, and so it is desirable to understand the size and number of Markov equivalence classes (MECs) combinatorially. In this paper, we address this enumerative question using a pair of generating functions that encode the number and size of MECs on a skeleton $G$, and in doing so we connect this problem to classical problems in combinatorial optimization. The first is a graph polynomial that counts the number of MECs on $G$ by their number of immoralities. Using connections to the independent set problem, we show that computing a DAG on $G$ with the maximum possible number of immoralities is NP-hard. The second generating function counts the MECs on $G$ according to their size. Via computer enumeration, we show that this generating function is distinct for every connected graph on $p$ nodes for all $p\leq 10$.

math.CO↗

arXiv2017

Counting Markov Equivalence Classes for DAG models on Trees

DAG models are statistical models satisfying a collection of conditional independence relations encoded by the nonedges of a directed acyclic graph (DAG) $\mathcal{G}$. Such models are used to model complex cause-effect systems across a variety of research fields. From observational data alone, a DAG model $\mathcal{G}$ is only recoverable up to Markov equivalence. Combinatorially, two DAGs are Markov equivalent if and only if they have the same underlying undirected graph (i.e. skeleton) and the same set of the induced subDAGs $i\to j \leftarrow k$, known as immoralities. Hence it is of interest to study the number and size of Markov equivalence classes (MECs). In a recent paper, the authors introduced a pair of generating functions that enumerate the number of MECs on a fixed skeleton by number of immoralities and by class size, and they studied the complexity of computing these functions. In this paper, we lay the foundation for studying these generating functions by analyzing their structure for trees and other closely related graphs. We describe these polynomials for some important families of graphs including paths, stars, cycles, spider graphs, caterpillars, and complete binary trees. In doing so, we recover important connections to independence polynomials, and extend some classical identities that hold for Fibonacci numbers. We also provide tight lower and upper bounds for the number and size of MECs on any tree. Finally, we use computational methods to show that the number and distribution of high degree nodes in a triangle-free graph dictates the number and size of MECs.

math.CO↗

arXiv2016

Exact formulas for the normalizing constants of Wishart distributions for graphical models

Gaussian graphical models have received considerable attention during the past four decades from the statistical and machine learning communities. In Bayesian treatments of this model, the G-Wishart distribution serves as the conjugate prior for inverse covariance matrices satisfying graphical constraints. While it is straightforward to posit the unnormalized densities, the normalizing constants of these distributions have been known only for graphs that are chordal, or decomposable. Up until now, it was unknown whether the normalizing constant for a general graph could be represented explicitly, and a considerable body of computational literature emerged that attempted to avoid this apparent intractability. We close this question by providing an explicit representation of the G-Wishart normalizing constant for general graphs.

math.ST↗

arXiv2016

Total positivity in Markov structures

We discuss properties of distributions that are multivariate totally positive of order two (MTP2) related to conditional independence. In particular, we show that any independence model generated by an MTP2 distribution is a compositional semigraphoid which is upward-stable and singleton-transitive. In addition, we prove that any MTP2 distribution satisfying an appropriate support condition is faithful to its concentration graph. Finally, we analyze factorization properties of MTP2 distributions and discuss ways of constructing MTP2 distributions; in particular we give conditions on the log-linear parameters of a discrete distribution which ensure MTP2 and characterize conditional Gaussian distributions which satisfy MTP2.

math.ST↗

arXiv2016

Maximum Likelihood Estimation for Linear Gaussian Covariance Models

We study parameter estimation in linear Gaussian covariance models, which are $p$-dimensional Gaussian models with linear constraints on the covariance matrix. Maximum likelihood estimation for this class of models leads to a non-convex optimization problem which typically has many local maxima. Using recent results on the asymptotic distribution of extreme eigenvalues of the Wishart distribution, we provide sufficient conditions for any hill-climbing method to converge to the global maximum. Although we are primarily interested in the case in which $n>\!\!>p$, the proofs of our results utilize large-sample asymptotic theory under the scheme $n/p \to γ> 1$. Remarkably, our numerical simulations indicate that our results remain valid for $p$ as small as $2$. An important consequence of this analysis is that for sample sizes $n \simeq 14 p$, maximum likelihood estimation for linear Gaussian covariance models behaves as if it were a convex optimization problem.

math.ST↗

arXiv2016

Exact goodness-of-fit testing for the Ising model

The Ising model is one of the simplest and most famous models of interacting systems. It was originally proposed to model ferromagnetic interactions in statistical physics and is now widely used to model spatial processes in many areas such as ecology, sociology, and genetics, usually without testing its goodness of fit. Here, we propose various test statistics and an exact goodness-of-fit test for the finite-lattice Ising model. The theory of Markov bases has been developed in algebraic statistics for exact goodness-of-fit testing using a Monte Carlo approach. However, finding a Markov basis is often computationally intractable. Thus, we develop a Monte Carlo method for exact goodness-of-fit testing for the Ising model which avoids computing a Markov basis and also leads to a better connectivity of the Markov chain and hence to a faster convergence. We show how this method can be applied to analyze the spatial organization of receptors on the cell membrane.

math.ST↗

arXiv2015

Extremal Positive Semidefinite Matrices for Graphs without $K_5$ Minors

For a graph $G$ with $p$ vertices the closed convex cone $\mathbb{S}^p_{\succeq0}(G)$ consists of all real positive semidefinite $p\times p$ matrices with zeros in the off-diagonal entries corresponding to nonedges of $G$. The extremal rays of this cone and their associated ranks have applications to matrix completion problems, maximum likelihood estimation in Gaussian graphical models in statistics, and Gauss elimination for sparse matrices. For a graph $G$ without $K_5$ minors, we show that the normal vectors to the facets of the $(\pm1)$-cut polytope of $G$ specify the off-diagonal entries of extremal matrices in $\mathbb{S}^p_{\succeq0}(G)$. We also prove that the constant term of the linear equation of each facet-supporting hyperplane is the rank of its corresponding extremal matrix in $\mathbb{S}^p_{\succeq0}(G)$. Furthermore, we show that if $G$ is series-parallel then this gives a complete characterization of all possible extremal ranks of $\mathbb{S}^p_{\succeq0}(G)$, consequently solving the sparsity order problem for series-parallel graphs.

math.CO↗

arXiv2015

Exponential Varieties

Exponential varieties arise from exponential families in statistics. These real algebraic varieties have strong positivity and convexity properties, familiar from toric varieties and their moment maps. Among them are varieties of inverses of symmetric matrices satisfying linear constraints. This class includes Gaussian graphical models. We develop a general theory of exponential varieties. These are derived from hyperbolic polynomials and their integral representations. We compare the multidegrees and ML degrees of the gradient map for hyperbolic polynomials.

math.AG↗

arXiv2015

Faithfulness and learning hypergraphs from discrete distributions

The concepts of faithfulness and strong-faithfulness are important for statistical learning of graphical models. Graphs are not sufficient for describing the association structure of a discrete distribution. Hypergraphs representing hierarchical log-linear models are considered instead, and the concept of parametric (strong-) faithfulness with respect to a hypergraph is introduced. Strong-faithfulness ensures the existence of uniformly consistent parameter estimators and enables building uniformly consistent procedures for a hypergraph search. The strength of association in a discrete distribution can be quantified with various measures, leading to different concepts of strong-faithfulness. Lower and upper bounds for the proportions of distributions that do not satisfy strong-faithfulness are computed for different parameterizations and measures of association.

stat.ME↗