arXiv subjects

Mathias Drton

Publications and source records attributed to Mathias Drton.

At least 91 records · Page 5Linked to original sources

arXiv2018

Graphical Models for Non-Negative Data Using Generalized Score Matching

A common challenge in estimating parameters of probability density functions is the intractability of the normalizing constant. While in such cases maximum likelihood estimation may be implemented using numerical integration, the approach becomes computationally intensive. In contrast, the score matching method of Hyvärinen (2005) avoids direct calculation of the normalizing constant and yields closed-form estimates for exponential families of continuous distributions over $\mathbb{R}^m$. Hyvärinen (2007) extended the approach to distributions supported on the non-negative orthant $\mathbb{R}_+^m$. In this paper, we give a generalized form of score matching for non-negative data that improves estimation efficiency. We also generalize the regularized score matching method of Lin et al. (2016) for non-negative Gaussian graphical models, with improved theoretical guarantees.

stat.ME↗

arXiv2018

Robust and sparse Gaussian graphical modeling under cell-wise contamination

Graphical modeling explores dependences among a collection of variables by inferring a graph that encodes pairwise conditional independences. For jointly Gaussian variables, this translates into detecting the support of the precision matrix. Many modern applications feature high-dimensional and contaminated data that complicate this task. In particular, traditional robust methods that down-weight entire observation vectors are often inappropriate as high-dimensional data may feature partial contamination in many observations. We tackle this problem by giving a robust method for sparse precision matrix estimation based on the $γ$-divergence under a cell-wise contamination model. Simulation studies demonstrate that our procedure outperforms existing methods especially for highly contaminated data.

stat.ME↗

arXiv2017

Empirical Likelihood for Linear Structural Equation Models with Dependent Errors

We consider linear structural equation models that are associated with mixed graphs. The structural equations in these models only involve observed variables, but their idiosyncratic error terms are allowed to be correlated and non-Gaussian. We propose empirical likelihood (EL) procedures for inference, and suggest several modifications, including a profile likelihood, in order to improve tractability and performance of the resulting methods. Through simulations, we show that when the error distributions are non-Gaussian, the use of EL and the proposed modifications may increase statistical efficiency and improve assessment of significance.

stat.CO↗

arXiv2017

Symmetric Rank Covariances: a Generalised Framework for Nonparametric Measures of Dependence

The need to test whether two random vectors are independent has spawned a large number of competing measures of dependence. We are interested in nonparametric measures that are invariant under strictly increasing transformations, such as Kendall's tau, Hoeffding's D, and the more recently discovered Bergsma--Dassios sign covariance. Each of these measures exhibits symmetries that are not readily apparent from their definitions. Making these symmetries explicit, we define a new class of multivariate nonparametric measures of dependence that we refer to as Symmetric Rank Covariances. This new class generalises all of the above measures and leads naturally to multivariate extensions of the Bergsma--Dassios sign covariance. Symmetric Rank Covariances may be estimated unbiasedly using U-statistics for which we prove results on computational efficiency and large-sample behavior. The algorithms we develop for their computation include, to the best of our knowledge, the first efficient algorithms for the well-known Hoeffding's D statistic in the multivariate setting.

math.ST↗

arXiv2017

Determinantal Generalizations of Instrumental Variables

Linear structural equation models relate the components of a random vector using linear interdependencies and Gaussian noise. Each such model can be naturally associated with a mixed graph whose vertices correspond to the components of the random vector. The graph contains directed edges that represent the linear relationships between components, and bidirected edges that encode unobserved confounding. We study the problem of generic identifiability, that is, whether a generic choice of linear and confounding effects can be uniquely recovered from the joint covariance matrix of the observed random vector. An existing combinatorial criterion for establishing generic identifiability is the half-trek criterion (HTC), which uses the existence of trek systems in the mixed graph to iteratively discover generically invertible linear equation systems in polynomial time. By focusing on edges one at a time, we establish new sufficient and necessary conditions for generic identifiability of edge effects extending those of the HTC. In particular, we show how edge coefficients can be recovered as quotients of subdeterminants of the covariance matrix, which constitutes a determinantal generalization of formulas obtained when using instrumental variables for identification.

math.ST↗

arXiv2016

Algebraic Problems in Structural Equation Modeling

The paper gives an overview of recent advances in structural equation modeling. A structural equation model is a multivariate statistical model that is determined by a mixed graph, also known as a path diagram. Our focus is on the covariance matrices of linear structural equation models. In the linear case, each covariance is a rational function of parameters that are associated to the edges and nodes of the graph. We statistically motivate algebraic problems concerning the rational map that parametrizes the covariance matrix. We review combinatorial tools such as the trek rule, projection to ancestral sets, and a graph decomposition due to Jin Tian. Building on these tools, we discuss advances in parameter identification, i.e., the study of (generic) injectivity of the parametrization, and explain recent results on determinantal relations among the covariances. The paper is based on lectures given at the 8th Mathematical Society of Japan Seasonal Institute.

math.ST↗

arXiv2016

Testing independence in high dimensions with sums of rank correlations

We treat the problem of testing independence between m continuous variables when m can be larger than the available sample size n. We consider three types of test statistics that are constructed as sums or sums of squares of pairwise rank correlations. In the asymptotic regime where both m and n tend to infinity, a martingale central limit theorem is applied to show that the null distributions of these statistics converge to Gaussian limits, which are valid with no specific distributional or moment assumptions on the data. Using the framework of U-statistics, our result covers a variety of rank correlations including Kendall's tau and a dominating term of Spearman's rank correlation coefficient (rho), but also degenerate U-statistics such as Hoeffding's $D$, or the $τ^*$ of Bergsma and Dassios (2014). As in the classical theory for U-statistics, the test statistics need to be scaled differently when the rank correlations used to construct them are degenerate U-statistics. The power of the considered tests is explored in rate-optimality theory under Gaussian equicorrelation alternatives as well as in numerical experiments for specific cases of more general alternatives.

math.ST↗

arXiv2016

Computation of maximum likelihood estimates in cyclic structural equation models

Software for computation of maximum likelihood estimates in linear structural equation models typically employs general techniques from non-linear optimization, such as quasi-Newton methods. In practice, careful tuning of initial values is often required to avoid convergence issues. As an alternative approach, we propose a block-coordinate descent method that cycles through the considered variables, updating only the parameters related to a given variable in each step. We show that the resulting block update problems can be solved in closed form even when the structural equation model comprises feedback cycles. Furthermore, we give a characterization of the models for which the block-coordinate descent algorithm is well-defined, meaning that for generic data and starting values all block optimization problems admit a unique solution. For the characterization, we represent each model by its mixed graph (also known as path diagram), which leads to criteria that can be checked in time that is polynomial in the number of considered variables.

stat.CO↗

arXiv2016

Structure Learning in Graphical Modeling

A graphical model is a statistical model that is associated to a graph whose nodes correspond to variables of interest. The edges of the graph reflect allowed conditional dependencies among the variables. Graphical models admit computationally convenient factorization properties and have long been a valuable tool for tractable modeling of multivariate distributions. More recently, applications such as reconstructing gene regulatory networks from gene expression data have driven major advances in structure learning, that is, estimating the graph underlying a model. We review some of these advances and discuss methods such as the graphical lasso and neighborhood selection for undirected graphical models (or Markov random fields), and the PC algorithm and score-based search methods for directed graphical models (or Bayesian networks). We further review extensions that account for effects of latent variables and heterogeneous data sources.

stat.ME↗

arXiv2016

A Bayesian information criterion for singular models

We consider approximate Bayesian model choice for model selection problems that involve models whose Fisher-information matrices may fail to be invertible along other competing submodels. Such singular models do not obey the regularity conditions underlying the derivation of Schwarz's Bayesian information criterion (BIC) and the penalty structure in BIC generally does not reflect the frequentist large-sample behavior of their marginal likelihood. While large-sample theory for the marginal likelihood of singular models has been developed recently, the resulting approximations depend on the true parameter value and lead to a paradox of circular reasoning. Guided by examples such as determining the number of components of mixture models, the number of factors in latent factor models or the rank in reduced-rank regression, we propose a resolution to this paradox and give a practical extension of BIC for singular model selection problems.

stat.ME↗

arXiv2016

Estimation of High-Dimensional Graphical Models Using Regularized Score Matching

Graphical models are widely used to model stochastic dependences among large collections of variables. We introduce a new method of estimating undirected conditional independence graphs based on the score matching loss, introduced by Hyvarinen (2005), and subsequently extended in Hyvarinen (2007). The regularized score matching method we propose applies to settings with continuous observations and allows for computationally efficient treatment of possibly non-Gaussian exponential family models. In the well-explored Gaussian setting, regularized score matching avoids issues of asymmetry that arise when applying the technique of neighborhood selection, and compared to existing methods that directly yield symmetric estimates, the score matching approach has the advantage that the considered loss is quadratic and gives piecewise linear solution paths under $\ell_1$ regularization. Under suitable irrepresentability conditions, we show that $\ell_1$-regularized score matching is consistent for graph estimation in sparse high-dimensional settings. Through numerical experiments and an application to RNAseq data, we confirm that regularized score matching achieves state-of-the-art performance in the Gaussian case and provides a valuable tool for computationally efficient estimation in non-Gaussian graphical models.

stat.ME↗

arXiv2016

Large-Sample Theory for the Bergsma-Dassios Sign Covariance

The Bergsma-Dassios sign covariance is a recently proposed extension of Kendall's tau. In contrast to tau or also Spearman's rho, the new sign covariance $τ^*$ vanishes if and only if the two considered random variables are independent. Specifically, this result has been shown for continuous as well as discrete variables. We develop large-sample distribution theory for the empirical version of $τ^*$. In particular, we use theory for degenerate U-statistics to derive asymptotic null distributions under independence and demonstrate in simulations that the limiting distributions give useful approximations.

math.ST↗

arXiv2016

Wald tests of singular hypotheses

Motivated by the problem of testing tetrad constraints in factor analysis, we study the large-sample distribution of Wald statistics at parameter points at which the gradient of the tested constraint vanishes. When based on an asymptotically normal estimator, the Wald statistic converges to a rational function of a normal random vector. The rational function is determined by a homogeneous polynomial and a covariance matrix. For quadratic forms and bivariate monomials of arbitrary degree, we show unexpected relationships to chi-square distributions that explain conservative behavior of certain Wald tests. For general monomials, we offer a conjecture according to which the reciprocal of a certain quadratic form in the reciprocals of dependent normal random variables is chi-square distributed.

math.ST↗

arXiv2015

Efficient Computation of the Bergsma-Dassios Sign Covariance

In an extension of Kendall's $τ$, Bergsma and Dassios (2014) introduced a covariance measure $τ^*$ for two ordinal random variables that vanishes if and only if the two variables are independent. For a sample of size $n$, a direct computation of $t^*$, the empirical version of $τ^*$, requires $O(n^4)$ operations. We derive an algorithm that computes the statistic using only $O(n^2\log(n))$ operations.

stat.CO↗

arXiv2015

Marginal likelihood and model selection for Gaussian latent tree and forest models

Gaussian latent tree models, or more generally, Gaussian latent forest models have Fisher-information matrices that become singular along interesting submodels, namely, models that correspond to subforests. For these singularities, we compute the real log-canonical thresholds (also known as stochastic complexities or learning coefficients) that quantify the large-sample behavior of the marginal likelihood in Bayesian inference. This provides the information needed for a recently introduced generalization of the Bayesian information criterion. Our mathematical developments treat the general setting of Laplace integrals whose phase functions are sums of squared differences between monomials and constants. We clarify how in this case real log-canonical thresholds can be computed using polyhedral geometry, and we show how to apply the general theory to the Laplace integrals associated with Gaussian latent tree and forest models. In simulations and a data example, we demonstrate how the mathematical knowledge can be applied in model selection.

stat.ME↗

arXiv2015

Maximum Likelihood Estimates for Gaussian Mixtures Are Transcendental

Gaussian mixture models are central to classical statistics, widely used in the information sciences, and have a rich mathematical structure. We examine their maximum likelihood estimates through the lens of algebraic statistics. The MLE is not an algebraic function of the data, so there is no notion of ML degree for these models. The critical points of the likelihood function are transcendental, and there is no bound on their number, even for mixtures of two univariate Gaussians.

math.ST↗

arXiv2015

Identifiability of directed Gaussian graphical models with one latent source

We study parameter identifiability of directed Gaussian graphical models with one latent variable. In the scenario we consider, the latent variable is a confounder that forms a source node of the graph and is a parent to all other nodes, which correspond to the observed variables. We give a graphical condition that is sufficient for the Jacobian matrix of the parametrization map to be full rank, which entails that the parametrization is generically finite-to-one, a fact that is sometimes also referred to as local identifiability. We also derive a graphical condition that is necessary for such identifiability. Finally, we give a condition under which generic parameter identifiability can be determined from identifiability of a model associated with a subgraph. The power of these criteria is assessed via an exhaustive algebraic computational study on models with 4, 5, and 6 observable variables.

math.ST↗

arXiv2015

Generic Identifiability of Linear Structural Equation Models by Ancestor Decomposition

Linear structural equation models, which relate random variables via linear interdependencies and Gaussian noise, are a popular tool for modeling multivariate joint distributions. These models correspond to mixed graphs that include both directed and bidirected edges representing the linear relationships and correlations between noise terms, respectively. A question of interest for these models is that of parameter identifiability, whether or not it is possible to recover edge coefficients from the joint covariance matrix of the random variables. For the problem of determining generic parameter identifiability, we present an algorithm that extends an algorithm from prior work by Foygel, Draisma, and Drton (2012). The main idea underlying our new algorithm is the use of ancestral subsets of vertices in the graph in application of a decomposition idea of Tian (2005).

stat.CO↗