Differentiable Causal Discovery for Singular Linear Models under Confounding

Abstract

Score-based causal discovery in the presence of unobserved confounders requires both a consistent scoring criterion and an efficient search over graph structures. Linear causal models with correlated errors are naturally represented by acyclic directed mixed graphs (ADMGs), whose induced model families include singular statistical models for which the standard Bayesian Information Criterion is inconsistent. We develop a score-based framework for causal discovery over linear Gaussian ADMGs grounded in singular learning theory. We first verify that the conditions for consistency of the Widely Applicable Bayesian Information Criterion (WBIC) are satisfied by the linear causal model class. We then propose a scalable approximation of the WBIC via Automatic Differentiation Variational Inference (ADVI). Finally, we introduce a differentiable search over ADMGs through Gumbel-Softmax relaxations of both directed and bidirected adjacency. Experiments on synthetic and real-world benchmarks demonstrate improvements over the BIC-scored baseline on both singular and generic ADMGs under unobserved confounding.