Searcharxiv⌕ Search

arXiv subjects

Armeen Taeb

Publications and source records attributed to Armeen Taeb.

At least 19 recordsLinked to original sources

Interpretable confidence regions for causal discovery

Most work in causal discovery focuses on point estimation of the underlying causal graph, while uncertainty in the learned structure has received substantially less attention. We develop confidence regions for the completed partially directed acyclic graph (CPDAG) identified from observational data. Our approach builds on perturbed PC (Chang et al., 2026), which generates plausible CPDAGs by perturbing the conditional independence tests underlying the widely used PC algorithm. The threshold used in these tests controls how perturbations translate into variation across the resulting CPDAGs. We propose a new thresholding strategy that suppresses spurious variation across runs while preserving asymptotic coverage, producing a substantially tighter collection of plausible CPDAGs. This collection is summarized through confidence balls and model confidence intervals constructed from the partially ordered set of CPDAGs under model inclusion. We also develop new theory and algorithms for the exact computation of these regions. Simulations show that our confidence regions attain good coverage while being substantially smaller and more interpretable than the original perturbed PC collections.

stat.ME↗

Mixed-Integer Programming for Causal Discovery with Unknown Hard-Intervention Targets

When intervention targets are uncertain, learning causal relationships and identifying directly perturbed variables become a joint task. We develop an exact mixed-integer program based on optimized parent-set scores (PS-MIP) for joint graph and intervention-target estimation in linear Gaussian structural equation models under stochastic hard interventions. The method can incorporate candidate-edge restrictions and partial target knowledge when available. For each random variable, PS-MIP evaluates candidate sets of direct causes, or parent sets, by optimizing the likelihood parameters and compatible target patterns, retaining both the score and a minimizing target pattern. A global mixed-integer program then selects parent sets subject to acyclicity. Under stated conditions, this reduction preserves the joint objective and reconstructs the graph, targets, and likelihood parameters. Because the local scores are precomputed, PS-MIP replaces a direct large bilinear mixed-integer program with a linear selection problem; empirically it certifies global optimality on far more instances than the direct formulation (126 of 180 versus 34) and is often orders of magnitude faster. On synthetic data, PS-MIP achieves favorable false-discovery--true-discovery trade-offs relative to several benchmark algorithms for causal discovery with unknown intervention targets, along with strong intervention-target recovery in denser settings. An exploratory study on real data from a controlled physical system further demonstrates recovery beyond simulation.

math.OC↗

Integrating multi-source local information for global undirected graph reconstruction via integer programming

Graphical models are widely used to represent conditional independence structures, but learning them is challenging when data are fragmented across sources that measure different subsets of variables and may not be directly shareable. We develop a general framework for reconstructing a shared global graph from heterogeneous, partially observed sources. Our framework permits both the observed variable sets and source-specific distributions to differ. Under a source-wise faithfulness condition, local conditional independence graphs correspond to latent projections of the shared global graph, which is identifiable up to the equivalence class induced by the observed variable sets. The framework integrates raw data, estimated source graphs, or a combination of both, accommodating settings with restricted access to individual-level data, such as privacy constraints. We develop mixed-integer optimization formulations and scalable approximate algorithms for several classes of graphical models. Unlike standard high-dimensional analyses, our guarantees are governed by new combinatorial notions of signal strength that capture how information propagates across partially observed sources. Extensive simulations further show that integration can substantially improve estimation of source-specific projected graphs relative to separate source methods. Real-data application illustrates the framework's use with partially-observed sources of fatty acid measurements.

stat.ME↗

Convex Mixed-Integer Programming for Invariant Predictions from Multi-environment Data

Heterogeneous data provide an opportunity to identify causal predictors by revealing which predictive relationships remain invariant. Under a linear model, we introduce a least-squares coefficient-and-risk (LSCR) invariance principle, requiring both ordinary least squares coefficients and prediction risks to remain constant across environments. Our model neither specifies an intervention mechanism nor assumes a structural causal model for the entire system. We develop an invariance score that quantifies departures from LSCR invariance and use its optimization to construct two complementary procedures for causal predictor identification. First, we propose an estimator that jointly minimizes prediction risk and invariance penalty, and establish non-asymptotic guarantees for estimation and recovery of the causal parents under suitable identifiability conditions. Second, without requiring full identifiability, we develop a testing procedure that minimizes the invariance score over all predictor sets excluding a given variable, enabling valid causal discoveries with asymptotic error control via a multiplier bootstrap. Both procedures admit convex mixed-integer optimization formulations that avoid exhaustive subset enumeration while providing certificates of optimality. Simulations and real data experiments demonstrate substantial statistical and computational improvements over existing methods.

stat.ME↗

Consensus Tree Estimation with False Discovery Control via Partially Ordered Sets

Trees are data objects that hierarchically organize categories. Collections of trees arise in a diverse variety of fields, including evolutionary biology, machine learning, social sciences and anatomy. Summarizing a collection of trees by a single representative is challenging, in part due to the dimensions of both the sample space and the parameter space. We frame consensus tree estimation as a structured feature-selection problem, where leaves and edges are the features. We introduce a partial order on trees, use it to define false discoveries for a candidate summary tree, and develop novel estimation algorithms that control the false discovery rate at a nominal level for a broad class of generative models. We also use the partial order to assess the stability of features in a selected tree. Importantly, our method accommodates unequal leaf sets and non-binary trees, which commonly arise in modern datasets. Our feature-selection perspective yields finite-sample and model-free guarantees and provides a foundation for integrating multiple testing tools into tree estimation. We apply the method to study the origins of complex life. Our estimated tree recapitulates well-known divisions but highlights that there is insufficient data to determine the most recent archaeal ancestor of eukaryotic life.

stat.ME↗

Model-oriented Graph Distances via Partially Ordered Sets

A well-defined distance on the parameter space is key to evaluating estimators, ensuring consistency, and building confidence sets. While there are typically standard distances to adopt in a continuous space, this is not the case for combinatorial parameters such as graphs that represent statistical models. Defined on the graphs alone, existing proposals like the structural Hamming distance ignore the structure of the model space and can thus exhibit undesirable behaviors. We propose a model-oriented framework for defining the distance between graphs that is applicable across different graph classes. Our approach treats each graph as a statistical model and organizes the graphs in a partially ordered set based on model inclusion. This induces a neighborhood structure, from which we define the model-oriented distance as the length of a shortest path through neighbors, yielding a metric in the space of graphs. We apply this framework to probabilistic undirected graphs, causal directed acyclic graphs, {causal acyclic directed mixed graphs}, probabilistic completed partially directed acyclic graphs, and causal maximally oriented partially directed acyclic graphs. We analyze theoretical and empirical behaviors of the model-oriented distance. By exploiting the underlying poset structures, we develop algorithms for computing and bounding the proposed distance that scale to moderate-sized graphs. Finally, we showcase its utility for quantifying the robustness of adjustment sets to errors in specifying the causal graph.

math.ST↗

Quantifying uncertainty and stability among highly correlated predictors: a subspace perspective

We study the problem of linear feature selection when features are highly correlated. Such settings pose two fundamental challenges. First, how should model similarity be defined? Simply counting features in common can be misleading: two models may share no features, yet highly correlated features can make the two models very similar in terms of predictive ability. Second, how can feature stability be assessed across runs of a variable selection method? High correlation can yield very different feature sets, so counting how often a feature is selected may label most features as unstable, and selecting stable features would result in models that are too small with poor predictive performance. In essence, these issues arise because existing notions of similarity and stability are "discrete" in nature. To overcome these challenges, we propose a novel framework based on feature subspaces -- the subspaces spanned by selected columns of the feature matrix. This new perspective leads to "continuous" measures of similarity and stability, as well as false positive error, all of which are defined in terms of "closeness" of feature subspaces. Our measures naturally account for feature correlation and reduce to existing discrete notions when features are uncorrelated. To obtain stable models, we propose and theoretically analyze a subspace-based generalization of stability selection (Meinshausen & Bühlmann 2010, Taeb et al. 2020), which combines a discrete model search with a continuous subspace-based assessment of stability. On synthetic and real gene expression data, our method improves on existing stability-based approaches by (i) producing multiple stable models that capture feature interchangeability, and (ii) generating larger models with better predictive performance. Our method is implemented in the R package substab.

stat.ME↗

Convex Mixed-Integer Programming for Causal Additive Models with Optimization and Statistical Guarantees

We study the problem of learning a directed acyclic graph from data generated according to an additive, non-linear structural equation model with Gaussian noise. We express each non-linear function through a basis expansion, and derive a maximum likelihood estimator with a group l0-regularization that penalizes the number of edges in the graph. The resulting estimator is formulated through a convex mixed-integer program, enabling the use of branch-and-bound methods to obtain a solution that is guaranteed to be accurate up to a pre-specified optimality gap. Our formulation can naturally encode background knowledge, such as the presence or absence of edges and partial order constraints among the variables. We establish consistency guarantees for our estimator in terms of graph recovery, even when the number of variables grows with the sample size. Additionally, by connecting the optimality guarantees with our statistical error bounds, we derive an early stopping criterion that allows terminating the branch-and-bound procedure while preserving consistency. Compared with existing approaches that either assume equal error variances, restrict to linear structural equation models, or rely on heuristic procedures, our method enjoys both optimization and statistical guarantees. Extensive simulations and real-data analysis show that the proposed method achieves markedly better graph recovery performance.

stat.ME↗

An Asymptotically Optimal Coordinate Descent Algorithm for Learning Bayesian Networks from Gaussian Models

This paper studies the problem of learning Bayesian networks from continuous observational data, generated according to a linear Gaussian structural equation model. We consider an $\ell_0$-penalized maximum likelihood estimator for this problem which is known to have favorable statistical properties but is computationally challenging to solve, especially for medium-sized Bayesian networks. We propose a new coordinate descent algorithm to approximate this estimator and prove several remarkable properties of our procedure: the algorithm converges to a coordinate-wise minimum, and despite the non-convexity of the loss function, as the sample size tends to infinity, the objective value of the coordinate descent solution converges to the optimal objective value of the $\ell_0$-penalized maximum likelihood estimator. Finite-sample statistical consistency guarantees are also established. To the best of our knowledge, our proposal is the first coordinate descent procedure endowed with optimality and statistical guarantees in the context of learning Bayesian networks. Numerical experiments on synthetic and real data demonstrate that our coordinate descent method can obtain near-optimal solutions while being scalable.

stat.ML↗

A spectral method for multi-view subspace learning using the product of projections

Multi-view data provides complementary information on the same set of observations, with multi-omics and multimodal sensor data being common examples. Analyzing such data typically requires distinguishing between shared (joint) and unique (individual) signal subspaces from noisy, high-dimensional measurements. Despite many proposed methods, the conditions for reliably identifying joint and individual subspaces remain unclear. We rigorously quantify these conditions, which depend on the ratio of the signal rank to the ambient dimension, principal angles between true subspaces, and noise levels. Our approach characterizes how spectrum perturbations of the product of projection matrices, derived from each view's estimated subspaces, affect subspace separation. Using these insights, we provide an easy-to-use and scalable estimation algorithm. In particular, we employ rotational bootstrap and random matrix theory to partition the observed spectrum into joint, individual, and noise subspaces. Diagnostic plots visualize this partitioning, providing practical and interpretable insights into the estimation performance. In simulations, our method estimates joint and individual subspaces more accurately than existing approaches. Applications to multi-omics data from colorectal cancer patients and nutrigenomic study of mice demonstrate improved performance in downstream predictive tasks.

stat.ML↗

Model Selection over Partially Ordered Sets

In problems such as variable selection and graph estimation, models are characterized by Boolean logical structure such as presence or absence of a variable or an edge. Consequently, false positive error or false negative error can be specified as the number of variables/edges that are incorrectly included or excluded in an estimated model. However, there are several other problems such as ranking, clustering, and causal inference in which the associated model classes do not admit transparent notions of false positive and false negative errors due to the lack of an underlying Boolean logical structure. In this paper, we present a generic approach to endow a collection of models with partial order structure, which leads to a hierarchical organization of model classes as well as natural analogs of false positive and false negative errors. We describe model selection procedures that provide false positive error control in our general setting and we illustrate their utility with numerical experiments.

stat.ME↗

Extremal graphical modeling with latent variables via convex optimization

Extremal graphical models encode the conditional independence structure of multivariate extremes and provide a powerful tool for quantifying the risk of rare events. Prior work on learning these graphs from data has focused on the setting where all relevant variables are observed. For the popular class of Hüsler-Reiss models, we propose the \texttt{eglatent} method, a tractable convex program for learning extremal graphical models in the presence of latent variables. Our approach decomposes the Hüsler-Reiss precision matrix into a sparse component encoding the graphical structure among the observed variables after conditioning on the latent variables, and a low-rank component encoding the effect of a few latent variables on the observed variables. We provide finite-sample guarantees of \texttt{eglatent} and show that it consistently recovers the conditional graph as well as the number of latent variables. We highlight the improved performances of our approach on synthetic and real data.

stat.ME↗

Integer Programming for Learning Directed Acyclic Graphs from Non-identifiable Gaussian Models

We study the problem of learning directed acyclic graphs from continuous observational data, generated according to a linear Gaussian structural equation model. State-of-the-art structure learning methods for this setting have at least one of the following shortcomings: i) they cannot provide optimality guarantees and can suffer from learning sub-optimal models; ii) they rely on the stringent assumption that the noise is homoscedastic, and hence the underlying model is fully identifiable. We overcome these shortcomings and develop a computationally efficient mixed-integer programming framework for learning medium-sized problems that accounts for arbitrary heteroscedastic noise. We present an early stopping criterion under which we can terminate the branch-and-bound procedure to achieve an asymptotically optimal solution and establish the consistency of this approximate solution. In addition, we show via numerical experiments that our method outperforms state-of-the-art algorithms and is robust to noise heteroscedasticity, whereas the performance of some competing methods deteriorates under strong violations of the identifiability assumption. The software implementation of our method is available as the Python package \emph{micodag}.

stat.ME↗

Causality-oriented robustness: exploiting general noise interventions

Since distribution shifts are common in real-world applications, there is a pressing need to develop prediction models that are robust against such shifts. Existing frameworks, such as empirical risk minimization or distributionally robust optimization, either lack generalizability for unseen distributions or rely on postulated distance measures. Alternatively, causality offers a data-driven and structural perspective to robust predictions. However, the assumptions necessary for causal inference can be overly stringent, and the robustness offered by such causal models often lacks flexibility. In this paper, we focus on causality-oriented robustness and propose Distributional Robustness via Invariant Gradients (DRIG), a method that exploits general noise interventions in training data for robust predictions against unseen interventions, and naturally interpolates between in-distribution prediction and causality. In a linear setting, we prove that DRIG yields predictions that are robust among a data-dependent class of distribution shifts. Furthermore, we show that our framework includes anchor regression as a special case, and that it yields prediction models that protect against more diverse perturbations. We establish finite-sample results and extend our approach to semi-supervised domain adaptation to further improve prediction performance. Finally, we empirically validate our methods on synthetic simulations and on single-cell and intensive health care datasets.

stat.ME↗

Characterization and Greedy Learning of Gaussian Structural Causal Models under Unknown Interventions

We consider the problem of recovering the causal structure underlying observations from different experimental conditions when the targets of the interventions in each experiment are unknown. We assume a linear structural causal model with additive Gaussian noise and consider interventions that perturb their targets while maintaining the causal relationships in the system. Different models may entail the same distributions, offering competing causal explanations for the given observations. We fully characterize this equivalence class and offer identifiability results, which we use to derive a greedy algorithm called GnIES to recover the equivalence class of the data-generating model without knowledge of the intervention targets. In addition, we develop a novel procedure to generate semi-synthetic data sets with known causal ground truth but distributions closely resembling those of a real data set of choice. We leverage this procedure and evaluate the performance of GnIES on an array of synthetic and semi-synthetic data sets, and real data from a biological system and a tightly controlled physical system. We provide, in the Python packages gnies and sempler, implementations of GnIES and our semi-synthetic data generation procedure.

stat.ME↗

Learning and scoring Gaussian latent variable causal models with unknown additive interventions

With observational data alone, causal structure learning is a challenging problem. The task becomes easier when having access to data collected from perturbations of the underlying system, even when the nature of these is unknown. Existing methods either do not allow for the presence of latent variables or assume that these remain unperturbed. However, these assumptions are hard to justify if the nature of the perturbations is unknown. We provide results that enable scoring causal structures in the setting with additive, but unknown interventions. Specifically, we propose a maximum-likelihood estimator in a structural equation model that exploits system-wide invariances to output an equivalence class of causal structures from perturbation data. Furthermore, under certain structural assumptions on the population model, we provide a simple graphical characterization of all the DAGs in the interventional equivalence class. We illustrate the utility of our framework on synthetic data as well as real data involving California reservoirs and protein expressions. The software implementation is available as the Python package \emph{utlvce}.

stat.ME↗

Inverse Problems and Data Assimilation

We provide a clear and concise introduction to the subjects of inverse problems and data assimilation, and their inter-relations. The first part of our notes covers inverse problems; this refers to the study of how to estimate unknown model parameters from data. The second part of our notes covers data assimilation; this refers to a particular class of inverse problems in which the unknown parameter is the initial condition (and/or state) of a dynamical system, and the data comprises partial and noisy observations of the state. The third and final part of our notes describes the use of data assimilation methods to solve generic inverse problems by introducing an artificial algorithmic time. Our notes cover, among other topics, maximum a posteriori estimation, (stochastic) gradient descent, variational Bayes, Monte Carlo, importance sampling and Markov chain Monte Carlo for inverse problems; and 3DVAR, 4DVAR, extended and ensemble Kalman filters, and particle filters for data assimilation. Each of parts one and two starts with a chapter on the Bayesian formulation, in which the problem solution is given by a posterior distribution on the unknown parameter. Then the following chapter specializes the Bayesian formulation to a linear-Gaussian setting where explicit characterization of the posterior is possible and insightful. The next two chapters explore methods to extract information from the posterior in nonlinear and non-Gaussian settings using optimization and Gaussian approximations. The final two chapters describe sampling methods that can reproduce the full posterior in the large sample limit. Each chapter closes with a bibliography containing citations to alternative pedagogical literature and to relevant research literature. We also include a set of exercises at the end of parts one and two. Our notes are thus useful for both classroom teaching and self-guided study.

stat.ME↗

Provable concept learning for interpretable predictions using variational autoencoders

In safety-critical applications, practitioners are reluctant to trust neural networks when no interpretable explanations are available. Many attempts to provide such explanations revolve around pixel-based attributions or use previously known concepts. In this paper we aim to provide explanations by provably identifying \emph{high-level, previously unknown ground-truth concepts}. To this end, we propose a probabilistic modeling framework to derive (C)oncept (L)earning and (P)rediction (CLAP) -- a VAE-based classifier that uses visually interpretable concepts as predictors for a simple classifier. Assuming a generative model for the ground-truth concepts, we prove that CLAP is able to identify them while attaining optimal classification accuracy. Our experiments on synthetic datasets verify that CLAP identifies distinct ground-truth concepts on synthetic datasets and yields promising results on the medical Chest X-Ray dataset.

cs.LG↗