Searcharxiv⌕ Search

arXiv subjects

Yangbo He

Publications and source records attributed to Yangbo He.

15 recordsLinked to original sources
arXiv2025

Robust Bayesian Dynamic Programming for On-policy Risk-sensitive Reinforcement Learning

We propose a novel framework for risk-sensitive reinforcement learning (RSRL) that incorporates robustness against transition uncertainty. We define two distinct yet coupled risk measures: an inner risk measure addressing state and cost randomness and an outer risk measure capturing transition dynamics uncertainty. Our framework unifies and generalizes most existing RL frameworks by permitting general coherent risk measures for both inner and outer risk measures. Within this framework, we construct a risk-sensitive robust Markov decision process (RSRMDP), derive its Bellman equation, and provide error analysis under a given posterior distribution. We further develop a Bayesian Dynamic Programming (Bayesian DP) algorithm that alternates between posterior updates and value iteration. The approach employs an estimator for the risk-based Bellman operator that combines Monte Carlo sampling with convex optimization, for which we prove strong consistency guarantees. Furthermore, we demonstrate that the algorithm converges to a near-optimal policy in the training environment and analyze both the sample complexity and the computational complexity under the Dirichlet posterior and CVaR. Finally, we validate our approach through two numerical experiments. The results exhibit excellent convergence properties while providing intuitive demonstrations of its advantages in both risk-sensitivity and robustness. Empirically, we further demonstrate the advantages of the proposed algorithm through an application on option hedging.

q-fin.RM↗
arXiv2025

Covariate Balancing Value Estimation for Optimal Individualized Treatment Rules

Learning an optimal individualized treatment rule depends on reliable value comparisons across the candidate class. Standard doubly robust estimators are consistent when either the propensity score or outcome regression model is correctly specified, but they do not directly control the remaining bias in value estimation when both models are misspecified. In this paper, we propose a covariate balancing doubly robust estimator that combines propensity score estimation based on covariate balancing with an outcome regression component selected using an empirical variance criterion based on the influence function. Using prespecified covariate functions, the balancing procedure induces an effective balancing space to which the weighted propensity score error is orthogonal. Consequently, the proposed value estimator is consistent if either the propensity score model is correct or the rule-relevant outcome regression error lies in this space. The latter condition does not require a correctly specified outcome regression model and can hold even when both working models are misspecified. Under correct propensity score specification, the estimator has the smallest asymptotic variance within the proposed covariate balancing doubly robust class. Simulations evaluate value estimation in finite samples and the regret of learned rules, and an application to a leukemia dataset illustrates the proposed method in practice.

stat.ME↗
arXiv2025

Identification and estimation of causal peer effects using instrumental variables

In social science researches, causal inference regarding peer effects often faces significant challenges due to homophily bias and contextual confounding. For example, unmeasured health conditions (e.g., influenza) and psychological states (e.g., happiness, loneliness) can spread among closely connected individuals, such as couples or siblings. To address these issues, we define four effect estimands for dyadic data to characterize direct effects and spillover effects. We employ dual instrumental variables to achieve nonparametric identification of these causal estimands in the presence of unobserved confounding. We then derive the efficient influence functions for these estimands under the nonparametric model. Additionally, we develop a triply robust and locally efficient estimator that remains consistent even under partial misspecification of the observed data model. The proposed robust estimators can be easily adapted to flexible approaches such as machine learning estimation methods, provided that certain rate conditions are satisfied. Finally, we illustrate our approach through simulations and an empirical application evaluating the peer effects of retirement on fluid cognitive perception among couples.

stat.ME↗
arXiv2025

VACT: A Video Automatic Causal Testing System and a Benchmark

With the rapid advancement of text-conditioned Video Generation Models (VGMs), the quality of generated videos has significantly improved, bringing these models closer to functioning as ``*world simulators*'' and making real-world-level video generation more accessible and cost-effective. However, the generated videos often contain factual inaccuracies and lack understanding of fundamental physical laws. While some previous studies have highlighted this issue in limited domains through manual analysis, a comprehensive solution has not yet been established, primarily due to the absence of a generalized, automated approach for modeling and assessing the causal reasoning of these models across diverse scenarios. To address this gap, we propose VACT: an **automated** framework for modeling, evaluating, and measuring the causal understanding of VGMs in real-world scenarios. By combining causal analysis techniques with a carefully designed large language model assistant, our system can assess the causal behavior of models in various contexts without human annotation, which offers strong generalization and scalability. Additionally, we introduce multi-level causal evaluation metrics to provide a detailed analysis of the causal performance of VGMs. As a demonstration, we use our framework to benchmark several prevailing VGMs, offering insight into their causal reasoning capabilities. Our work lays the foundation for systematically addressing the causal understanding deficiencies in VGMs and contributes to advancing their reliability and real-world applicability.

cs.AI↗
arXiv2024

Local Causal Discovery with Background Knowledge

Causality plays a pivotal role in various fields of study. Based on the framework of causal graphical models, previous works have proposed identifying whether a variable is a cause or non-cause of a target in every Markov equivalent graph solely by learning a local structure. However, the presence of prior knowledge, often represented as a partially known causal graph, is common in many causal modeling applications. Leveraging this prior knowledge allows for the further identification of causal relationships. In this paper, we first propose a method for learning the local structure using all types of causal background knowledge, including direct causal information, non-ancestral information and ancestral information. Then we introduce criteria for identifying causal relationships based solely on the local structure in the presence of prior knowledge. We also apply out method to fair machine learning, and experiments involving local structure learning, causal relationship identification, and fair machine learning demonstrate that our method is both effective and efficient.

stat.ML↗
arXiv2023

Mediation analysis with unmeasured confounding between parallel mediators and outcome

Mediation analysis extending beyond single mediators has gained significant attention in recent years. However, related methods often assume the absence of unmeasured mediator-outcome confounding. To address this, we develop a mediation analysis framework that accounts for such confounding within a linear structural equation model with parallel mediators. Specifically, we introduce a pseudo proxy variable to capture unmeasured confounding, allowing us to identify causal parameters. Leveraging this proxy, we propose a partially penalized method to identify mediators that significantly affect the outcome. The resultant estimates are consistent, and the estimates of nonzero parameters are asymptotically normal. Motivated by these results, we further introduce a procedure that can consistently select active mediation pathways with large probability. Simulation studies demonstrate the superior performance of the proposed approach. Finally, we apply our approach to genomic data, identifying gene expressions that potentially mediate the impact of a genetic variant on mouse obesity.

stat.ME↗
arXiv2023

Identifying Causal Effects Using Instrumental Variables from the Auxiliary Dataset

Instrumental variable approaches have gained popularity for estimating causal effects in the presence of unmeasured confounders. However, the availability of instrumental variables in the primary dataset is often challenged due to stringent and untestable assumptions. This paper presents a novel method to identify and estimate causal effects by utilizing instrumental variables from the auxiliary dataset, incorporating a structural equation model, even in scenarios with nonlinear treatment effects. Our approach involves using two datasets: one called the primary dataset with joint observations of treatment and outcome, and another auxiliary dataset providing information about the instrument and treatment. Our strategy differs from most existing methods by not depending on the simultaneous measurements of instrument and outcome. The central idea for identifying causal effects is to establish a valid substitute through the auxiliary dataset, addressing unmeasured confounders. This is achieved by developing a control function and projecting it onto the function space spanned by the treatment variable. We then propose a three-step estimator for estimating causal effects and derive its asymptotic results. We illustrate the proposed estimator through simulation studies, and the results demonstrate favorable performance. We also conduct a real data analysis to evaluate the causal effect between vitamin D status and body mass index.

stat.ME↗
arXiv2023

Identifiability of causal effects with non-Gaussianity and auxiliary covariates

Assessing causal effects in the presence of unmeasured confounding is challenging. Although auxiliary variables, such as instrumental variables, are commonly used to identify causal effects, they are often unavailable in practice due to stringent and untestable conditions. To address this issue, previous researches have utilized linear structural equation models to show that the causal effect is identifiable when noise variables of the treatment and outcome are both non-Gaussian. In this paper, we investigate the problem of identifying the causal effect using the auxiliary covariate and non-Gaussianity from the treatment. Our key idea is to characterize the impact of unmeasured confounders using an observed covariate, assuming they are all Gaussian. We demonstrate that the causal effect can be identified using a measured covariate, and then extend the identification results to the multi-treatment setting. We further develop a simple estimation procedure for estimating causal effects and derive a $\sqrt{n}$-consistent estimator. Finally, we evaluate the performance of our estimator through simulation studies and apply our method to investigate the effect of the trade on income.

stat.ME↗
arXiv2022

On the Representation of Pairwise Causal Background Knowledge and Its Applications in Causal Inference

Pairwise causal background knowledge about the existence or absence of causal edges and paths is frequently encountered in observational studies. Such constraints allow the shared directed and undirected edges in the constrained subclass of Markov equivalent DAGs to be represented as a causal maximally partially directed acyclic graph (MPDAG). In this paper, we first provide a sound and complete graphical characterization of causal MPDAGs and introduce a minimal representation of a causal MPDAG. Then, we give a unified representation for three types of pairwise causal background knowledge, including direct, ancestral and non-ancestral causal knowledge, by introducing a novel concept called direct causal clause (DCC). Using DCCs, we study the consistency and equivalence of pairwise causal background knowledge and show that any pairwise causal background knowledge set can be uniquely and equivalently decomposed into the causal MPDAG representing the refined Markov equivalence class and a minimal residual set of DCCs. Polynomial-time algorithms are also provided for checking consistency and equivalence, as well as for finding the decomposed MPDAG and the residual DCCs. Finally, with pairwise causal background knowledge, we prove a sufficient and necessary condition to identify causal effects and surprisingly find that the identifiability of causal effects only depends on the decomposed MPDAG. We also develop a local IDA-type algorithm to estimate the possible values of an unidentifiable effect. Simulations suggest that pairwise causal background knowledge can significantly improve the identifiability of causal effects.

cs.AI↗
arXiv2022

Identification and estimation of causal effects in the presence of confounded principal strata

The principal stratification has become a popular tool to address a broad class of causal inference questions, particularly in dealing with non-compliance and truncation-by-death problems. The causal effects within principal strata which are determined by joint potential values of the intermediate variable, also known as the principal causal effects, are often of interest in these studies. Analyses of principal causal effects from observed data in the literature mostly rely on ignorability of the treatment assignment, which requires practitioners to accurately measure as many as covariates so that all possible confounding sources are captured. However, collecting all potential confounders in observational studies is often difficult and costly, the ignorability assumption may thus be questionable. In this paper, by leveraging available negative controls that have been increasingly used to deal with uncontrolled confounding, we consider identification and estimation of causal effects when the treatment and principal strata are confounded by unobserved variables. Specifically, we show that the principal causal effects can be nonparametrically identified by invoking a pair of negative controls that are both required not to directly affect the outcome. We then relax this assumption and establish identification of principal causal effects under various semiparametric or parametric models. We also propose an estimation method of principal causal effects. Extensive simulation studies show good performance of the proposed approach and a real data application from the National Longitudinal Survey of Young Men is used for illustration.

stat.ME↗
arXiv2021

A Local Method for Identifying Causal Relations under Markov Equivalence

Causality is important for designing interpretable and robust methods in artificial intelligence research. We propose a local approach to identify whether a variable is a cause of a given target under the framework of causal graphical models of directed acyclic graphs (DAGs). In general, the causal relation between two variables may not be identifiable from observational data as many causal DAGs encoding different causal relations are Markov equivalent. In this paper, we first introduce a sufficient and necessary graphical condition to check the existence of a causal path from a variable to a target in every Markov equivalent DAG. Next, we provide local criteria for identifying whether a variable is a cause/non-cause of a target based only on the local structure instead of the entire graph. Finally, we propose a local learning algorithm for this causal query via learning the local structure of the variable and some additional statistical independence tests related to the target. Simulation studies show that our local algorithm is efficient and effective, compared with other state-of-art methods.

stat.ML↗
arXiv2020

On Low Rank Directed Acyclic Graphs and Causal Structure Learning

Despite several advances in recent years, learning causal structures represented by directed acyclic graphs (DAGs) remains a challenging task in high dimensional settings when the graphs to be learned are not sparse. In this paper, we propose to exploit a low rank assumption regarding the (weighted) adjacency matrix of a DAG causal model to help address this problem. We utilize existing low rank techniques to adapt causal structure learning methods to take advantage of this assumption and establish several useful results relating interpretable graphical conditions to the low rank assumption. Specifically, we show that the maximum rank is highly related to hubs, suggesting that scale-free networks, which are frequently encountered in practice, tend to be low rank. Our experiments demonstrate the utility of the low rank adaptations for a variety of data models, especially with relatively large and dense graphs. Moreover, with a validation procedure, the adaptations maintain a superior or comparable performance even when graphs are not restricted to be low rank.

cs.LG↗
arXiv2016

Formulas for Counting the Sizes of Markov Equivalence Classes of Directed Acyclic Graphs

The sizes of Markov equivalence classes of directed acyclic graphs play important roles in measuring the uncertainty and complexity in causal learning. A Markov equivalence class can be represented by an essential graph and its undirected subgraphs determine the size of the class. In this paper, we develop a method to derive the formulas for counting the sizes of Markov equivalence classes. We first introduce a new concept of core graph. The size of a Markov equivalence class of interest is a polynomial of the number of vertices given its core graph. Then, we discuss the recursive and explicit formula of the polynomial, and provide an algorithm to derive the size formula via symbolic computation for any given core graph. The proposed size formula derivation sheds light on the relationships between the size of a Markov equivalence class and its representation graph, and makes size counting efficient, even when the essential graphs contain non-sparse undirected subgraphs.

stat.ML↗
arXiv2014

Reversible MCMC on Markov equivalence classes of sparse directed acyclic graphs

Graphical models are popular statistical tools which are used to represent dependent or causal complex systems. Statistically equivalent causal or directed graphical models are said to belong to a Markov equivalent class. It is of great interest to describe and understand the space of such classes. However, with currently known algorithms, sampling over such classes is only feasible for graphs with fewer than approximately 20 vertices. In this paper, we design reversible irreducible Markov chains on the space of Markov equivalent classes by proposing a perfect set of operators that determine the transitions of the Markov chain. The stationary distribution of a proposed Markov chain has a closed form and can be computed easily. Specifically, we construct a concrete perfect set of operators on sparse Markov equivalence classes by introducing appropriate conditions on each possible operator. Algorithms and their accelerated versions are provided to efficiently generate Markov chains and to explore properties of Markov equivalence classes of sparse directed acyclic graphs (DAGs) with thousands of vertices. We find experimentally that in most Markov equivalence classes of sparse DAGs, (1) most edges are directed, (2) most undirected subgraphs are small and (3) the number of these undirected subgraphs grows approximately linearly with the number of vertices. The article contains supplement arXiv:1303.0632, http://dx.doi.org/10.1214/13-AOS1125SUPP

stat.ML↗