arXiv subjects

Elizabeth A. Stuart

Publications and source records attributed to Elizabeth A. Stuart.

At least 19 recordsLinked to original sources

arXiv2026

Causal Effects of Modified Treatment Policies under Positivity Violations: A Partial Identification Approach

Modified treatment policies (MTPs) are interventions based on each individual's natural treatment value. We study mean outcomes under MTPs for continuous treatments, including exposure mixtures. Positivity is the standard sufficient condition for identifying these mean outcomes without extrapolation: policy-generated values remain supported given covariates. With multivariate treatments or continuous covariates, treatment--covariate combinations can be sparse or unsupported. Retaining the policy, our partial-identification framework decomposes its mean outcome into a point-identified contribution inside a positivity region and one outside. We bound the latter by imposing Lipschitz continuity on conditional mean potential outcomes rather than a parametric extrapolation model. The restriction compares each outside mean with the mean at an anchor inside the region. Metric projection minimizes width among one-anchor intervals but concentrates anchors on a lower-dimensional boundary, making the endpoints not pathwise differentiable. Our novel interior-displaced projection moves anchors inward, restoring pathwise differentiability. With a known region, we derive influence functions, characterize when they are efficient, and obtain asymptotically normal estimators and confidence intervals. In simulations, our intervals attain at least nominal coverage where those assuming positivity undercover. In a pesticide-mixture application, protective associations suggested by methods assuming positivity are not robust to modest outcome variation beyond the estimated region.

stat.ME↗

arXiv2026

Federated Learning with Incomplete Data: When to Use Complete Cases and When to Weight

Privacy constraints have driven the rise of federated learning (FL), which enables multi-site analyses without sharing individual participant data. Existing FL estimators largely assume complete data, whereas multi-site studies often face missingness. We develop a framework for FL with missing data, identifying conditions under which the complete case (CC) estimator is preferred over the inverse probability weighting (IPW) estimator. For settings where the CC estimator leads to bias, we introduce a calibrated weight estimation approach that combines candidate weighting models across sites and remains consistent if at least one is correctly specified at each site; we further show that pooling many weighting candidate models with redundant information degrades the calibrated estimator, so a small set is preferable. Consistency conditions are stated at the site level, ensuring that the federated estimator inherits validity from site-level properties. We prove consistency and derive a sandwich variance estimator that accounts for uncertainty in the outcome model, and in both the estimated weighting models and the calibration step. Additionally, we show that all estimators require only one or a few communication rounds, making them practical under real-world data-governance constraints. We illustrate the framework by evaluating risk factors for 90-day mortality among patients with pleural infections treated with intrapleural enzyme therapy.

stat.ME↗

arXiv2026

State policy heterogeneity analyses: considerations and proposals

State-level policy studies often conduct heterogeneity analyses that quantify how treatment effects vary across state characteristics. These analyses may be used to inform state-specific policy decisions, or to infer how the effect of a policy changes in combination with other state characteristics. However, in state-level settings with varied contexts and policy landscapes, multiple versions of similar policies, and differential policy implementation, the causal quantities targeted by these analyses may not align with the inferential goals. This paper clarifies these issues by distinguishing several causal estimands relevant to heterogeneity analyses in state-policy settings, including state-specific treatment effects (ITE), conditional average treatment effects (CATE), and controlled direct effects (CDE). We argue that the CATE is often the easiest to identify and estimate, but may not be the most policy relevant target of inference. Moreover, the widespread practice of coarsening distinct policies or implementations into a single indicator further complicates the interpretation of these analyses. Motivated by these limitations, we propose bounding ITEs as an alternative inferential goal, yielding ranges for each state's policy effect under explicit assumptions that quantify deviations from the ideal identifying conditions. These bounds target a well-defined and policy-relevant quantity, the effect for specific states. We develop this approach within a difference-in-differences framework and discuss how sensitivity parameters may be informed using pre-treatment data. Through simulations we demonstrate that bounding state-specific effects can more reliably determine the sign of the ITEs than CATE estimates. We then illustrate this method to examine the effect of the Affordable Care Act Medicaid expansion on high-volume buprenorphine prescribing.

stat.ME↗

arXiv2025

Demystifying Proximal Causal Inference

Proximal causal inference (PCI) has emerged as a promising framework for identifying and estimating causal effects in the presence of unobserved confounders. While many traditional causal inference methods rely on the assumption of no unobserved confounding, this assumption is likely often violated. PCI addresses this challenge by relying on an alternative set of assumptions regarding the relationships between treatment, outcome, and auxiliary variables that serve as proxies for unmeasured confounders. We review existing identification results, discuss the assumptions necessary for valid causal effect estimation via PCI, and compare different PCI estimation methods. We offer practical guidance on operationalizing PCI, with a focus on selecting and evaluating proxy variables using domain knowledge, measurement error perspectives, and negative control analogies. Through conceptual examples, we demonstrate tensions in proxy selection and discuss the importance of clearly defining the unobserved confounding mechanism. By bridging formal results with applied considerations, this work aims to demystify PCI, encourage thoughtful use in practice, and identify open directions for methodological development and empirical research.

stat.ME↗

arXiv2025

Weight a Minute: Understanding Variability in PATE Estimates Across Target Populations

Clinical study populations often differ meaningfully from the broader populations to which results are intended to generalize. Weighting methods such as inverse probability of sampling weights (IPSW) reweight study participants to resemble a target population, but the accuracy of these estimates depends heavily on how well the chosen population represents the population of substantive interest. We conduct a simulation study grounded in empirical covariate distributions from several real-world data sources spanning a continuum from highly selective to broadly inclusive populations. Using treatment effect scenarios with varying levels of effect modification, we evaluate IPSW estimators of the population average treatment effect (PATE) across multiple candidate target populations. We quantify the bias that arises when the dataset used to operationalize the target population differs from the intended inference population, even when IPSW is correctly specified. Our results show that bias increases systematically as target populations diverge from a well-representative population, and that weighting to a poorly aligned target can introduce more bias than not weighting at all. These findings highlight that selecting an appropriate target population dataset is a critical design choice for valid generalization.

stat.ME↗

arXiv2025

Sensitivity Analysis when Generalizing Causal Effects from Multiple Studies to a Target Population: Motivation from the ECHO Program

Unobserved effect modifiers can induce bias when generalizing causal effect estimates to target populations. In this work, we extend a sensitivity analysis framework assessing the robustness of study results to unobserved effect modification that adapts to various generalizability scenarios, including multiple (conditionally) randomized trials, observational studies, or combinations thereof. This framework is interpretable and does not rely on distributional or functional assumptions about unknown parameters. We demonstrate how to leverage the multi-study setting to detect violation of the generalizability assumption through hypothesis testing, showing with simulations that the proposed test achieves high power under real-world sample sizes. Finally, we apply our sensitivity analysis framework to analyze the generalized effect estimate of secondhand smoke exposure on birth weight using cohort sites from the Environmental influences on Child Health Outcomes (ECHO) study.

stat.ME↗

arXiv2025

Large Language Models for Full-Text Methods Assessment: A Case Study on Mediation Analysis

Systematic reviews are crucial for synthesizing scientific evidence but remain labor-intensive, especially when extracting detailed methodological information. Large language models (LLMs) offer potential for automating methodological assessments, promising to transform evidence synthesis. Here, using causal mediation analysis as a representative methodological domain, we benchmarked state-of-the-art LLMs against expert human reviewers across 180 full-text scientific articles. Model performance closely correlated with human judgments (accuracy correlation 0.71; F1 correlation 0.97), achieving near-human accuracy on straightforward, explicitly stated methodological criteria. However, accuracy sharply declined on complex, inference-intensive assessments, lagging expert reviewers by up to 15%. Errors commonly resulted from superficial linguistic cues -- for instance, models frequently misinterpreted keywords like "longitudinal" or "sensitivity" as automatic evidence of rigorous methodological approache, leading to systematic misclassifications. Longer documents yielded lower model accuracy, whereas publication year showed no significant effect. Our findings highlight an important pattern for practitioners using LLMs for methods review and synthesis from full texts: current LLMs excel at identifying explicit methodological features but require human oversight for nuanced interpretations. Integrating automated information extraction with targeted expert review thus provides a promising approach to enhance efficiency and methodological rigor in evidence synthesis across diverse scientific fields.

cs.CL↗

arXiv2025

Precision Mental Health: Predicting Heterogeneous Treatment Effects for Depression through Data Integration

When treating depression, clinicians are interested in determining the optimal treatment for a given patient, which is challenging given the amount of treatments available. To advance individualized treatment allocation, integrating data across multiple randomized controlled trials (RCTs) can enhance our understanding of treatment effect heterogeneity by increasing available information. However, extending these inferences to individuals outside of the original RCTs remains crucial for clinical decision-making. We introduce a two-stage meta-analytic method that predicts conditional average treatment effects (CATEs) in target patient populations by leveraging the distribution of CATEs across RCTs. Our approach generates 95\% prediction intervals for CATEs in target settings using first-stage models that can incorporate parametric regression or non-parametric methods such as causal forests or Bayesian additive regression trees (BART). We validate our method through simulation studies and operationalize it to integrate multiple RCTs comparing depression treatments, duloxetine and vortioxetine, to generate prediction intervals for target patient profiles. Our analysis reveals no strong evidence of effect heterogeneity across trials, with the exception of potential age-related variability. Importantly, we show that CATE prediction intervals capture broader uncertainty than study-specific confidence intervals when warranted, reflecting both within-study and between-study variability.

stat.AP↗

arXiv2025

Transporting results from a trial to an external target population when trial participation impacts adherence

Randomized clinical trials are considered the gold standard for informing treatment guidelines, but results may not generalize to real-world populations. Generalizability is hindered by distributional differences in baseline covariates and treatment-outcome mediators. Approaches to address differences in covariates are well established, but approaches to address differences in mediators are more limited. Here we consider the setting where trial activities that differ from usual care settings (e.g., monetary compensation, follow-up visits frequency) affect treatment adherence. When treatment and adherence data are unavailable for the real-world target population, we cannot identify the mean outcome under a specific treatment assignment (i.e., mean potential outcome) in the target. Therefore, we propose a sensitivity analysis in which a parameter for the relative difference in adherence to a specific treatment between the trial and the target, possibly conditional on covariates, must be specified. We discuss options for specification of the sensitivity analysis parameter based on external knowledge including setting a range to estimate bounds or specifying a probability distribution from which to repeatedly draw parameter values (i.e., use Monte Carlo sampling). We introduce two estimators for the mean counterfactual outcome in the target that incorporates this sensitivity parameter, a plug-in estimator and a one-step estimator that is double robust and supports the use of machine learning for estimating nuisance models. Finally, we apply the proposed approach to the motivating application where we transport the risk of relapse under two different medications for the treatment of opioid use disorder from a trial to a real-world population.

stat.ME↗

arXiv2025

A Cautionary Tale on Integrating Studies with Disparate Outcome Measures for Causal Inference

Data integration approaches are increasingly used to enhance the efficiency and generalizability of studies. However, a key limitation of these methods is the assumption that outcome measures are identical across datasets -- an assumption that often does not hold in practice. Consider the following opioid use disorder (OUD) studies: the XBOT trial and the POAT study, both evaluating the effect of medications for OUD on withdrawal symptom severity (not the primary outcome of either trial). While XBOT measures withdrawal severity using the subjective opiate withdrawal scale, POAT uses the clinical opiate withdrawal scale. We analyze this realistic yet challenging setting where outcome measures differ across studies and where neither study records both types of outcomes. Our paper studies whether and when integrating studies with disparate outcome measures leads to efficiency gains. We introduce three sets of assumptions -- with varying degrees of strength -- linking both outcome measures. Our theoretical and empirical results highlight a cautionary tale: integration can improve asymptotic efficiency only under the strongest assumption linking the outcomes. However, misspecification of this assumption leads to bias. In contrast, a milder assumption may yield finite-sample efficiency gains, yet these benefits diminish as sample size increases. We illustrate these trade-offs via a case study integrating the XBOT and POAT datasets to estimate the comparative effect of two medications for opioid use disorder on withdrawal symptoms. By systematically varying the assumptions linking the SOW and COW scales, we show potential efficiency gains and the risks of bias. Our findings emphasize the need for careful assumption selection when fusing datasets with differing outcome measures, offering guidance for researchers navigating this common challenge in modern data integration.

stat.ME↗

arXiv2025

Choosing an analytic approach: Key study design considerations in state policy evaluation

This paper reviews and details methods for state policy evaluation to guide selection of a research approach based on evaluation setting and available data. We highlight key design considerations for an analysis, including treatment and control group selection, timing of policy adoption, expected effect heterogeneity, and data considerations. We then provide an overview of analytic approaches and differentiate between methods based on evaluation context, such as settings with no control units, a single treated unit, multiple treated units, or with multiple treatment cohorts. Methods discussed include interrupted time series models, difference-in-differences estimators, autoregressive models, and synthetic control methods, along with method extensions which address issues like staggered policy adoption and heterogenous treatment effects. We end with an illustrative example, applying the developed framework to evaluate the impacts of state-level naloxone standing order policies on overdose rates. Overall, we provide researchers with an approach for deciding on methods for state policy evaluations, which can be used to select study designs and inform methodological choices.

stat.ME↗

arXiv2025

Assessing Bias and Precision in State Policy Evaluations: A Comparative Analysis of Time-Varying Estimators Using Policy Simulations

Using state-level opioid overdose mortality data from 1999-2016, we simulated four time-varying treatment scenarios, which correspond to real-world policy dynamics (ramp up, ramp down, temporary and inconsistent). We then evaluated seven commonly used policy evaluation methods: two-way fixed effects event study, debiased autoregressive model, augmented synthetic control, difference-in-differences with staggered adoption, event study with heterogeneous treatment, two-stage differences-in-differences and differences-in-differences imputation. Statistical performance was assessed by comparing bias, standard errors, coverage, and root mean squared error over 1,000 simulations. Results Our findings indicate that estimator performance varied across policy scenarios. In settings where policy effectiveness diminished over time, synthetic control methods recovered effects with lower bias and higher variance. Difference-in-difference approaches, while offering reasonable coverage under some scenarios, struggled when effects were non-monotonic. Autoregressive methods, although demonstrating lower variability, underestimated uncertainty. Overall, a clear bias-variance tradeoff emerged, underscoring that no single method uniformly excelled across scenarios. Conclusions This study highlights the importance of tailoring the choice of estimator to the expected trajectory of policy effects. In dynamic time-varying settings, particularly when a policy has an anticipated diminishing impact, methods like augmented synthetic controls may offer advantages despite reduced precision. Researchers should carefully consider these tradeoffs to ensure robust and credible state-policy evaluations.

stat.ME↗

arXiv2024

Self-separated and self-connected models for mediator and outcome missingness in mediation analysis

Missing data is a common challenge in studying treatment effects. In the context of mediation analysis, this paper addresses missingness in the mediator and outcome, focusing on identification. We first consider self-separated missingness models where identification is achieved by conditional independence assumptions. This model class is somewhat limited as it is constrained by the need to remove a certain number of connections from the model. We then turn to self-connected missingness models where identification relies on information from shadow variables. This model class turns out to contain substantial variation, allowing models with built-in shadow variables (mediator, outcome or covariates) and models with auxiliary shadow variables at different positions in the causal structure. To improve the practical value of the missingness mechanisms, we allow where possible for dependencies due to unobserved causes of the missingness, a feature often neglected. In this exploration, we review existing models, connect to new models, and develop theory where needed. This results in templates for identification in the mediation setting, generally useful identification techniques, and perhaps most importantly a synthesis and substantial extension of shadow variable theory. Two examples relate the models to practical considerations.

stat.ME↗

arXiv2024

Statistical methods to estimate the impact of gun policy on gun violence

Gun violence is a critical public health and safety concern in the United States. There is considerable variability in policy proposals meant to curb gun violence, ranging from increasing gun availability to deter potential assailants (e.g., concealed carry laws or arming school teachers) to restricting access to firearms (e.g., universal background checks or banning assault weapons). Many studies use state-level variation in the enactment of these policies in order to quantify their effect on gun violence. In this paper, we discuss the policy trial emulation framework for evaluating the impact of these policies, and show how to apply this framework to estimating impacts via difference-in-differences and synthetic controls when there is staggered adoption of policies across jurisdictions, estimating the impacts of right-to-carry laws on violent crime as a case study.

stat.AP↗

arXiv2024

Practical challenges in mediation analysis: A guide for applied researchers

Mediation analysis is a statistical approach that can provide insights regarding the intermediary processes by which an intervention or exposure affects a given outcome. Mediation analyses rose to prominence, particularly in social science research, with the publication of the seminal paper by Baron and Kenny and is now commonly applied in many research disciplines, including health services research. Despite the growth in popularity, applied researchers may still encounter challenges in terms of conducting mediation analyses in practice. In this paper, we provide an overview of conceptual and methodological challenges that researchers face when conducting mediation analyses. Specifically, we discuss the following key challenges: (1) Conceptually differentiating mediators from other third variables, (2) Extending beyond the single mediator context, (3) Identifying appropriate datasets in which measurement and temporal ordering supports the hypothesized mediation model, (4) Selecting mediation effects that reflect the scientific question of interest, (5) Assessing the validity of underlying assumptions of no omitted confounders, (6) Addressing measurement error regarding the mediator, and (7) Clearly reporting results from mediation analyses. We discuss each challenge and highlight ways in which the applied researcher can approach these challenges.

stat.AP↗

arXiv2023

Identification of complier and noncomplier average causal effects in the presence of latent missing-at-random (LMAR) outcomes: a unifying view and choices of assumptions

The study of treatment effects is often complicated by noncompliance and missing data. In the one-sided noncompliance setting where of interest are the complier and noncomplier average causal effects (CACE and NACE), we address outcome missingness of the \textit{latent missing at random} type (LMAR, also known as \textit{latent ignorability}). That is, conditional on covariates and treatment assigned, the missingness may depend on compliance type. Within the instrumental variable (IV) approach to noncompliance, methods have been proposed for handling LMAR outcome that additionally invoke an exclusion restriction type assumption on missingness, but no solution has been proposed for when a non-IV approach is used. This paper focuses on effect identification in the presence of LMAR outcome, with a view to flexibly accommodate different principal identification approaches. We show that under treatment assignment ignorability and LMAR only, effect nonidentifiability boils down to a set of two connected mixture equations involving unidentified stratum-specific response probabilities and outcome means. This clarifies that (except for a special case) effect identification generally requires two additional assumptions: a \textit{specific missingness mechanism} assumption and a \textit{principal identification} assumption. This provides a template for identifying effects based on separate choices of these assumptions. We consider a range of specific missingness assumptions, including those that have appeared in the literature and some new ones. Incidentally, we find an issue in the existing assumptions, and propose a modification of the assumptions to avoid the issue. Results under different assumptions are illustrated using data from the Baltimore Experience Corps Trial.

stat.ME↗

arXiv2023

Shared Control Individuals in Health Policy Evaluations with Application to Medical Cannabis Laws

Health policy researchers often have questions about the effects of a policy implemented at some cluster-level unit, e.g., states, counties, hospitals, etc. on individual-level outcomes collected over multiple time periods. Stacked difference-in-differences is an increasingly popular way to estimate these effects. This approach involves estimating treatment effects for each policy-implementing unit, then, if scientifically appropriate, aggregating them to an average effect estimate. However, when individual-level data are available and non-implementing units are used as comparators for multiple policy-implementing units, data from untreated individuals may be used across multiple analyses, thereby inducing correlation between effect estimates. Existing methods do not quantify or account for this sharing of controls. Here, we describe a stacked difference-in-differences study investigating the effects of state medical cannabis laws on treatment for chronic pain management that motivated this work, discuss a framework for estimating and managing this correlation due to shared control individuals, and show how accounting for it affects the substantive results.

stat.ME↗

arXiv2023

Identifying Optimal Methods for Addressing Confounding Bias When Estimating the Effects of State-Level Policies

Background: Policy evaluation studies that assess how state-level policies affect health-related outcomes are foundational to health and social policy research. The relative ability of newer analytic methods to address confounding, a key source of bias in observational studies, has not been closely examined. Methods: We conducted a simulation study to examine how differing magnitudes of confounding affected the performance of four methods used for policy evaluations: (1) the two-way fixed effects (TWFE) difference-in-differences (DID) model; (2) a one-period lagged autoregressive (AR) model; (3) augmented synthetic control method (ASCM); and (4) the doubly robust DID approach with multiple time periods from Callaway-Sant'Anna (CSA). We simulated our data to have staggered policy adoption and multiple confounding scenarios (i.e., varying the magnitude and nature of confounding relationships). Results: Bias increased for each method: (1) as confounding magnitude increases; (2) when confounding is generated with respect to prior outcome trends (rather than levels), and (3) when confounding associations are nonlinear (rather than linear). The AR and ASCM have notably lower root mean squared error than the TWFE model and CSA approach for all scenarios; the exception is nonlinear confounding by prior trends, where CSA excels. Coverage rates are unreasonably high for ASCM (e.g., 100%), reflecting large model-based standard errors and wide confidence intervals in practice. Conclusions: Our simulation study indicated that no single method consistently outperforms the others. But a researcher's toolkit should include all methodological options. Our simulations and associated R package can help researchers choose the most appropriate approach for their data.

stat.ME↗