arXiv subjects

Issa J. Dahabreh

Publications and source records attributed to Issa J. Dahabreh.

At least 19 recordsLinked to original sources

arXiv2026

Towards Best Practices for Covariate Adjustment in Regulatory Trials: From Fixed to Data-Adaptive Approaches

While randomization justifies the use of unadjusted effect estimators in randomized trials, there is growing interest in covariate adjustment to improve precision. Adjusting for baseline variables that are prognostic of the outcome can reduce estimator variance, resulting in narrower confidence intervals and increased statistical power. Recent guidance by the U.S. Food and Drug Administration supports fixed adjustment for prognostic covariates using parametric regression models. However, this guidance does not address more flexible approaches using data-adaptive or machine learning methods. We offer our perspectives on covariate adjustment to improve analytic precision. We focus on estimating the average effect for the target population in trials with minimal outcome missingness. We provide a non-technical overview of effect estimators that are unadjusted and effect estimators using fixed versus data-adaptive adjustment. We offer practical suggestions for conducting adjusted analyses that are data-adaptive, fully pre-specified, transparently and reproducibly implemented, robust to model misspecification, and guaranteed to improve precision relative to unadjusted analyses --- all while preserving statistical validity and the causal effect of interest. We hope that sharing our perspectives will foster broader discussion and eventual acceptance of principled, pre-specified, data-adaptive covariate adjustment in randomized trials.

stat.ME↗

arXiv2026

Causally-interpretable meta-analysis using aggregate data

Evidence syntheses and meta-analyses are used to inform clinical practice guidelines and health economic evaluations. However, heterogeneity of treatment effects poses a significant challenge. Conventional meta-analysis addresses heterogeneity through random-effect assumptions, which are not supported by design and lead to estimates that may not apply to any real-world population. Causally-interpretable meta-analysis (CIMA) offers a rigorous framework for specification, identification, and estimation of causal effects when combining information from multiple randomized trials. Initial development of CIMA focused on using individual data from randomized trials, but such data are often unavailable in practice. Here, we propose a new version of CIMA that only requires aggregate data from trials, addressing the limitations of traditional meta-analysis methods while relying only on aggregate data. The method leverages the trials' reported estimates of marginal and one-at-a-time subgroup treatment effects and descriptive statistics for baseline covariates to build moment equations for identifying and estimating a parametric conditional average treatment effect (CATE) function. The average treatment effect in a new target population is obtained by marginalizing the CATE function over the individual covariate data that defines the target population. The method can also be used to obtain causally-interpretable indirect treatment comparisons in the target population. We establish the asymptotic properties of the method, assess its finite-sample performance in simulation studies, and illustrate the application of the method by re-analyzing a published meta-analysis for SGLT2 inhibitors in patients with heart failure.

stat.ME↗

arXiv2026

Using the target trial framework for combining information: external comparator analyses and other applications

We describe how the target trial framework can be used to plan and report analyses that attempt to answer causal questions by combining information from multiple, diverse sources. Such analyses may involve comparisons of treatments evaluated in different populations, for example when an index trial is combined with other data sources in external comparator analyses, or when extending causal inferences from a randomized trial to a new target population in generalizability and transportability analyses. When planning such analyses, the specification of the target trial supports the explicit definition of the target population with an associated sampling model. We propose this as an additional component for the target trial framework, especially relevant for analyses that combine information, because it influences the choice of eligibility criteria, the specification of the causal model, the choice of causal contrasts, and reasoning about identification strategies. Furthermore, the framework encourages careful mapping of data elements from multiple data sources to a single target trial. This mapping process can highlight potentially irreconcilable misalignments between data sources with respect to specific components of the framework -- for example, in the definitions of eligibility criteria, treatment assignment, and treatment receipt. Such misalignments can arise when attempts to specify a target trial that aligns with a specific data source introduce or worsen misalignments with other proposed data sources. The extent of such misalignments may warrant switching to other data sources, or prospectively obtaining data, to emulate the proposed target trial. We conclude that the target trial framework promotes transparent discussion about the design of and assumptions made in analyses that answer causal questions by combining information from diverse sources.

stat.ME↗

arXiv2026

Generalizing conditional average treatment effects from nested randomized trials to all trial-eligible individuals

Randomized controlled trials often enroll participants whose characteristics differ from those of a target population, which can limit the generalizability of the estimated treatment effects when effect modifiers differ across populations. While existing generalizability methods primarily focus on estimating the average treatment effect (ATE) in the target population, such summaries may obscure important heterogeneity that is relevant for clinical and policy decision-making. In this work, we illustrate an approach for estimating the conditional average treatment effect (CATE) in a target population of trial-eligible individuals as a function of prespecified effect modifiers within a nested trial setting. Our approach combines semiparametric theory with flexible estimation: we first estimate nuisance functions using data-adaptive methods and construct pseudo-outcomes from conditional influence functions, then estimate the CATE function via local linear (kernel) regression. Sample splitting and cross-fitting are used to reduce overfitting bias and ensure asymptotic valid inference. Finite-sample performance is assessed via simulations and illustrated in the Coronary Artery Surgery Study (CASS).

stat.ME↗

arXiv2026

Identification strategies for combining an experimental study with external data

There is increasing interest in combining information from experimental studies, including randomized and single-group trials, with information from external experimental or observational data sources. Such efforts are usually motivated by the desire to compare treatments evaluated in different studies -- for instance, by constructing external comparator groups for some index study -- or to estimate treatment effects with greater precision. Proposals to combine experimental studies with external data were made at least as early as the 1970s, but in recent years have come under increasing consideration within clinical practice and by regulatory agencies involved in drug and device evaluation, particularly with the increasing availability of trial and observational data. In this paper, we describe basic study templates that combine information from experimental studies with external data, and use the potential (counterfactual) outcomes framework to elaborate identification strategies for potential outcome means and average treatment effects. We argue that these identification strategies inherit ideas relevant to the study of causation in single-source studies and the related literature on combining information (e.g., generalizability and transportability methods), but merit consideration as a separate class of causal problems because they differ in terms of their scientific motivations, definitions of the target population, sampling, data structures, and identifiability conditions. In formalizing identification strategies for the analyses described herein, we hope to provide a conceptual foundation to support the systematic use and evaluation of such efforts.

stat.ME↗

arXiv2026

Constructing external comparator groups via transportability in mean or in effect measure

Learning about causal effects in target populations and their subsets may be facilitated by combining information from multiple sources. One major class of study designs that combine information involves appending an index study with data from an external comparator, which may facilitate head-to-head comparisons of treatments initially studied in different populations. We delineate external comparator analyses under two distinct, but related, identification strategies. The first strategy relies on exchangeability (transportability) of potential outcome means, which uses information only on the treatments that are to be compared. The second strategy relies on transportability in effect measure, requiring additional use of information on a third treatment common to the populations that have been combined. In a time-fixed setting with a point treatment and non-failure time outcome, we examine identification and estimation under a basic setup where information from an index trial is combined with a second, and external to the index trial, data source. We propose estimators for identifying observed data functionals, with a particular focus on semiparametric efficient augmented weighting estimators that incorporate models for the probability of trial participation, the probability of treatment, and conditional outcome means. We derive the asymptotic properties of these augmented weighting estimators -- including robustness to model misspecification and slower rates of convergence for some nuisance function models -- and use simulation to compare their finite sample performance to estimators based only on outcome modeling or weighting. Last, we provide a practical demonstration of the proposed methods by combining the ACCEPT and PHOENIX 1 randomized trials to evaluate the effect of various biologic agents on plaque psoriasis, a chronic inflammatory disorder.

stat.ME↗

arXiv2026

Robust estimation of heterogeneous treatment effects in randomized trials leveraging external data

Randomized trials are typically designed to detect average treatment effects but often lack the statistical power to uncover individual-level treatment effect heterogeneity, limiting their value for personalized decision-making. To address this, we propose the QR-learner, a model-agnostic learner that estimates conditional average treatment effects (CATE) within the trial population by leveraging external data from other trials or observational studies. The proposed method is robust: it can reduce the mean squared error relative to a trial-only CATE learner, and is guaranteed to recover the true CATE even when the external data are not aligned with the trial. Moreover, we introduce a procedure that combines the QR-learner with a trial-only CATE learner and show that it asymptotically matches or exceeds both component learners in terms of mean squared error. We examine the performance of our approach in simulation studies and apply the methods to a real-world dataset, demonstrating improvements in both CATE estimation and statistical power for detecting heterogeneous effects.

stat.ML↗

arXiv2025

Efficient Randomized Experiments Using Foundation Models

Randomized experiments are the preferred approach for evaluating the effects of interventions, but they are costly and often yield estimates with substantial uncertainty. On the other hand, in silico experiments leveraging foundation models offer a cost-effective alternative that can potentially attain higher statistical precision. However, the benefits of in silico experiments come with a significant risk: statistical inferences are not valid if the models fail to accurately predict experimental responses to interventions. In this paper, we propose a novel approach that integrates the predictions from multiple foundation models with experimental data while preserving valid statistical inference. Our estimator is consistent and asymptotically normal, with asymptotic variance no larger than the standard estimator based on experimental data alone. Importantly, these statistical properties hold even when model predictions are arbitrarily biased. Empirical results across several randomized experiments show that our estimator offers substantial precision gains, equivalent to a reduction of up to 20% in the sample size needed to match the same precision as the standard estimator based on experimental data alone.

cs.LG↗

arXiv2025

The role of assignment in defining and identifying causal effects in randomized trials

In randomized trials, the per-protocol effect, that is, the effect of being assigned a treatment strategy and receiving treatment according to the assigned strategy, is sometimes thought to reflect the effect of the treatment strategy itself, without intervention on assignment. Here, we argue by example that this is not necessarily the case. We examine a causal structure for a randomized trial where these two causal estimands -- the per-protocol effect and the effect of the treatment strategy -- are not equal, and where their corresponding identifying observed data functionals are not the same, but both require information on assignment for identification. Our example highlights the conceptual difference between the per-protocol effect and the effect of the treatment strategy, the conditions under which these causal estimands are equal, and suggests that in some cases their identification requires information on assignment, even when assignment is randomized. Furthermore, both per-protocol effects and effects of treatment may be unidentifiable without information on treatment assignment, unless one makes additional assumptions -- informally, that assignment does not affect the outcome except through treatment (i.e., an exclusion-restriction assumption), and that assignment is not a confounder of the treatment-outcome association conditional on other variables in the analysis. Our analyses suggest a need to more clearly define the role of assignment when specifying causal effects of interest in randomized trials, which has implications for identification, analysis methods, and the interpretation of trial results.

stat.ME↗

arXiv2025

Robust integration of external control data in randomized trials

One approach for increasing the efficiency of randomized trials is the use of "external controls" -- individuals who received the control treatment studied in the trial during routine practice or in prior experimental studies. Existing external control methods, however, can be biased if the populations underlying the trial and the external control data are not exchangeable. Here, we characterize a randomization-aware class of treatment effect estimators in the population underlying the trial that remain consistent and asymptotically normal when using external control data, even when exchangeability does not hold. We consider two members of this class of estimators: the well-known augmented inverse probability weighting trial-only estimator, which is the efficient estimator when only trial data are used; and a potentially more efficient member of the class when exchangeability holds and external control data are available, which we refer to as the optimized randomization-aware estimator. To achieve robust integration of external control data in trial analyses, we then propose a combined estimator based on the efficient trial-only estimator and the optimized randomization-aware estimator. We show that the combined estimator is consistent and no less efficient than the most efficient of the two component estimators, whether the exchangeability assumption holds or not. We examine the estimators' performance in simulations and we illustrate their use with data from two trials of paliperidone extended-release for schizophrenia.

stat.ME↗

arXiv2025

Estimating and evaluating counterfactual prediction models

Counterfactual prediction methods are required when a model will be deployed in a setting where treatment policies differ from the setting where the model was developed, or when a model provides predictions under hypothetical interventions to support decision-making. However, estimating and evaluating counterfactual prediction models is challenging because, unlike traditional (factual) prediction, one does not observe the potential outcomes for all individuals under all treatment strategies of interest. Here, we discuss how to estimate a counterfactual prediction model, how to assess the model's performance, and how to perform model and tuning parameter selection. We provide identification and estimation results for counterfactual prediction models and for multiple measures of counterfactual model performance, including loss-based measures, the area under the receiver operating characteristic curve, and the calibration curve. Importantly, our results allow valid estimates of model performance under counterfactual intervention even if the candidate prediction model is misspecified, permitting a wider array of use cases. We illustrate these methods using simulation and apply them to the task of developing a statin-naive risk prediction model for cardiovascular disease.

stat.ME↗

arXiv2024

Interpretable meta-analysis of model or marker performance

Conventional meta analysis of model performance conducted using datasources from different underlying populations often result in estimates that cannot be interpreted in the context of a well defined target population. In this manuscript we develop methods for meta-analysis of several measures of model performance that are interpretable in the context of a well defined target population when the populations underlying the datasources used in the meta analysis are heterogeneous. This includes developing identifiablity conditions, inverse-weighting, outcome model, and doubly robust estimator. We illustrate the methods using simulations and data from two large lung cancer screening trials.

stat.ME↗

arXiv2024

Cluster randomized trials designed to support generalizable inferences

Background: When planning a cluster randomized trial, evaluators often have access to an enumerated cohort representing the target population of clusters. Practicalities of conducting the trial, such as the need to oversample clusters with certain characteristics to improve trial economy or to support inference about subgroups of clusters, may preclude simple random sampling from the cohort into the trial, and thus interfere with the goal of producing generalizable inferences about the target population. Methods: We describe a nested trial design where the randomized clusters are embedded within a cohort of trial-eligible clusters from the target population and where clusters are selected for inclusion in the trial with known sampling probabilities that may depend on cluster characteristics (e.g., allowing clusters to be chosen to facilitate trial conduct or to examine hypotheses related to their characteristics). We develop and evaluate methods for analyzing data from this design to generalize causal inferences to the target population underlying the cohort. Results: We present identification and estimation results for the expectation of the average potential outcome and for the average treatment effect, in the entire target population of clusters and in its non-randomized subset. In simulation studies we show that all the estimators have low bias but markedly different precision. Conclusions: Cluster randomized trials where clusters are selected for inclusion with known sampling probabilities that depend on cluster characteristics, combined with efficient estimation methods, can precisely quantify treatment effects in the target population, while addressing objectives of trial conduct that require oversampling clusters on the basis of their characteristics.

stat.ME↗

arXiv2024

Generalizing and transporting causal inferences from randomized trials in the presence of trial engagement effects

Trial engagement effects are effects of trial participation on the outcome that are not mediated by treatment assignment. Most work on extending (generalizing or transporting) causal inferences from a randomized trial to a target population has, explicitly or implicitly, assumed that trial engagement effects are absent, allowing evidence about the effects of the treatments examined in trials to be applied to non-experimental settings. Here, we define novel causal estimands and present identification results for generalizability and transportability analyses in the presence of trial engagement effects. Our approach allows for trial engagement effects under assumptions of no causal interaction between trial participation and treatment assignment on the absolute or relative scales. We show that under these assumptions, even in the presence of trial engagement effects, the trial data can be combined with covariate data from the target population to identify average treatment effects in the context of usual care as implemented in the target population (i.e., outside the experimental setting). The identifying observed data functionals under these no-interaction assumptions are the same as those obtained under the stronger identifiability conditions that have been invoked in prior work. Therefore, our results suggest a new interpretation for previously proposed generalizability and transportability estimators; this interpretation may be useful in analyses under causal structures where background knowledge suggests that trial engagement effects are present but interactions between trial participation and treatment are negligible.

stat.ME↗

arXiv2023

Sensitivity analysis for studies transporting prediction models

We consider the estimation of measures of model performance in a target population when covariate and outcome data are available on a sample from some source population and covariate data, but not outcome data, are available on a simple random sample from the target population. When outcome data are not available from the target population, identification of measures of model performance is possible under an untestable assumption that the outcome and population (source or target population) are independent conditional on covariates. In practice, this assumption is uncertain and, in some cases, controversial. Therefore, sensitivity analysis may be useful for examining the impact of assumption violations on inferences about model performance. Here, we propose an exponential tilt sensitivity analysis model and develop statistical methods to determine how sensitive measures of model performance are to violations of the assumption of conditional independence between outcome and population. We provide identification results and estimators for the risk in the target population, examine the large-sample properties of the estimators, and apply the estimators to data on individuals with stable ischemic heart disease.

stat.ME↗

arXiv2023

Generalizability analyses with a partially nested trial design: the Necrotizing Enterocolitis Surgery Trial

We discuss generalizability analyses under a partially nested trial design, where part of the trial is nested within a cohort of trial-eligible individuals, while the rest of the trial is not nested. This design arises, for example, when only some centers participating in a trial are able to collect data on non-randomized individuals, or when data on non-randomized individuals cannot be collected for the full duration of the trial. Our work is motivated by the Necrotizing Enterocolitis Surgery Trial (NEST) that compared initial laparotomy versus peritoneal drain for infants with necrotizing enterocolitis or spontaneous intestinal perforation. During the first phase of the study, data were collected from randomized individuals as well as consenting non-randomized individuals; during the second phase of the study, however, data were only collected from randomized individuals, resulting in a partially nested trial design. We propose methods for generalizability analyses with partially nested trial designs. We describe identification conditions and propose estimators for causal estimands in the target population of all trial-eligible individuals, both randomized and non-randomized, in the part of the data where the trial is nested, while using trial information spanning both parts. We evaluate the estimators in a simulation study.

stat.ME↗

arXiv2022

Generalizing and transporting inferences about the effects of treatment assignment subject to non-adherence

We discuss the identifiability of causal estimands for generalizability and transportability analyses, both under perfect and imperfect adherence to treatment assignment. We consider a setting where the trial data contain information on baseline covariates, assignment at baseline, intervention at baseline (point treatment), and outcomes; and where the data from non-randomized individuals only contain information on baseline covariates. In this setting, we review identification results under perfect adherence and study two examples in which non-adherence severely limits the ability to transport inferences about the effects of treatment assignment to the target population. In the first example, trial participation has a direct effect on treatment receipt and, through treatment receipt, on the outcome (a "trial engagement effect" via adherence). In the second example, participation in the trial has unmeasured common causes with treatment receipt. In both examples, the effect of assignment on the outcome in the target population is not identifiable. In the first example, however, the effect of joint interventions to scale-up trial activities that affect adherence and assign treatment is identifiable. We conclude that generalizability and transportability analyses should consider trial engagement effects via adherence and selection for participation on the basis of unmeasured factors that influence adherence.

stat.ME↗

arXiv2022

Robust Estimation of Loss-Based Measures of Model Performance under Covariate Shift

We present methods for estimating loss-based measures of the performance of a prediction model in a target population that differs from the source population in which the model was developed, in settings where outcome and covariate data are available from the source population but only covariate data are available on a simple random sample from the target population. Prior work adjusting for differences between the two populations has used various weighting estimators with inverse odds or density ratio weights. Here, we develop more robust estimators for the target population risk (expected loss) that can be used with data-adaptive (e.g., machine learning-based) estimation of nuisance parameters. We examine the large-sample properties of the estimators and evaluate finite sample performance in simulations. Last, we apply the methods to data from lung cancer screening using nationally representative data from the National Health and Nutrition Examination Survey (NHANES) and extend our methods to account for the complex survey design of the NHANES.

stat.ME↗