arXiv subjects

John B. Carlin

Publications and source records attributed to John B. Carlin.

13 recordsLinked to original sources

arXiv2026

On the use of auxiliary variables in multiple imputation when estimating the average causal effect with missing data

Estimating the average causal effect (ACE) using observational data is a key focus in causal inference for which missing data present an important challenge. Multiple imputation (MI) is a widely used method for handling missing data and can yield unbiased estimates when the imputation is compatible with the substantive analysis. One of the advantages of MI is its scope to include so-called "auxiliary variables", defined as variables associated with incomplete variables that are excluded from the substantive analysis. Although many studies have looked at the use of auxiliary variables in MI for improving precision, the study of auxiliary variables that are necessary for the identifiability (or "recoverability") of the ACE in the presence of missing data has been scant. In this work, we investigate the use of auxiliary variables, both mediators and non-mediators, across a range of typical univariable and multivariable missingness mechanisms depicted by missingness directed acyclic graphs (m-DAGs). For each setting, we derive recoverability results, then evaluate MI-based and complete-case methods for estimating the ACE using correctly specified g-computation, considering different strategies for incorporating auxiliary variables and varying degrees of compatibility for MI models. Based on findings from the simulation studies, we provide practical guidance, highlighting that distinguishing appropriately between mediator and non-mediator auxiliary variables is important to avoid bias as is the use of compatible and flexible (non-parametric) MI methods that incorporate these variables.

stat.ME↗

arXiv2026

A roadmap for systematic identification and analysis of multiple biases in causal inference

Observational studies examining causal effects rely on unverifiable assumptions, the violation of which can induce multiple biases. Quantitative bias analysis (QBA) methods examine the sensitivity of findings to such violations, generally, by producing estimates under alternative assumptions, incorporating external information. Although substantial guidance exists for implementing QBA, there is limited guidance on how to systematically determine the assumptions underlying a primary causal analysis and the potential violations that should guide bias analysis. Consequently, many assumptions remain implicit, leading to selective and therefore misleading QBA. To address this gap, we propose a roadmap for systematically identifying and analysing multiple biases. Briefly, this consists of (1) articulating the assumptions underlying the primary analysis through specification and emulation of the ideal trial that defines the causal estimand and depicting these assumptions using a causal diagram; (2) extending the diagram to depict alternative assumptions under which biases may arise; (3) obtaining a single estimate that simultaneously corrects for all potential biases. We illustrate the roadmap using an investigation of the effect of breastfeeding on risk of childhood asthma, and through simulations illustrate the need for analysing multiple biases jointly rather than one at a time.

stat.ME↗

arXiv2025

The ideal trial: defining causal estimands that balance relevance and feasibility in target trial emulations and actual randomized trials

Causal inference is the goal of randomized trials and many observational studies. The first step in a formal causal inference framework is to define the causal estimand, and in both types of study this can be intuitively defined as the effect in an ideal trial: a hypothetical perfect randomized experiment (with representative sample, perfect adherence, etc.). The target trial framework is increasingly used for causal inference in observational studies, but clarity is lacking in how a target trial should be specified and how it relates to an ideal trial. In this paper, we review the concept of the ideal trial and highlight the need to balance relevance for decision-making in the real world and feasibility of estimation when specifying it. We then consider the question of how a target trial should be specified, outlining the challenges of a recommended approach, commonly seen in applications, that puts the focus heavily on feasibility of estimation: to specify the target trial such that it is closely aligned with the observational data (e.g. uses the same eligibility criteria). We argue that with this "aligned" approach, biases may remain relative to the estimand of ultimate practical interest, defined by the ideal trial, which mirror the often-overlooked biases of actual trials. We conclude that consideration of the ideal trial and of how the target trial and its emulation or the actual trial differ from it is necessary to identify and manage all bias sources in both settings. An example from respiratory epidemiology is used for illustration.

stat.ME↗

arXiv2025

Causal machine learning methods and use of cross-fitting in settings with high-dimensional confounding

Observational epidemiological studies commonly seek to estimate the causal effect of an exposure on an outcome. Adjustment for potential confounding bias in modern studies is challenging due to the presence of high-dimensional confounding, which occurs when there are many confounders relative to sample size or complex relationships between continuous confounders and exposure and outcome. Doubly robust methods such as Augmented Inverse Probability Weighting (AIPW) and Targeted Maximum Likelihood Estimation (TMLE) have the potential to address these challenges, using data-adaptive approaches and cross-fitting, but despite recent advances limited evaluation and guidance are available on their implementation in realistic settings where high-dimensional confounding is present. Motivated by an early-life cohort study, we conducted an extensive simulation study to compare the relative performance of AIPW and TMLE using data-adaptive approaches for estimating the average causal effect (ACE). We evaluated the benefits of using cross-fitting with a varying number of folds, as well as the impact of using a reduced versus full (larger, more diverse) library in the Super Learner ensemble learning approach used for implementation. We found that AIPW and TMLE performed similarly in most cases for estimating the ACE, but TMLE was more stable. Cross-fitting improved the performance of both methods, but was more important for variance estimation and coverage than for point estimates, with the number of folds a less important consideration. Using a full Super Learner library was important to reduce bias and variance in complex scenarios typical of modern health research studies.

stat.ME↗

arXiv2025

Handling multivariable missing data in causal mediation analysis estimating interventional effects

The interventional effects approach to causal mediation analysis is increasingly common in epidemiologic research, given its potential to address policy-relevant questions about hypothetical mediator interventions. Multiple imputation (MI) is widely used for handling missing data in epidemiologic studies. However, guidance is lacking on best practices for using MI when estimating interventional mediation effects, specifically regarding the role of the missingness mechanism in the method's performance, how to appropriately specify the MI model when g-computation is used for effect estimation, and suitable approaches to variance estimation. To address this gap, we conducted simulations based on the Victorian Adolescent Health Cohort Study. We considered seven missingness mechanisms involving varying assumptions about the influence of an intermediate confounder, a mediator, and/or the outcome on missingness in key variables. We compared the performance of complete-case analysis, six MI approaches using fully conditional specification (differing in how the imputation model was tailored), and a "substantive model compatible" multiple imputation-fully conditional specification approach. We evaluated MIBoot (MI, then bootstrap) and BootMI (bootstrap, then MI) approaches for variance estimation. All MI approaches, apart from those clearly diverging from best practice, yielded approximately unbiased estimates when none of the intermediate confounder, mediator, and outcome variables influenced missingness in any of these variables, and showed non-negligible bias otherwise. We observed the largest bias for interventional effects when each of the intermediate confounders, mediators, and outcomes influenced their own missingness. BootMI returned variance estimates with smaller bias than MIBoot.

stat.AP↗

arXiv2024

Sensitivity analysis methods for outcome missingness using substantive-model-compatible multiple imputation and their application in causal inference

When using multiple imputation (MI) for missing data, maintaining compatibility between the imputation model and substantive analysis is important for avoiding bias. For example, some causal inference methods incorporate an outcome model with exposure-confounder interactions that must be reflected in the imputation model. Two approaches for compatible imputation with multivariable missingness have been proposed: Substantive-Model-Compatible Fully Conditional Specification (SMCFCS) and a stacked-imputation-based approach (SMC-stack). If the imputation model is correctly specified, both approaches are guaranteed to be unbiased under the "missing at random" assumption. However, this assumption is violated when the outcome causes its own missingness, which is common in practice. In such settings, sensitivity analyses are needed to assess the impact of alternative assumptions on results. An appealing solution for sensitivity analysis is delta-adjustment using MI, specifically "not-at-random" (NAR)FCS. However, the issue of imputation model compatibility has not been considered in sensitivity analysis, with a naive implementation of NARFCS being susceptible to bias. To address this gap, we propose two approaches for compatible sensitivity analysis when the outcome causes its own missingness. The proposed approaches, NAR-SMCFCS and NAR-SMC-stack, extend SMCFCS and SMC-stack, respectively, with delta-adjustment for the outcome. We evaluate these approaches using a simulation study that is motivated by a case study, to which the methods were also applied. The simulation results confirmed that a naive implementation of NARFCS produced bias in effect estimates, while NAR-SMCFCS and NAR-SMC-stack were approximately unbiased. The proposed compatible approaches provide promising avenues for conducting sensitivity analysis to missingness assumptions in causal inference.

stat.ME↗

arXiv2024

On the uses and abuses of regression models: a call for reform of statistical practice and teaching

Regression methods dominate the practice of biostatistical analysis, but biostatistical training emphasises the details of regression models and methods ahead of the purposes for which such modelling might be useful. More broadly, statistics is widely understood to provide a body of techniques for "modelling data", underpinned by what we describe as the "true model myth": that the task of the statistician/data analyst is to build a model that closely approximates the true data generating process. By way of our own historical examples and a brief review of mainstream clinical research journals, we describe how this perspective has led to a range of problems in the application of regression methods, including misguided "adjustment" for covariates, misinterpretation of regression coefficients and the widespread fitting of regression models without a clear purpose. We then outline a new approach to the teaching and application of biostatistical methods, which situates them within a framework that first requires clear definition of the substantive research question at hand within one of three categories: descriptive, predictive, or causal. Within this approach, the development and application of (multivariable) regression models, as well as other advanced biostatistical methods, should proceed differently according to the type of question. Regression methods will no doubt remain central to statistical practice as they provide a powerful tool for representing variation in a response or outcome variable as a function of "input" variables, but their conceptualisation and usage should follow from the purpose at hand.

stat.ME↗

arXiv2024

Handling missing data when estimating causal effects with Targeted Maximum Likelihood Estimation

Targeted Maximum Likelihood Estimation (TMLE) is increasingly used for doubly robust causal inference, but how missing data should be handled when using TMLE with data-adaptive approaches is unclear. Based on the Victorian Adolescent Health Cohort Study, we conducted a simulation study to evaluate eight missing data methods in this context: complete-case analysis, extended TMLE incorporating outcome-missingness model, missing covariate missing indicator method, five multiple imputation (MI) approaches using parametric or machine-learning models. Six scenarios were considered, varying in exposure/outcome generation models (presence of confounder-confounder interactions) and missingness mechanisms (whether outcome influenced missingness in other variables and presence of interaction/non-linear terms in missingness models). Complete-case analysis and extended TMLE had small biases when outcome did not influence missingness in other variables. Parametric MI without interactions had large bias when exposure/outcome generation models included interactions. Parametric MI including interactions performed best in bias and variance reduction across all settings, except when missingness models included a non-linear term. When choosing a method to handle missing data in the context of TMLE, researchers must consider the missingness mechanism and, for MI, compatibility with the analysis method. In many settings, a parametric MI approach that incorporates interactions and non-linearities is expected to perform well.

stat.ME↗

arXiv2023

Recoverability and estimation of causal effects under typical multivariable missingness mechanisms

In the context of missing data, the identifiability or "recoverability" of the average causal effect (ACE) depends on causal and missingness assumptions. The latter can be depicted by adding variable-specific missingness indicators to causal diagrams, creating "missingness-directed acyclic graphs" (m-DAGs). Previous research described ten canonical m-DAGs, representing typical multivariable missingness mechanisms in epidemiological studies, and determined the recoverability of the ACE in the absence of effect modification. We extend the research by determining the recoverability of the ACE in settings with effect modification and conducting a simulation study evaluating the performance of widely used missing data methods when estimating the ACE using correctly specified g-computation, which has not been previously studied. Methods assessed were complete case analysis (CCA) and various multiple imputation (MI) implementations regarding the degree of compatibility with the outcome model used in g-computation. Simulations were based on an example from the Victorian Adolescent Health Cohort Study (VAHCS), where interest was in estimating the ACE of adolescent cannabis use on mental health in young adulthood. In the canonical m-DAGs that excluded unmeasured common causes of missingness indicators, we derived the recoverable ACE if no incomplete variable causes its missingness, and non-recoverable otherwise. Besides, the simulation showed that compatible MI approaches may enable approximately unbiased ACE estimation, unless the outcome causes its missingness or it causes the missingness of a variable that causes its missingness. Researchers must consider sensitivity analysis methods incorporating external information in the latter setting. The VAHCS case study illustrates the practical implications of these findings.

stat.ME↗

arXiv2022

Evaluation of multiple imputation to address intended and unintended missing data in case-cohort studies with a binary endpoint

Case-cohort studies are conducted within cohort studies, wherein collection of exposure data is limited to a subset of the cohort, leading to a large proportion of missing data by design. Standard analysis uses inverse probability weighting (IPW) to address this intended missing data, but little research has been conducted into how best to perform analysis when there is also unintended missingness. Multiple imputation (MI) has become a default standard for handling unintended missingness, but when used in combination with IPW, the imputation model needs to take account of the weighting to ensure compatibility with the analysis model. Alternatively, MI could be used to handle both the intended and unintended missingness. While the performance of a solely MI approach has been investigated in the context of a case-cohort study with a time-to-event outcome, it is unclear how this approach performs with binary outcomes. We conducted a simulation study to assess and compare the performance of approaches using only MI, only IPW, and a combination of MI and IPW, for handling intended and unintended missingness in this setting. We also applied the approaches to a case study. Our results show that the combined approach is approximately unbiased for estimation of the exposure effect when the sample size is large, and was the least biased with small sample sizes, while MI-only or IPW-only exhibited larger biases in both sample size settings. These findings suggest that MI is the preferred approach to handle intended and unintended missing data in case-cohort studies with binary outcomes.

stat.ME↗

arXiv2022

A comparison of strategies for selecting auxiliary variables for multiple imputation

Multiple imputation (MI) is a popular method for handling missing data. Auxiliary variables can be added to the imputation model(s) to improve MI estimates. However, the choice of which auxiliary variables to include in the imputation model is not always straightforward. Including too few may lead to important information being discarded, but including too many can cause problems with convergence of the estimation procedures for imputation models. Several data-driven auxiliary variable selection strategies have been proposed. This paper uses a simulation study and a case study to provide a comprehensive comparison of the performance of eight auxiliary variable selection strategies, with the aim of providing practical advice to users of MI. A complete case analysis and an MI analysis with all auxiliary variables included in the imputation model (the full model) were also performed for comparison. Our simulation study results suggest that the full model outperforms all auxiliary variable selection strategies, providing further support for adopting an inclusive auxiliary variable strategy where possible. Auxiliary variable selection using the Least Absolute Selection and Shrinkage Operator (LASSO) was the best performing auxiliary variable selection strategy overall and is a promising alternative when the full model fails. All MI analysis strategies that we were able to apply to the case study led to similar estimates.

stat.ME↗

arXiv2020

Evaluation of approaches for accommodating interactions and non-linear terms in multiple imputation of incomplete three-level data

Three-level data structures arising from repeated measures on individuals clustered within larger units are common in health research studies. Missing data are prominent in such studies and are often handled via multiple imputation (MI). Although several MI approaches can be used to account for the three-level structure, including adaptations to single- and two-level approaches, when the substantive analysis model includes interactions or quadratic effects these too need to be accommodated in the imputation model. In such analyses, substantive model compatible (SMC) MI has shown great promise in the context of single-level data. While there have been recent developments in multilevel SMC MI, to date only one approach that explicitly handles incomplete three-level data is available. Alternatively, researchers can use pragmatic adaptations to single- and two-level MI approaches, or two-level SMC-MI approaches. We describe the available approaches and evaluate them via simulation in the context of a three three-level random effects analysis models involving an interaction between the incomplete time-varying exposure and time, an interaction between the time-varying exposure and an incomplete time-fixed confounder, or a quadratic effect of the exposure. Results showed that all approaches considered performed well in terms of bias and precision when the target analysis involved an interaction with time, but the three-level SMC MI approach performed best when the target analysis involved an interaction between the time-varying exposure and an incomplete time-fixed confounder, or a quadratic effect of the exposure. We illustrate the methods using data from the Childhood to Adolescence Transition Study.

stat.ME↗

arXiv2017

Bayesian modelling of lung function data from multiple-breath washout tests

Paediatric respiratory researchers have widely adopted the multiple-breath washout (MBW) test because it allows assessment of lung function in unsedated infants and is well suited to longitudinal studies of lung development and disease. However, a substantial proportion of MBW tests in infants fail current acceptability criteria. We hypothesised that a model-based approach to analysing the data, in place of traditional simple empirical summaries, would enable more efficient use of these tests. We therefore developed a novel statistical model for infant MBW data and applied it to 1,197 tests from 432 individuals from a large birth cohort study. We focus on Bayesian estimation of the lung clearance index (LCI), the most commonly used summary of lung function from MBW tests. Our results show that the model provides an excellent fit to the data and shed further light on statistical properties of the standard empirical approach. Furthermore, the modelling approach enables LCI to be estimated using tests with different degrees of completeness, something not possible with the standard approach. Our model therefore allows previously unused data to be used rather than discarded, as well as routine use of shorter tests without significant loss of precision. Beyond our specific application, our work illustrates a number of important aspects of Bayesian modelling in practice, such as the importance of hierarchical specifications to account for repeated measurements and the value of model checking via posterior predictive distributions.

stat.AP↗