arXiv subjects

Ian R. White

Publications and source records attributed to Ian R. White.

15 recordsLinked to original sources

arXiv2026

Introducing precision-weighted bias as a performance measure to inform the inclusion of adaptive designs in meta-analysis

We propose a novel, intuitive measure of statistical performance: precision-weighted bias. Precision-weighted bias is defined as the unconditional bias of an estimator weighted by the degree of information (precision) it contains. Current guidelines, such as GRADE and CONSORT, often view the potential for increased bias in adaptive designs as a deterrent for the inclusion of such designs in systematic reviews. However, we demonstrate that the bias in a common-effect meta-analysis is approximately equal to the precision-weighted average of the precision-weighted biases of its constituent studies, rather than of their unweighted unconditional biases. Through simulation studies, we show that while adaptive designs may exhibit unweighted bias, they frequently have zero precision-weighted bias. Consequently, including these designs often results in a negligible change to the overall meta-analysis bias. These results suggest that precision-weighted bias is a superior indicator for determining whether to include an adaptive design in a meta-analysis. We recommend that precision-weighted bias be used as a standard complement to unweighted unconditional and conditional bias in simulation studies to support more inclusive and accurate evidence synthesis.

stat.ME↗

arXiv2024

Handling missing data when estimating causal effects with Targeted Maximum Likelihood Estimation

Targeted Maximum Likelihood Estimation (TMLE) is increasingly used for doubly robust causal inference, but how missing data should be handled when using TMLE with data-adaptive approaches is unclear. Based on the Victorian Adolescent Health Cohort Study, we conducted a simulation study to evaluate eight missing data methods in this context: complete-case analysis, extended TMLE incorporating outcome-missingness model, missing covariate missing indicator method, five multiple imputation (MI) approaches using parametric or machine-learning models. Six scenarios were considered, varying in exposure/outcome generation models (presence of confounder-confounder interactions) and missingness mechanisms (whether outcome influenced missingness in other variables and presence of interaction/non-linear terms in missingness models). Complete-case analysis and extended TMLE had small biases when outcome did not influence missingness in other variables. Parametric MI without interactions had large bias when exposure/outcome generation models included interactions. Parametric MI including interactions performed best in bias and variance reduction across all settings, except when missingness models included a non-linear term. When choosing a method to handle missing data in the context of TMLE, researchers must consider the missingness mechanism and, for MI, compatibility with the analysis method. In many settings, a parametric MI approach that incorporates interactions and non-linearities is expected to perform well.

stat.ME↗

arXiv2023

Estimation of treatment policy estimands for continuous outcomes using off treatment sequential multiple imputation

The estimands framework outlined in ICH E9 (R1) describes the components needed to precisely define the effects to be estimated in clinical trials, which includes how post-baseline "intercurrent" events (IEs) are to be handled. In late-stage clinical trials, it is common to handle intercurrent events like "treatment discontinuation" using the treatment policy strategy and target the treatment effect on all outcomes regardless of treatment discontinuation. For continuous repeated measures, this type of effect is often estimated using all observed data before and after discontinuation using either a mixed model for repeated measures (MMRM) or multiple imputation (MI) to handle any missing data. In basic form, both of these estimation methods ignore treatment discontinuation in the analysis and therefore may be biased if there are differences in patient outcomes after treatment discontinuation compared to patients still assigned to treatment, and missing data being more common for patients who have discontinued treatment. We therefore propose and evaluate a set of MI models that can accommodate differences between outcomes before and after treatment discontinuation. The models are evaluated in the context of planning a phase 3 trial for a respiratory disease. We show that analyses ignoring treatment discontinuation can introduce substantial bias and can sometimes underestimate variability. We also show that some of the MI models proposed can successfully correct the bias but inevitably lead to increases in variance. We conclude that some of the proposed MI models are preferable to the traditional analysis ignoring treatment discontinuation, but the precise choice of MI model will likely depend on the trial design, disease of interest and amount of observed and missing data following treatment discontinuation.

stat.ME↗

arXiv2022

Phases of methodological research in biostatistics - building the evidence base for new methods

Although the biostatistical scientific literature publishes new methods at a very high rate, many of these developments are not trustworthy enough to be adopted by the scientific community. We propose a framework to think about how a piece of methodological work contributes to the evidence base for a method. Similarly to the well-known phases of clinical research in drug development, we define four phases of methodological research. These four phases cover (I) providing logical reasoning and proofs, (II) providing empirical evidence, first in a narrow target setting, then (III) in an extended range of settings and for various outcomes, accompanied by appropriate application examples, and (IV) investigations that establish a method as sufficiently well-understood to know when it is preferred over others and when it is not. We provide basic definitions of the four phases but acknowledge that more work is needed to facilitate unambiguous classification of studies into phases. Methodological developments that have undergone all four proposed phases are still rare, but we give two examples with references. Our concept rebalances the emphasis to studies in phase III and IV, i.e., carefully planned methods comparison studies and studies that explore the empirical properties of existing methods in a wider range of problems.

stat.ME↗

arXiv2022

Improving clinical trial interpretation with ACCEPT analyses

Effective decision making from randomised controlled clinical trials relies on robust interpretation of the numerical results. However, the language we use to describe clinical trials can cause confusion both in trial design and in comparing results across trials. ACceptability Curve Estimation using Probability Above Threshold (ACCEPT) aids comparison between trials (even where of different designs) by harmonising reporting of results, acknowledging different interpretations of the results may be valid in different situations, and moving the focus from comparison to a pre-specified value to interpretation of the trial data. ACCEPT can be applied to historical trials or incorporated into statistical analysis plans for future analyses. An online tool enables ACCEPT on up to three trials simultaneously.

stat.AP↗

arXiv2022

A comparison of strategies for selecting auxiliary variables for multiple imputation

Multiple imputation (MI) is a popular method for handling missing data. Auxiliary variables can be added to the imputation model(s) to improve MI estimates. However, the choice of which auxiliary variables to include in the imputation model is not always straightforward. Including too few may lead to important information being discarded, but including too many can cause problems with convergence of the estimation procedures for imputation models. Several data-driven auxiliary variable selection strategies have been proposed. This paper uses a simulation study and a case study to provide a comprehensive comparison of the performance of eight auxiliary variable selection strategies, with the aim of providing practical advice to users of MI. A complete case analysis and an MI analysis with all auxiliary variables included in the imputation model (the full model) were also performed for comparison. Our simulation study results suggest that the full model outperforms all auxiliary variable selection strategies, providing further support for adopting an inclusive auxiliary variable strategy where possible. Auxiliary variable selection using the Least Absolute Selection and Shrinkage Operator (LASSO) was the best performing auxiliary variable selection strategy overall and is a promising alternative when the full model fails. All MI analysis strategies that we were able to apply to the case study led to similar estimates.

stat.ME↗

arXiv2021

Planning a method for covariate adjustment in individually-randomised trials: a practical guide

Background: It has long been advised to account for baseline covariates in the analysis of confirmatory randomised trials, with the main statistical justifications being that this increases power and, when a randomisation scheme balanced covariates, permits a valid estimate of experimental error. There are various methods available to account for covariates but it is not clear how to choose among them. Methods: Taking the perspective of writing a statistical analysis plan, we consider how to choose between the three most promising broad approaches: direct adjustment, standardisation and inverse-probability-of-treatment weighting. Results: The three approaches are similar in being asymptotically efficient, in losing efficiency with mis-specified covariate functions, and in handling designed balance. If a marginal estimand is targeted (for example, a risk difference or survival difference), then direct adjustment should be avoided because it involves fitting non-standard models that are subject to convergence issues. Convergence is most likely with IPTW. Robust standard errors used by IPTW are anti-conservative at small sample sizes. All approaches can use similar methods to handle missing covariate data. With missing outcome data, each method has its own way to estimate a treatment effect in the all-randomised population. We illustrate some issues in a reanalysis of GetTested, a randomised trial designed to assess the effectiveness of an electonic sexually-transmitted-infection testing and results service. Conclusions: No single approach is always best: the choice will depend on the trial context. We encourage trialists to consider all three methods more routinely.

stat.ME↗

arXiv2021

Covariate adjustment in randomised trials: canonical link functions protect against model mis-specification

Covariate adjustment has the potential to increase power in the analysis of randomised trials, but mis-specification of the adjustment model could cause error. We explore what error is possible when the adjustment model omits a covariate by randomised treatment interaction, in a setting where the covariate is perfectly balanced between randomised treatments. We use mathematical arguments and analyses of single hypothetical data sets. We show that analysis by a generalised linear model with the canonical link function leads to no error under the null -- that is, if treatment effect is truly zero under the adjusted model then it is also zero under the unadjusted model. However, using non-canonical link functions does not give this property and leads to potentially important error under the null. The error is present even in large samples and hence constitutes bias. We conclude that covariate adjustment analyses of randomised trials should avoid non-canonical links. If a marginal risk difference is the target of estimation then this should not be estimated using an identity link; alternative preferable methods include standardisation and inverse probability of treatment weighting.

stat.ME↗

arXiv2020

Introducing the treatment hierarchy question in network meta-analysis

Background: Comparative effectiveness research using network meta-analysis can present a hierarchy of competing treatments, from the least to most preferable option. However, the research question associated with the hierarchy of multiple interventions is never clearly defined in published reviews. Methods and Results: We introduce the notion of a treatment hierarchy question that describes the criterion for choosing a specific treatment over one or more competing alternatives. For example, stakeholders might ask which treatment is most likely to improve mean survival by at least 2 years or which treatment is associated with the longest mean survival. The answers to these two questions are not necessarily the same. We discuss the most commonly used ranking metrics (quantities that describe or compare the estimated treatment-specific effects), how the metrics produce a treatment hierarchy and the type of treatment hierarchy question that each metric can answer. We show that the ranking metrics encompass the uncertainty in the estimation of the treatment effects in different ways, which results in different treatment hierarchies. Conclusions: Network meta-analyses that aim to rank treatments should state in the protocol the treatment hierarchy question they aim to address and employ the appropriate ranking metric to answer it.

stat.ME↗

arXiv2019

Handling an uncertain control group event risk in non-inferiority trials: non-inferiority frontiers and the power-stabilising transformation

Background. Non-inferiority (NI) trials are increasingly used to evaluate new treatments expected to have secondary advantages over standard of care, but similar efficacy on the primary outcome. When designing a NI trial with a binary primary outcome, the choice of effect measure for the NI margin has an important effect on sample size calculations; furthermore, if the control event risk observed is markedly different from that assumed, the trial can quickly lose power or the results become difficult to interpret. Methods. We propose a new way of designing NI trials to overcome the issues raised by unexpected control event risks by specifying a NI frontier, i.e. a curve defining the most appropriate non-inferiority margin for each possible value of control event risk. We propose a fixed arcsine difference frontier, the power-stabilising transformation for binary outcomes. We propose and compare three ways of designing a trial using this frontier. Results. Testing and reporting on the arcsine scale leads to results which are challenging to interpret clinically. Working on the arcsine scale generally requires a larger sample size compared to the risk difference scale. Therefore, working on the risk difference scale, modifying the margin after observing the control event risk, might be preferable, as it requires a smaller sample size. However, this approach tends to slightly inflate type I error rate; a solution is to use a lower significance level for testing. When working on the risk ratio scale, the same approach leads to power levels above the nominal one, maintaining type I error under control. Conclusions. Our proposed methods of designing NI trials using power-stabilising frontiers make trial design more resilient to unexpected values of the control event risk, at the only cost of requiring larger sample sizes when the goal is to report results on the risk difference scale.

stat.ME↗

arXiv2017

Multiple imputation for multilevel data with continuous and binary variables

We present and compare multiple imputation methods for multilevel continuous and binary data where variables are systematically and sporadically missing. The methods are compared from a theoretical point of view and through an extensive simulation study motivated by a real dataset comprising multiple studies. Simulations are reproducible. The comparisons show why these multiple imputation methods are the most appropriate to handle missing values in a multilevel setting and why their relative performances can vary according to the missing data pattern, the multilevel structure and the type of missing variables. This study shows that valid inferences can only be obtained if the dataset gathers a large number of clusters. In addition, it highlights that heteroscedastic MI methods provide more accurate inferences than homoscedastic methods, which should be reserved for data with few individuals per cluster. Finally, the method of Quartagno and Carpenter (2016a) appears generally accurate for binary variables, the method of Resche-Rigon and White (2016) with large clusters, and the approach of Jolani et al. (2015) with small clusters.

stat.ME↗

arXiv2017

A causal modelling framework for reference-based imputation and tipping point analysis

We consider estimating the "de facto" or effectiveness estimand in a randomised placebo-controlled or standard-of-care-controlled drug trial with quantitative outcome, where participants who discontinue an investigational treatment are not followed up thereafter. Carpenter et al (2013) proposed reference-based imputation methods which use a reference arm to inform the distribution of post-discontinuation outcomes and hence to inform an imputation model. However, the reference-based imputation methods were not formally justified. We present a causal model which makes an explicit assumption in a potential outcomes framework about the maintained causal effect of treatment after discontinuation. We show that the "jump to reference", "copy reference" and "copy increments in reference" reference-based imputation methods, with the control arm as the reference arm, are special cases of the causal model with specific assumptions about the causal treatment effect. Results from simulation studies are presented. We also show that the causal model provides a flexible and transparent framework for a tipping point sensitivity analysis in which we vary the assumptions made about the causal effect of discontinued treatment. We illustrate the approach with data from two longitudinal clinical trials.

stat.ME↗

arXiv2017

A mean score method for sensitivity analysis to departures from the missing at random assumption in randomised trials

Most analyses of randomised trials with incomplete outcomes make untestable assumptions and should therefore be subjected to sensitivity analyses. However, methods for sensitivity analyses are not widely used. We propose a mean score approach for exploring global sensitivity to departures from missing at random or other assumptions about incomplete outcome data in a randomised trial. We assume a single outcome analysed under a generalised linear model. One or more sensitivity parameters, specified by the user, measure the degree of departure from missing at random in a pattern mixture model. Advantages of our method are that its sensitivity parameters are relatively easy to interpret and so can be elicited from subject matter experts; it is fast and non-stochastic; and its point estimate, standard error and confidence interval agree perfectly with standard methods when particular values of the sensitivity parameters make those standard methods appropriate. We illustrate the method using data from a mental health trial.

stat.ME↗

arXiv2016

Propensity score analysis with partially observed confounders: how should multiple imputation be used?

Inverse probability of treatment weighting (IPTW) is a popular propensity score (PS)-based approach to estimate causal effects in observational studies at risk of confounding bias. A major issue when estimating the PS is the presence of partially observed covariates. Multiple imputation (MI) is a natural approach to handle missing data on covariates, but its use in the PS context raises three important questions: (i) should we apply Rubin's rules to the IPTW treatment effect estimates or to the PS estimates themselves? (ii) does the outcome have to be included in the imputation model? (iii) how should we estimate the variance of the IPTW estimator after MI? We performed a simulation study focusing on the effect of a binary treatment on a binary outcome with three confounders (two of them partially observed). We used MI with chained equations to create complete datasets and compared three ways of combining the results: combining treatment effect estimates (MIte); combining the PS across the imputed datasets (MIps); or combining the PS parameters and estimating the PS of the average covariates across the imputed datasets (MIpar). We also compared the performance of these methods to complete case (CC) analysis and the missingness pattern (MP) approach, a method which uses a different PS model for each pattern of missingness. We also studied empirically the consistency of these 3 MI estimators. Under a missing at random (MAR) mechanism, CC and MP analyses were biased in most cases when estimating the marginal treatment effect, whereas MI approaches had good performance in reducing bias as long as the outcome was included in the imputation model. However, only MIte was unbiased in all the studied scenarios and Rubin's rules provided good variance estimates for MIte.

stat.ME↗

arXiv2013

Multiple imputation of covariates by fully conditional specification: accommodating the substantive model

Missing covariate data commonly occur in epidemiological and clinical research, and are often dealt with using multiple imputation (MI). Imputation of partially observed covariates is complicated if the substantive model is non-linear (e.g. Cox proportional hazards model), or contains non-linear (e.g. squared) or interaction terms, and standard software implementations of MI may impute covariates from models that are incompatible with such substantive models. We show how imputation by fully conditional specification, a popular approach for performing MI, can be modified so that covariates are imputed from models which are compatible with the substantive model. We investigate through simulation the performance of this proposal, and compare it to existing approaches. Simulation results suggest our proposal gives consistent estimates for a range of common substantive models, including models which contain non-linear covariate effects or interactions, provided data are missing at random and the assumed imputation models are correctly specified and mutually compatible.

stat.ME↗