Searcharxiv⌕ Search

arXiv subjects

Paul Gustafson

Publications and source records attributed to Paul Gustafson.

At least 19 recordsLinked to original sources
arXiv2026

Improving interpretation of latent class models for diagnostic tests by recognizing their measurands via directed acyclic graphs (DAGs)

Summary: In the absence of a perfect diagnostic test for a target condition, multiple imperfect tests may be used to arrive at a clinical diagnosis. Latent class analysis can be used to model such data with the objective of estimating test accuracy and target condition prevalence. Such models typically assume two latent classes - target condition positive and target condition negative. However, as we will illustrate in this manuscript, this would be an oversimplification if the different tests do not share the target condition as their measurand. We show how a Directed Acyclic Graph (DAG) can be used to illustrate the relationships between the relevant variables - the observed imperfect test results, their latent measurands, the latent target condition of interest and observed covariates - revealing any conditional dependence relations. The DAG helps determine the number of latent classes, underlying the observed data, and their labels. We show how the likelihood function changes due to incorporating the measurand of each test. We study the impact on identifiability of the model. Using simulation studies we show how ignoring the measurand of an imperfect test, when it is distinct from the target condition, can lead to biased estimates of test accuracy and prevalence. We illustrate the value of the proposed approach by re-analyzing two datasets used in previously published latent class analyses of tests for pediatric tuberculosis and leptospirosis.

stat.ME↗
arXiv2026

Methods for adjusting for covariate measurement error in flexible modelling of functional form: results of a blinded, controlled neutral comparison simulation study

Covariate measurement error is pervasive in epidemiological research and distorts estimated exposure-outcome associations, yet correction methods have been studied almost exclusively under linear modelling assumptions. Their behaviour when the underlying association is non-linear and is itself estimated with flexible regression, remains poorly characterised. We report a blinded, multi-stage neutral comparison simulation study, conducted within the STRATOS initiative, evaluating measurement error correction coupled with flexible modelling of functional form. Six families of correction methods (pointwise and coefficient-based Simulation Extrapolation [SIMEX], Bayesian inference on the logit and risk scales, Multiple Imputation [MI], and Regression Calibration [RC]) were each combined with B-splines (BS), penalised splines (PS), fractional polynomials (FP), and natural splines (NS), yielding 23 analytic methods. Methods were applied to case-control data generated under five functional forms (J-shape, linear, two threshold models, and saturation) across simulated datasets spanning varying sample sizes, replication substudy sizes, error magnitudes, and error distributions, with classical additive error and a replication substudy for error calibration. Performance was assessed by the log mean squared error of the estimated function over the central 95 % of the exposure distribution. Pointwise SIMEX was the most accurate and most robust approach overall, followed by Bayesian methods and RC when paired with PS, FP, or NS; MI performed less well, and Bayesian estimation with unpenalised BS performed worst. PS, FP, and NS were near-equivalent, whereas BS was consistently inferior. No single method dominated across all scenarios, underscoring the value of sensitivity analyses.

stat.ME↗
arXiv2026

Methods for adjusting for covariate measurement error in flexible modelling of functional form: designing a blinded, controlled neutral comparison simulation study

This article describes the design of a neutral comparison study in the context of empirical studies where the interest is in learning the functional relationship between a continuous errorprone exposure variable and a binary outcome. The performance of combinations of measurement error correction methods and flexible regression modeling techniques was compared using a simulation study. The project involved four independent teams, one devoted to data generation and evaluation, the other three to specific methods of measurement error correction (Simulation-Extrapolation, Regression-Calibration and Multiple imputation, Bayesian method). The study was conducted in three successive stages. In Stage 1, the first team simulated five datasets differing only by the true exposure-outcome functional form and distribution of true exposure. Furthermore, the implementation of flexible modeling methods (B-splines, P-splines, and fractional polynomials) was standardized. The three methods teams, blinded to the underlying data generation process, created the codes to implement their methods, and provided their results to the first team who evaluated them. These codes were then used by this team in the next Stages of the project. In Stage 2, the team simulated 150 additional datasets where other design parameters varied while using the same five exposureoutcome functions. Stage 3 consisted of simulating independent replications of each of the 150 scenarios considered in Stage 2 to quantify the sampling variance of the estimates. This work emphasizes the relevance of neutral comparison studies to fairly evaluate statistical methods aimed at addressing a complex analytical challenge, and demonstrates their feasibility through a large collaborative project.

stat.ME↗
arXiv2026

Don't Disregard the Data for Lack of a Likelihood: Bayesian Synthetic Likelihood for Enhanced Multilevel Network Meta-Regression

Multilevel network meta-regression (ML-NMR) enables population-adjusted indirect treatment comparisons by combining individual patient data (IPD) with aggregate data. When individual-level covariates are unavailable, ML-NMR marginalizes over the covariate distribution, but this strategy cannot exploit subgroup-level summary results that are often available and potentially highly informative. We propose using Bayesian Synthetic Likelihood (BSL) to leverage this ancillary summary information and present an implementation strategy for Hamiltonian Monte Carlo (HMC), a gradient-based Markov chain Monte Carlo (MCMC) algorithm. At each MCMC iteration, the BSL method imputes missing covariates by sampling from the model-implied conditional distribution, computes synthetic subgroup summaries from the imputed data, and matches these synthetic summaries to observed summaries via a multivariate normal synthetic likelihood. Fitting this model with HMC presents multiple challenges: first, gradients cannot be computed exactly but must be estimated stochastically; and second, the model's likelihood may be non-differentiable at certain points, a pathology that can deeply frustrate the performance of HMC. We address these challenges with pre-drawn random numbers, continuous relaxation of the likelihood, and Pareto-smoothed importance sampling. This work (1) introduces a novel application of BSL to missing data problems where summary statistics from the complete dataset are available despite substantial missingness in the individual-level data, (2) demonstrates how BSL strategies can be implemented within Stan's HMC framework, and (3) shows, using a network of plaque psoriasis trials, that BSL-enhanced ML-NMR can substantially improve upon standard ML-NMR by leveraging informative ancillary information.

stat.ME↗
arXiv2025

Leveraging Relational Evidence: Population Size Estimation on Tree-Structured Data with the Weighted Multiplier Method

Populations of interest are often hidden from data for a variety of reasons, though their magnitude remains important in determining resource allocation and appropriate policy. One popular approach to population size estimation, the multiplier method, is a back-calculation tool requiring only a marginal subpopulation size and an estimate of the proportion belonging to this subgroup. Another approach is to use Bayesian methods, which are inherently well-suited to incorporating multiple data sources. However, both methods have their drawbacks. A framework for applying the multiplier method which combines information from several known subpopulations has not yet been established; Bayesian models, though able to incorporate complex dependencies and various data sources, can be difficult for researchers in less technical fields to design and implement. Increasing data collection and linkage across diverse fields suggests accessible methods of estimating population size with synthesized data are needed. We propose an extension to the well-known multiplier method which is applicable to tree-structured data, where multiple subpopulations and corresponding proportions combine to generate a population size estimate via the minimum variance estimator. The methodology and resulting estimates are compared with those from a Bayesian hierarchical model, for both simulated and real world data. Subsequent analysis elucidates which data are key to estimation in each method, and examines robustness and feasibility of this new methodology.

stat.ME↗
arXiv2025

AutoWMM and JAGStree -- R packages for Population Size Estimation on Relational Tree-Structured Data

The weighted multiplier method (WMM) is an extension of the traditional method of back-calculation method to estimate the size of a target population, which synthesizes available evidence from multiple subgroups of the target population with known counts and estimated proportions by leveraging the tree-structure inherent to the data. Hierarchical Bayesian models offer an alternative to modeling population size estimation on such a structure, but require non-trivial theoretical and practical knowledge to implement. While the theory underlying the WMM methodology may be more accessible to researchers in diverse fields, a barrier still exists in execution of this method, which requires significant computation. We develop two \texttt{R} packages to help facilitate population size estimation on trees using both the WMM and hierarchical Bayesian modeling; \textit{AutoWMM} simplifies WMM estimation for any general tree topology, and \textit{JAGStree} automates the creation of suitable JAGS MCMC modeling code for these same networks.

stat.CO↗
arXiv2025

Estimating the Number of Opioid Overdoses in British Columbia Using Relational Evidence with Tree Structure

In many fields, populations of interest are hidden from data for a variety of reasons, though their magnitude remains important in determining resource allocation and appropriate policy. In public health and epidemiology, linkages or relationships between sources of data may exist due to intake structure of care providers, referrals, or other related health programming. These relationships often admit a tree structure, with the target population represented by the root, and paths from root-to-leaf representing pathways of care after a health event. In the Canadian province of British Columbia (BC), significant efforts have been made in creating an opioid overdose cohort, a tree-like linked data structure which tracks the movement of individuals along pathways of care after an overdose. In this application, the root node represents the target population, the total number of overdose events occurring in BC during the specified time period. We compare and contrast two methods of estimating the target population size - a weighted multiplier method based on back-calculating estimates from a number of paths and combining these estimates via a variance-minimizing weighted mean, and a fully Bayesian hierarchical model.

stat.AP↗
arXiv2025

Bayesian sample size calculations for external validation studies of risk prediction models

Contemporary sample size calculations for external validation of risk prediction models require users to specify fixed values of assumed model performance metrics alongside target precision levels (e.g., 95% CI widths). However, due to the finite samples of previous studies, our knowledge of true model performance in the target population is uncertain, and so choosing fixed values represents an incomplete picture. As well, for net benefit (NB) as a measure of clinical utility, the relevance of conventional precision-based inference is doubtful. In this work, we propose a general Bayesian framework for multi-criteria sample size considerations for prediction models for binary outcomes. For statistical metrics of performance (e.g., discrimination and calibration), we propose sample size rules that target desired expected precision or desired assurance probability that the precision criteria will be satisfied. For NB, we propose rules based on Optimality Assurance (the probability that the planned study correctly identifies the optimal strategy) and Value of Information (VoI) analysis. We showcase these developments in a case study on the validation of a risk prediction model for deterioration of hospitalized COVID-19 patients. Compared to the conventional sample size calculation methods, a Bayesian approach requires explicit quantification of uncertainty around model performance, and thereby enables flexible sample size rules based on expected precision, assurance probabilities, and VoI. In our case study, calculations based on VoI for NB suggest considerably lower sample sizes are needed than when focusing on precision of calibration metrics.

stat.AP↗
arXiv2024

Expected value of sample information calculations for risk prediction model development

Risk prediction models are often advertised as deterministic functions that map covariates to predicted risks. However, they are typically trained using finite samples, and as such, their predictions are inherently uncertain. This uncertainty has been addressed in terms of uncertainty around metrics of model performance (e.g., confidence intervals around c-statistic), as well as uncertainty or instability of predictions. Correspondingly, sample size calculations for model development studies target the precision of estimates of summary statistics and the stability of predictions. However, when evaluating the clinical utility of a model (as in Net Benefit (NB) calculations in decision curve analysis), statistical inference is less relevant. From a decision-theoretic perspective, the finite size of the sample results in utility loss due to the discrepancy between the fitted model and the correct model. From this perspective, procuring more development data is associated with an expected gain in the utility of using the model. In this work, we define the Expected Value of Sample Information (EVSI) as the expected gain in clinical utility, defined in NB terms, by procuring an additional development sample of a given size. We propose a bootstrap-based algorithm for EVSI computations and demonstrate its feasibility and face validity in a case study. We conclude that decision-theoretic metrics can complement classical inferential methods when designing studies aimed at developing risk prediction models.

stat.ME↗
arXiv2024

Issues of parameterization and computation for posterior inference in partially identified models

A partially identified model, where the parameters can not be uniquely identified, often arises during statistical analysis. While researchers frequently use Bayesian inference to analyze the models, when Bayesian inference with an off-the-shelf MCMC sampling algorithm is applied to a partially identified model, the computational performance can be poor. It is found that using importance sampling with transparent reparameterization (TP) is one remedy. This method is preferable since the model is known to be rendered as identified with respect to the new parameterization, and at the same time, it may allow faster, i.i.d. Monte Carlo sampling by using conjugate convenience priors. In this paper, we explain the importance sampling method with the TP and a pseudo-TP. We introduce the pseudo-TP, an alternative to TP, since finding a TP is sometimes difficult. Then, we test the methods' performance in some scenarios and compare it to the performance of the off-the-shelf MCMC method - Gibbs sampling - applied in the original parameterization. While the importance sampling with TP (ISTP) shows generally better results than off-the-shelf MCMC methods, as seen in the compute time and trace plots, it is also seen that finding a TP which is necessary for the method may not be easy. On the other hand, the pseudo-TP method shows a mixed result and room for improvement since it relies on an approximation, which may not be adequate for a given model and dataset.

stat.CO↗
arXiv2024

Evaluating Treatment Benefit Predictors using Observational Data: Contending with Identification and Confounding Bias

A treatment benefit predictor (TBP) is a function that maps patient characteristics to an estimate of the treatment benefit for that patient. Such predictors support optimizing individualized treatment decisions, which are central to precision medicine. However, evaluating the predictive performance of a TBP is challenging, as this often must be conducted in a sample where treatment assignment is not random. After briefly reviewing several metrics for evaluating TBPs, we show conceptually how to evaluate a pre-specified TBP using observational data from the target population, for a binary treatment decision at a single time point. We exemplify with a particular measure of discrimination (the concentration of benefit index) and a particular measure of calibration (the moderate calibration curve). The population-level definitions of these metrics involve the latent treatment benefit variable, but we show identification by re-expressing the respective estimands in terms of the distribution of observable data only. We also show that in the absence of full confounding control, bias propagates in a more complex manner than when targeting more commonly encountered estimands. We find the patterns of biases are often unpredictable, and general intuition about the direction of bias in causal effect estimates does not hold in the present context.

stat.ME↗
arXiv2024

A fully Bayesian approach for the imputation and analysis of derived outcome variables with missingness

Derived variables are variables that are constructed from one or more source variables through established mathematical operations or algorithms. For example, body mass index (BMI) is a derived variable constructed from two source variables: weight and height. When using a derived variable as the outcome in a statistical model, complications arise when some of the source variables have missing values. In this paper, we propose how one can define a single fully Bayesian model to simultaneously impute missing values and sample from the posterior. We compare our proposed method with alternative approaches that rely on multiple imputation with examples including an analysis to estimate the risk of microcephaly (a derived variable based on sex, gestational age and head circumference at birth) in newborns exposed to the ZIKA virus.

stat.ME↗
arXiv2024

Integrating representative and non-representative survey data for efficient inference

Non-representative surveys are commonly used and widely available but suffer from selection bias that generally cannot be entirely eliminated using weighting techniques. Instead, we propose a Bayesian method to synthesize longitudinal representative unbiased surveys with non-representative biased surveys by estimating the degree of selection bias over time. We show using a simulation study that synthesizing biased and unbiased surveys together out-performs using the unbiased surveys alone, even if the selection bias may evolve in a complex manner over time. Using COVID-19 vaccination data, we are able to synthesize two large sample biased surveys with an unbiased survey to reduce uncertainty in now-casting and inference estimates while simultaneously retaining the empirical credible interval coverage. Ultimately, we are able to conceptually obtain the properties of a large sample unbiased survey if the assumed unbiased survey, used to anchor the estimates, is unbiased for all time-points.

stat.ME↗
arXiv2024

The expected value of sample information calculations for external validation of risk prediction models

In designing external validation studies of clinical prediction models, contemporary sample size calculation methods are based on the frequentist inferential paradigm. One of the widely reported metrics of model performance is net benefit (NB), and the relevance of conventional inference around NB as a measure of clinical utility is doubtful. Value of Information methodology quantifies the consequences of uncertainty in terms of its impact on clinical utility of decisions. We introduce the expected value of sample information (EVSI) for validation as the expected gain in NB from conducting an external validation study of a given size. We propose algorithms for EVSI computation, and in a case study demonstrate how EVSI changes as a function of the amount of current information and future study's sample size. Value of Information methodology provides a decision-theoretic lens to the process of planning a validation study of a risk prediction model and can complement conventional methods when designing such studies.

stat.AP↗
arXiv2023

Partial identification for discrete data with nonignorable missing outcomes

Nonignorable missing outcomes are common in real world datasets and often require strong parametric assumptions to achieve identification. These assumptions can be implausible or untestable, and so we may forgo them in favour of partially identified models that narrow the set of a priori possible values to an identification region. Here we propose a new nonparametric Bayes method that allows for the incorporation of multiple clinically relevant restrictions of the parameter space simultaneously. We focus on two common restrictions, instrumental variables and the direction of missing data bias, and investigate how these restrictions narrow the identification region for parameters of interest. Additionally, we propose a rejection sampling algorithm that allows us to quantify the evidence for these assumptions in the data. We compare our method to a standard Heckman selection model in both simulation studies and in an applied problem examining the effectiveness of cash-transfers for people experiencing homelessness.

stat.ME↗
arXiv2023

Methodological concerns about 'concordance-statistic for benefit' as a measure of discrimination in treatment benefit prediction

Prediction algorithms that quantify the expected benefit of a given treatment conditional on patient characteristics can critically inform medical decisions. Quantifying the performance of treatment benefit prediction algorithms is an active area of research. A recently proposed metric, the concordance statistic for benefit (cfb), evaluates the discriminative ability of a treatment benefit predictor by directly extending the concept of the concordance statistic from a risk model with a binary outcome to a model for treatment benefit. In this work, we scrutinize $cfb$ on multiple fronts. Through numerical examples and theoretical developments, we show that cfb is not a proper scoring rule. We also show that it is sensitive to the unestimable correlation between counterfactual outcomes and to the definition of matched pairs. We argue that measures of statistical dispersion applied to predicted benefits do not suffer from these issues and can be an alternative metric for the discriminatory performance of treatment benefit predictors.

stat.ME↗
arXiv2023

Defining a credible interval is not always possible with "point-null'' priors: A lesser-known correlate of the Jeffreys-Lindley paradox

In many common situations, a Bayesian credible interval will be, given the same data, very similar to a frequentist confidence interval, and researchers will interpret these intervals in a similar fashion. However, no predictable similarity exists when credible intervals are based on model-averaged posteriors whenever one of the two nested models under consideration is a so called ''point-null''. Not only can this model-averaged credible interval be quite different than the frequentist confidence interval, in some cases it may be undefined. This is a lesser-known correlate of the Jeffreys-Lindley paradox and is of particular interest given the popularity of the Bayes factor for testing point-null hypotheses.

math.ST↗