arXiv subjects

Julian Wolfson

Publications and source records attributed to Julian Wolfson.

18 recordsLinked to original sources

arXiv2026

A Statistical Framework for Data-Driven Discovery of Differential Performance in Clinical Risk Prediction Models

Predictive models employing artificial intelligence (AI) and machine learning (ML) are increasingly being used for decision support in healthcare settings. These models may exhibit differential performance across population subgroups defined by race, age, sex, and other factors and cause disparate clinical impacts, leading to intensive recent study of what has been termed "model fairness". While many methods have been proposed to assess risk prediction model fairness, these techniques generally require that the end user pre-specify the groups across which fairness is to be evaluated. In real-world settings, however, important model performance disparities may arise in unknown subgroups defined by multiple intersecting characteristics. To address this problem, we propose the unfairness tree (utree), a data-driven recursive partitioning framework for identifying subgroups with differential model performance. In simulations, the utree exhibits nominal empirical type I error rates and good ability to detect, quantify, and characterize performance discrepancies defined by higher-order variable interactions. In six mortality risk models fit to the GUSTO-I acute myocardial infarction trial dataset, utrees identified subgroup-specific performance patterns, with age, sex, blood pressure, and Killip class consistently associated with differential model performance.

stat.ME↗

arXiv2026

Causal Inference for Functional Treatments with Stochastic Policies

Wearable devices can accurately measure human behavior, providing a unique opportunity to understand how behavior impacts health. Recent studies leveraging functional regression methods have found a strong relationship between accelerometer-collected physical activity and mortality. However, to determine if physical activity patterns impact mortality it is necessary to understand the causal effects of policies for physical activity, i.e., a function-valued treatment. Functional treatments present several challenges for causal effect estimation: 1) defining a scientifically meaningful estimand that reflects real-world policies and satisfies positivity is nontrivial; and 2) the potential for temporal confounding over continuous time. To address these, we propose stochastic policies for functional treatments that allow estimation of causal effects of changing the treatment distribution without requiring a positivity assumption. We develop a novel method for such that modifies the treatment through a single basis function chosen by the analyst, allowing for clear control over treatment modification and temporal confounding feedback. We show asymptotic normality of our estimators and that they exhibit rate double robustness. We apply our methods to the National Health and Nutrition Examination Survey to determine the causal effect of increasing physical activity over three-hour periods on mortality.

stat.ME↗

arXiv2025

Partially Retargeted Balancing Weights for Causal Effect Estimation Under Positivity Violations

Positivity violations, which occur when some subgroups either always or never receive a treatment of interest, pose significant challenges for causal effect estimation with observational data. Recent balancing weight methods have proved to be highly effective in confounding control, however their utility is diminished in the presence of positivity violations, resulting in bias and excess variance. Approaches that deal with positivity violations, on the other hand, work by targeting a modified estimand that may be misaligned with the original research question. To address these challenges, we propose a novel balancing weights approach, which mitigates positivity violations while attempting to retain the original estimand by a targeted relaxation of the balancing constraints. Our proposed weighted estimator is consistent for the original estimand when either 1) the implied propensity score model is correct; or 2) all treatment effect modifiers are balanced to the target population. When these conditions do not hold, our estimator is consistent for a slightly modified treatment effect estimand. Furthermore, our proposed weighted estimator has reduced asymptotic variance when positivity does not hold. We evaluate our approach through applications to synthetic data, an observational study, and when transporting a treatment effect from a randomized trial.

stat.ME↗

arXiv2025

Multinomial thresholded LASSO for interpretable dimension reduction of human activity sequences

The widespread collection of data from mobile and wearable devices has created unprecedented opportunities to study human behavior in fine temporal resolution. One common structure for such data is categorical sequences: ordered, multinomial observations across many time points. These sequences present unique statistical challenges due to their high dimensionality and complex temporal dependence, including both short- and long-term correlations. Yet, there has been relatively little methodological development focusing on principled dimension reduction specifically tailored to this type of data. In this paper, we develop and evaluate approaches to identifying "key" sequence positions which distinguish sequence types. We frame this challenge as a regression problem, introduce a variety of regularization techniques that could be applied to achieve position-based dimension reduction, and evaluate them on the motivating dataset that reflects daily time use patterns collected via a smartphone application. Results show that the thresholded LASSO, a relatively underused technique, performs better than more established methods for data with complex sequential structure.

stat.ME↗

arXiv2025

Multi-Study Causal Forest (MCF): A flexible framework for data borrowing in the presence of varying treatment effect heterogeneity

Tailoring treatment assignment to specific individuals can improve the health outcomes, but a single study may offer inadequate information for this purpose. The ability to leverage information from an auxiliary data source deemed to be `most similar' to a primary data source has been shown to improve estimates of treatment effects. In this paper, we introduce a framework, the Multi-Study Causal Forest (MCF), to borrow individual patient-level data from an auxiliary data source in the presence of `varying sources' of treatment effect heterogeneity. We utilise a simulation study to demonstrate the superiority of the MCF in the presence of varying treatment allocation models (between-study heterogeneity) in addition to being able to account for the presence of within-study heterogeneity. This approach can combine data from randomised controlled trials, observational studies or a combination of both. We illustrate using Breast cancer data that the MCF performs favourably compared to an existing methodology in the presence of varying sources of (both between and within) heterogeneity.

stat.ME↗

arXiv2024

A Unified Framework for Causal Estimand Selection

Estimating the causal effect of a treatment or health policy with observational data can be challenging due to an imbalance of and a lack of overlap between treated and control covariate distributions. In the presence of limited overlap, researchers choose between 1) methods (e.g., inverse probability weighting) that imply traditional estimands but whose estimators are at risk of considerable bias and variance; and 2) methods (e.g., overlap weighting) which imply a different estimand, thereby modifying the target population to reduce variance. We propose a framework for navigating the tradeoffs between variance and bias due to imbalance and lack of overlap and the targeting of the estimand of scientific interest. We introduce a bias decomposition that encapsulates bias due to 1) the statistical bias of the estimator; and 2) estimand mismatch, i.e., deviation from the population of interest. We propose two design-based metrics and an estimand selection procedure that help illustrate the tradeoffs between these sources of bias and variance of the resulting estimators. Our procedure allows analysts to incorporate their domain-specific preference for preservation of the original research population versus reduction of statistical bias. We demonstrate how to select an estimand based on these preferences with an application to right heart catheterization data.

stat.ME↗

arXiv2024

Adjacency Matrix Decomposition Clustering for Human Activity Data

Mobile apps and wearable devices accurately and continuously measure human activity; patterns within this data can provide a wealth of information applicable to fields such as transportation and health. Despite the potential utility of this data, there has been limited development of analysis methods for sequences of daily activities. In this paper, we propose a novel clustering method and cluster evaluation metric for human activity data that leverages an adjacency matrix representation to cluster the data without the calculation of a distance matrix. Our technique is substantially faster than conventional methods based on computing pairwise distances via sequence alignment algorithms and also enhances interpretability of results. We compare our method to distance-based hierarchical clustering and nTreeClus through simulation studies and an application to data collected by Daynamica, an app that turns sensor data into a daily summary of a user's activities. Among days that contain a large portion of time spent at home, our method distinguishes days that also contain multiple hours of travel or other activities, while both comparison methods fail to identify these patterns. We further identify which day patterns classified by our method are associated with higher concern for contracting COVID-19 with implications for public health messaging.

stat.ME↗

arXiv2024

Counterfactual fairness for small subgroups

While methods for measuring and correcting differential performance in risk prediction models have proliferated in recent years, most existing techniques can only be used to assess fairness across relatively large subgroups. The purpose of algorithmic fairness efforts is often to redress discrimination against groups that are both marginalized and small, so this sample size limitation often prevents existing techniques from accomplishing their main aim. We take a three-pronged approach to address the problem of quantifying fairness with small subgroups. First, we propose new estimands built on the "counterfactual fairness" framework that leverage information across groups. Second, we estimate these quantities using a larger volume of data than existing techniques. Finally, we propose a novel data borrowing approach to incorporate "external data" that lacks outcomes and predictions but contains covariate and group membership information. This less stringent requirement on the external data allows for more possibilities for external data sources. We demonstrate practical application of our estimators to a risk prediction model used by a major Midwestern health system during the COVID-19 pandemic.

stat.ME↗

arXiv2023

A flexible framework for synthesizing human activity patterns with application to sequential categorical data

The ability to synthesize realistic data in a parametrizable way is valuable for a number of reasons, including privacy, missing data imputation, and evaluating the performance of statistical and computational methods. When the underlying data generating process is complex, data synthesis requires approaches that balance realism and simplicity. In this paper, we address the problem of synthesizing sequential categorical data of the type that is increasingly available from mobile applications and sensors that record participant status continuously over the course of multiple days and weeks. We propose the paired Markov Chain (paired-MC) method, a flexible framework that produces sequences that closely mimic real data while providing a straightforward mechanism for modifying characteristics of the synthesized sequences. We demonstrate the paired-MC method on two datasets, one reflecting daily human activity patterns collected via a smartphone application, and one encoding the intensities of physical activity measured by wearable accelerometers. In both settings, sequences synthesized by paired-MC better capture key characteristics of the real data than alternative approaches.

stat.ME↗

arXiv2023

An intersectional framework for counterfactual fairness in risk prediction

Along with the increasing availability of health data has come the rise of data-driven models to inform decision-making and policy. These models have the potential to benefit both patients and health care providers but can also exacerbate health inequities. Existing "algorithmic fairness" methods for measuring and correcting model bias fall short of what is needed for health policy in two key ways. First, methods typically focus on a single grouping along which discrimination may occur rather than considering multiple, intersecting groups. Second, in clinical applications, risk prediction is typically used to guide treatment, creating distinct statistical issues that invalidate most existing techniques. We present summary unfairness metrics that build on existing techniques in "counterfactual fairness" to address both challenges. We also develop a complete framework of estimation and inference tools for our metrics, including the unfairness value ("u-value"), used to determine the relative extremity of unfairness, and standard errors and confidence intervals employing an alternative to the standard bootstrap. We demonstrate application of our framework to a COVID-19 risk prediction model deployed in a major Midwestern health system.

stat.ME↗

arXiv2023

Reinforced Borrowing Framework: Leveraging Auxiliary Data for Individualized Inference

Increasingly during the past decade, researchers have sought to leverage auxiliary data for enhancing individualized inference. Many existing methods, such as multisource exchangeability models (MEM), have been developed to borrow information from multiple supplemental sources to support parameter inference in a primary source. MEM and its alternatives decide how much information to borrow based on the exchangeability of the primary and supplemental sources, where exchangeability is defined as equality of the target parameter. Other information that may also help determine the exchangeability of sources is ignored. In this article, we propose a generalized Reinforced Borrowing Framework (RBF) leveraging auxiliary data for enhancing individualized inference using a distance-embedded prior which utilizes data not only about the target parameter, but also uses different types of auxiliary information sources to "reinforce" inference on the target parameter. RBF improves inference with minimal additional computational burden. We demonstrate the application of RBF to a study investigating the impact of the COVID-19 pandemic on individual activity and transportation behaviors, where RBF achieves 20-40% lower MSE compared with existing methods.

stat.ME↗

arXiv2021

A flexible Bayesian framework for individualized inference via adaptive borrowing

The explosion in high-resolution data capture technologies in health has increased interest in making inferences about individual-level parameters. While technology may provide substantial data on a single individual, how best to use multisource population data to improve individualized inference remains an open research question. One possible approach, the multisource exchangeability model (MEM), is a Bayesian method for integrating data from supplementary sources into the analysis of a primary source. MEM was originally developed to improve inference for a single study by asymmetrically borrowing information from a set of similar previous studies and was further developed to apply a more computationally intensive symmetric borrowing in the context of basket trial; however, even for asymmetric borrowing, its computational burden grows exponentially with the number of supplementary sources, making it unsuitable for applications where hundreds or thousands of supplementary sources (i.e., individuals) could contribute to inference on a given individual. In this paper, we propose the data-driven MEM (dMEM), a two-stage approach that includes both source selection and clustering to enable the inclusion of an arbitrary number of sources to contribute to individualized inference in a computationally tractable and data-efficient way. We illustrate the application of dMEM to individual-level human behavior and mental well-being data collected via smartphones, where our approach increases individual-level estimation precision by 84% compared with a standard no-borrowing method and outperforms recently-proposed competing methods in 80% of individuals.

stat.ME↗

arXiv2021

visTree: Visualization of Subgroups for a Decision Tree

Decision trees are flexible prediction models which are constructed to quantify outcome-covariate relationships and characterize relevant population subgroups. However, the standard graphical representation of fitted decision trees highlights individual split points, and hence is suboptimal for visualizing defined subgroups. In this paper, we present a novel visual representation of decision trees which shifts the primary focus to characterizing subgroups, both in terms of their defining covariates and their outcome distribution. We implement our method in the \texttt{visTree} package, which builds on the toolkit and infrastructure provided by the \texttt{partykit} package and enables the visualization to be applied to varied decision trees. Individual functions are demonstrated using data from the Box Lunch study [French et al., 2014], a randomized trial to evaluate the effect of exposure to different lunch sizes on energy intake and body weight among working adults.

stat.AP↗

arXiv2020

Assessing effect heterogeneity of a randomized treatment using conditional inference trees

Treatment effect heterogeneity occurs when individual characteristics influence the effect of a treatment. We propose a novel approach that combines prognostic score matching and conditional inference trees to characterize effect heterogeneity of a randomized binary treatment. One key feature that distinguishes our method from alternative approaches is that it controls the Type I error rate, i.e., the probability of identifying effect heterogeneity if none exists and retains the underlying subgroups. This feature makes our technique particularly appealing in the context of clinical trials, where there may be significant costs associated with erroneously declaring that effects differ across population subgroups. TEHTrees are able to identify heterogeneous subgroups, characterize the relevant subgroups and estimate the associated treatment effects. We demonstrate the efficacy of the proposed method using a comprehensive simulation study and illustrate our method using a nutrition trial dataset to evaluate effect heterogeneity within a patient population.

stat.ME↗

arXiv2017

MEBoost: Variable Selection in the Presence of Measurement Error

We present a novel method for variable selection in regression models when covariates are measured with error. The iterative algorithm we propose, MEBoost, follows a path defined by estimating equations that correct for covariate measurement error. Via simulation, we evaluated our method and compare its performance to the recently-proposed Convex Conditioned Lasso (CoCoLasso) and to the "naive" Lasso which does not correct for measurement error. Increasing the degree of measurement error increased prediction error and decreased the probability of accurate covariate selection, but this loss of accuracy was least pronounced when using MEBoost. We illustrate the use of MEBoost in practice by analyzing data from the Box Lunch Study, a clinical trial in nutrition where several variables are based on self-report and hence measured with error.

stat.CO↗

arXiv2015

Who's good this year? Comparing the Information Content of Games in the Four Major US Sports

In the four major North American professional sports (baseball, basketball, football, and hockey), the primary purpose of the regular season is to determine which teams most deserve to advance to the playoffs. Interestingly, while the ultimate goal of identifying the best teams is the same, the number of regular season games played differs dramatically between the sports, ranging from 16 (football) to 82 (basketball and hockey) to 162 (baseball). Though length of season is partially determined by many factors including travel logistics, rest requirements, playoff structure and television contracts, it is hard to reconcile the 10-fold difference in the number of games between, for example, the NFL and MLB unless football games are somehow more "informative" than baseball games. In this paper, we aim to quantify the amount of information games yield about the relative strength of the teams involved. Our strategy is to assess how well simple paired comparison models fitted from $X%$ of the games within a season predict the outcomes of the remaining $(100-X)%$ of games, for multiple values of $X$. We compare the resulting predictive accuracy curves between seasons within the same sport and across all four sports, and find dramatic differences in the amount of information yielded by individual game results in the four major U.S. sports.

stat.AP↗

arXiv2014

A Naive Bayes machine learning approach to risk prediction using censored, time-to-event data

Predicting an individual's risk of experiencing a future clinical outcome is a statistical task with important consequences for both practicing clinicians and public health experts. Modern observational databases such as electronic health records (EHRs) provide an alternative to the longitudinal cohort studies traditionally used to construct risk models, bringing with them both opportunities and challenges. Large sample sizes and detailed covariate histories enable the use of sophisticated machine learning techniques to uncover complex associations and interactions, but observational databases are often ``messy,'' with high levels of missing data and incomplete patient follow-up. In this paper, we propose an adaptation of the well-known Naive Bayes (NB) machine learning approach for classification to time-to-event outcomes subject to censoring. We compare the predictive performance of our method to the Cox proportional hazards model which is commonly used for risk prediction in healthcare populations, and illustrate its application to prediction of cardiovascular risk using an EHR dataset from a large Midwest integrated healthcare system.

stat.ML↗

arXiv2014

Data mining for censored time-to-event data: A Bayesian network model for predicting cardiovascular risk from electronic health record data

Models for predicting the risk of cardiovascular events based on individual patient characteristics are important tools for managing patient care. Most current and commonly used risk prediction models have been built from carefully selected epidemiological cohorts. However, the homogeneity and limited size of such cohorts restricts the predictive power and generalizability of these risk models to other populations. Electronic health data (EHD) from large health care systems provide access to data on large, heterogeneous, and contemporaneous patient populations. The unique features and challenges of EHD, including missing risk factor information, non-linear relationships between risk factors and cardiovascular event outcomes, and differing effects from different patient subgroups, demand novel machine learning approaches to risk model development. In this paper, we present a machine learning approach based on Bayesian networks trained on EHD to predict the probability of having a cardiovascular event within five years. In such data, event status may be unknown for some individuals as the event time is right-censored due to disenrollment and incomplete follow-up. Since many traditional data mining methods are not well-suited for such data, we describe how to modify both modelling and assessment techniques to account for censored observation times. We show that our approach can lead to better predictive performance than the Cox proportional hazards model (i.e., a regression-based approach commonly used for censored, time-to-event data) or a Bayesian network with {\em{ad hoc}} approaches to right-censoring. Our techniques are motivated by and illustrated on data from a large U.S. Midwestern health care system.

stat.ML↗