arXiv subjects

Alejandro Schuler

Publications and source records attributed to Alejandro Schuler.

At least 19 recordsLinked to original sources

arXiv2026

Improving reproducibility by controlling random seed stability in machine learning based estimation via bagging

Predictions from machine learning algorithms can vary across random seeds, inducing instability in downstream debiased machine learning estimators. We formalize random seed stability via a concentration condition and prove that subbagging guarantees stability for any bounded-outcome regression algorithm. We introduce a new cross-fitting procedure, adaptive cross-bagging, which simultaneously eliminates seed dependence from both nuisance estimation and sample splitting in debiased machine learning. Numerical experiments confirm that the method achieves the targeted level of stability whereas alternatives do not. Our method incurs a small computational penalty relative to standard practice whereas alternative methods incur large penalties.

stat.ME↗

arXiv2026

Highly Adaptive Empirical Risk Minimization with Principal Components

The Highly Adaptive Lasso (HAL) delivers unprecedented guarantees in nonparametric minimum loss estimation under minimal smoothness assumptions, such as dimension-free minimax optimal rates. However, the practical use of HAL has been severely limited by its exponentially growing computationally prohibitive indicator basis expansion in moderate to high dimensions. Existing screening strategies drastically reduce this dimension but lack any theoretical justification. We introduce the Principal Component Highly Adaptive (PC-HA) family of estimators, which for the first time provide a principled and theoretically valid dimension reduction. We establish formal results on the score equations solved by these PC-HA estimators, allowing to transfer plug-in efficiency and pointwise asymptotic normality results from HAL to these PC-HA estimators, under comparable complexity control.

math.ST↗

arXiv2026

Highly Adaptive Principal Component Regression

The Highly Adaptive Lasso (HAL) is a nonparametric regression method that achieves almost dimension-free convergence rates under minimal smoothness assumptions, but its implementation can be computationally prohibitive in high dimensions due to the large design matrix it requires. The Highly Adaptive Ridge (HAR) has been proposed as a related ridge-regularized analogue. Building on both procedures, we introduce the Principal Component Highly Adaptive Lasso (PCHAL) and Principal Component Highly Adaptive Ridge (PCHAR). These estimators use an outcome-blind principal-component reduction of the HAL basis, offering substantial computational gains over HAL while achieving empirical performance comparable to HAL and HAR. We also describe an early-stopped gradient descent variant, which provides a convenient form of smooth spectral regularization without explicitly selecting a hard principal-component cutoff. Finally, we uncover that under special circumstances, the HAL kernel is identical to the covariance function of Brownian motion.

stat.ML↗

arXiv2025

"Within-trial" prognostic score adjustment is targeted maximum likelihood estimation

Adjustment for ``super'' or ``prognostic'' composite covariates has become more popular in randomized trials recently. These prognostic covariates are often constructed from historical data by fitting a predictive model of the outcome on the raw covariates. A natural question that we have been asked by applied researchers is whether this can be done without the historical data: can the prognostic covariate be constructed or derived from the trial data itself, possibly using different folds of the data, before adjusting for it? Here we clarify that such ``within-trial'' prognostic adjustment is nothing more than a form of targeted maximum likelihood estimation (TMLE), a well-studied procedure for optimal inference. We demonstrate the equivalence with a simulation study and discuss the pros and cons of within-trial prognostic adjustment (standard efficient estimation) relative to standard TMLE and standard prognostic adjustment with historical data.

stat.ME↗

arXiv2025

A Non-Parametric Sensitivity Analysis for Bounding Bias in Hybrid Control Trials

We study hybrid control trials (HCTs), in which a randomized controlled trial (RCT) is augmented with external control patients. Existing approaches for HCTs typically assume conditional exchangeability of the concurrent and external controls to identify trial-specific effects. When violated, this can induce substantial unquantified bias, which in turn limits the acceptability of HCTs in regulatory settings. We treat violations of mean exchangeability as omitted variable bias and develop a non-parametric sensitivity analysis that (i) applies to the efficient, doubly robust HCT estimator of the trial-specific ATE, and (ii) delivers sharp bounds on the bias induced by unmeasured covariates. Building on recent work in double machine learning, our approach characterizes the maximal bias in terms of two partial R-squared sensitivity parameters: the additional explanatory power that unmeasured confounders could have for the outcome regression and for trial participation. For any given choice of these parameters, we construct valid confidence bounds for bias-adjusted treatment effects and visualize critical causal gaps via contour plots and robustness values that show how strong unmeasured confounding would need to be to overturn nominally significant HCT findings. Through simulations, we show that the method (i) reliably upper-bounds true bias, (ii) restores type I error control in settings where na\"ive HCT analysis is anti-conservative, and (iii) can still deliver meaningful power gains and RCT sample-size reductions even under moderate violations of mean exchangeability. We illustrate the approach in a phase III trial on diabetes, supplemented with external controls. We discuss practical guidelines for designing and evaluating HCTs, including external-data selection, sample-size allocation, and interpretation of sensitivity contours.

stat.ME↗

arXiv2025

Targeted Deep Architectures: A TMLE-Based Framework for Robust Causal Inference in Neural Networks

Modern deep neural networks are powerful predictive tools yet often lack valid inference for causal parameters, such as treatment effects or entire survival curves. While frameworks like Double Machine Learning (DML) and Targeted Maximum Likelihood Estimation (TMLE) can debias machine-learning fits, existing neural implementations either rely on "targeted losses" that do not guarantee solving the efficient influence function equation or computationally expensive post-hoc "fluctuations" for multi-parameter settings. We propose Targeted Deep Architectures (TDA), a new framework that embeds TMLE directly into the network's parameter space with no restrictions on the backbone architecture. Specifically, TDA partitions model parameters - freezing all but a small "targeting" subset - and iteratively updates them along a targeting gradient, derived from projecting the influence functions onto the span of the gradients of the loss with respect to weights. This procedure yields plug-in estimates that remove first-order bias and produce asymptotically valid confidence intervals. Crucially, TDA easily extends to multi-dimensional causal estimands (e.g., entire survival curves) by merging separate targeting gradients into a single universal targeting update. Theoretically, TDA inherits classical TMLE properties, including double robustness and semiparametric efficiency. Empirically, on the benchmark IHDP dataset (average treatment effects) and simulated survival data with informative censoring, TDA reduces bias and improves coverage relative to both standard neural-network estimators and prior post-hoc approaches. In doing so, TDA establishes a direct, scalable pathway toward rigorous causal inference within modern deep architectures for complex multi-parameter targets.

cs.LG↗

arXiv2025

Powering RCTs for marginal effects with GLMs using prognostic score adjustment

In randomized clinical trials (RCTs), the accurate estimation of marginal treatment effects is crucial for determining the efficacy of interventions. Enhancing the statistical power of these analyses is a key objective for statisticians. The increasing availability of historical data from registries, prior trials, and health records presents an opportunity to improve trial efficiency. However, many methods for historical borrowing compromise strict type-I error rate control. Building on the work by Schuler et al. [2022] on prognostic score adjustment for linear models, this paper extends the methodology to the plug-in analysis proposed by Rosenblum et al. [2010] using generalized linear models (GLMs) to further enhance the efficiency of RCT analyses without introducing bias. Specifically, we train a prognostic model on historical control data and incorporate the resulting prognostic scores as covariates in the plug-in GLM analysis of the trial data. This approach leverages the predictive power of historical data to improve the precision of marginal treatment effect estimates. We demonstrate that this method achieves local semi-parametric efficiency under the assumption of an additive treatment effect on the link scale. We expand the GLM plug-in method to include negative binomial regression. Additionally, we provide a straightforward formula for conservatively estimating the asymptotic variance, facilitating power calculations that reflect these efficiency gains. Our simulation study supports the theory. Even without an additive treatment effect, we observe increased power or reduced standard error. While population shifts from historical to trial data may dilute benefits, they do not introduce bias.

stat.ME↗

arXiv2025

Score-Preserving Targeted Maximum Likelihood Estimation

Targeted maximum likelihood estimators (TMLEs) are asymptotically optimal among regular, asymptotically linear estimators. In small samples, however, we may be far from "asymptopia" and not reap the benefits of optimality. Here we propose a variant (score-preserving TMLE; SP-TMLE) that leverages an initial estimator defined as the solution of a large number of possibly data-dependent score equations. Instead of targeting only the efficient influence function in the TMLE update to knock out the plug-in bias, we also target the already-solved scores. Solving additional scores reduces the remainder term in the von-Mises expansion of our estimator because these scores may come close to spanning higher-order influence functions. The result is an estimator with better finite-sample performance. We demonstrate our approach in simulation studies leveraging the (relaxed) highly adaptive lasso (HAL) as our initial estimator. These simulations show that in small samples SP-TMLE has reduced bias relative to plug-in HAL and reduced variance relative to vanilla TMLE, blending the advantages of the two approaches. We also observe improved estimation of standard errors in small samples.

stat.ME↗

arXiv2025

RieszBoost: Gradient Boosting for Riesz Regression

Answering causal questions often involves estimating linear functionals of conditional expectations, such as the average treatment effect or the effect of a longitudinal modified treatment policy. By the Riesz representation theorem, these functionals can be expressed as the expected product of the conditional expectation of the outcome and the Riesz representer, a key component in doubly robust estimation methods. Traditionally, the Riesz representer is estimated indirectly by deriving its explicit analytical form, estimating its components, and substituting these estimates into the known form (e.g., the inverse propensity score). However, deriving or estimating the analytical form can be challenging, and substitution methods are often sensitive to practical positivity violations, leading to higher variance and wider confidence intervals. In this paper, we propose a novel gradient boosting algorithm to directly estimate the Riesz representer without requiring its explicit analytical form. This method is particularly suited for tabular data, offering a flexible, nonparametric, and computationally efficient alternative to existing methods for Riesz regression. Through simulation studies, we demonstrate that our algorithm performs on par with or better than indirect estimation techniques across a range of functionals, providing a user-friendly and robust solution for estimating causal quantities.

stat.ML↗

arXiv2024

Highly Adaptive Ridge

In this paper we propose the Highly Adaptive Ridge (HAR): a regression method that achieves a $n^{-1/3}$ dimension-free L2 convergence rate in the class of right-continuous functions with square-integrable sectional derivatives. This is a large nonparametric function class that is particularly appropriate for tabular data. HAR is exactly kernel ridge regression with a specific data-adaptive kernel based on a saturated zero-order tensor-product spline basis expansion. We use simulation and real data to confirm our theory. We demonstrate empirical performance better than state-of-the-art algorithms for small datasets in particular.

stat.ML↗

arXiv2024

Data-Adaptive Identification of Effect Modifiers through Stochastic Shift Interventions and Cross-Validated Targeted Learning

In epidemiology, identifying subpopulations that are particularly vulnerable to exposures and those who may benefit differently from exposure-reducing interventions is essential. Factors such as age, gender-specific vulnerabilities, and physiological states such as pregnancy are critical for policymakers when setting regulatory guidelines. However, current semi-parametric methods for estimating heterogeneous treatment effects are often limited to binary exposures and can function as black boxes, lacking clear, interpretable rules for subpopulation-specific policy interventions. This study introduces a novel method that uses cross-validated targeted minimum loss-based estimation (TMLE) paired with a data-adaptive target parameter strategy to identify subpopulations with the most significant differential impact of simulated policy interventions that reduce exposure. Our approach is assumption-lean, allowing for the integration of machine learning while still yielding valid confidence intervals. We demonstrate the robustness of our methodology through simulations and application to data from the National Health and Nutrition Examination Survey. Our analysis of NHANES data on persistent organic pollutants (POPs) and leukocyte telomere length (LTL) identified age as a significant effect modifier. Specifically, we found that exposure to 3,3',4,4',5-pentachlorobiphenyl (PCNB) consistently had a differential impact on LTL, with a one-standard deviation reduction in exposure leading to a more pronounced increase in LTL among younger populations compared to older ones. We offer our method as an open-source software package, EffectXshift, enabling researchers to investigate the effect modification of continuous exposures. The EffectXshift package provides clear and interpretable results, informing targeted public health interventions and policy decisions.

stat.ME↗

arXiv2024

Bridging Binarization: Causal Inference with Dichotomized Continuous Exposures

The average treatment effect (ATE) is a common parameter estimated in causal inference literature, but it is only defined for binary exposures. Thus, despite concerns raised by some researchers, many studies seeking to estimate the causal effect of a continuous exposure create a new binary exposure variable by dichotomizing the continuous values into two categories. In this paper, we affirm binarization as a statistically valid method for answering causal questions about continuous exposures by showing the equivalence between the binarized ATE and the difference in the average outcomes of two specific modified treatment policies. These policies impose cut-offs corresponding to the binarized exposure variable and assume preservation of relative self-selection. Relative self-selection is the ratio of the probability density of an individual having an exposure equal to one value of the continuous exposure variable versus another. The policies assume that, for any two values of the exposure variable with non-zero probability density after the cut-off, this ratio will remain unchanged. Through this equivalence, we clarify the assumptions underlying binarization and discuss how to properly interpret the resulting estimator. Additionally, we introduce a new target parameter that can be computed after binarization that considers the observed world as a benchmark. We argue that this parameter addresses more relevant causal questions than the traditional binarized ATE parameter. We present a simulation study to illustrate the implications of these assumptions when analyzing data and to demonstrate how to correctly implement estimators of the parameters discussed. Finally, we present an application of this method to evaluate the effect of a law in the state of California which seeks to limit exposures to oil and gas wells on birth outcomes to further illustrate the underlying assumptions.

stat.ME↗

arXiv2023

Unveiling Causal Mediation Pathways in High-Dimensional Mixed Exposures: A Data-Adaptive Target Parameter Strategy

Mediation analysis in causal inference typically concentrates on one binary exposure, using deterministic interventions to split the average treatment effect into direct and indirect effects through a single mediator. Yet, real-world exposure scenarios often involve multiple continuous exposures impacting health outcomes through varied mediation pathways, which remain unknown a priori. Addressing this complexity, we introduce NOVAPathways, a methodological framework that identifies exposure-mediation pathways and yields unbiased estimates of direct and indirect effects when intervening on these pathways. By pairing data-adaptive target parameters with stochastic interventions, we offer a semi-parametric approach for estimating causal effects in the context of high-dimensional, continuous, binary, and categorical exposures and mediators. In our proposed cross-validation procedure, we apply sequential semi-parametric regressions to a parameter-generating fold of the data, discovering exposure-mediation pathways. We then use stochastic interventions on these pathways in an estimation fold of the data to construct efficient estimators of natural direct and indirect effects using flexible machine learning techniques. Our estimator proves to be asymptotically linear under conditions necessitating n to the negative quarter consistency of nuisance function estimation. Simulation studies demonstrate the square root n consistency of our estimator when the exposure is quantized, whereas for truly continuous data, approximations in numerical integration prevent square root n consistency. Our NOVAPathways framework, part of the open-source SuperNOVA package in R, makes our proposed methodology for high-dimensional mediation analysis available to researchers, paving the way for the application of modified exposure policies which can delivery more informative statistical results for public policy.

stat.ME↗

arXiv2023

Prognostic Adjustment with Efficient Estimators to Unbiasedly Leverage Historical Data in Randomized Trials

Although randomized controlled trials (RCTs) are a cornerstone of comparative effectiveness, they typically have much smaller sample size than observational studies because of financial and ethical considerations. Therefore there is interest in using plentiful historical data (either observational data or prior trials) to reduce trial sizes. Previous estimators developed for this purpose rely on unrealistic assumptions, without which the added data can bias the treatment effect estimate. Recent work proposed an alternative method (prognostic covariate adjustment) that imposes no additional assumptions and increases efficiency in trial analyses. The idea is to use historical data to learn a prognostic model: a regression of the outcome onto the covariates. The predictions from this model, generated from the RCT subjects' baseline variables, are then used as a covariate in a linear regression analysis of the trial data. In this work, we extend prognostic adjustment to trial analyses with nonparametric efficient estimators, which are more powerful than linear regression. We provide theory that explains why prognostic adjustment improves small-sample point estimation and inference without any possibility of bias. Simulations corroborate the theory: efficient estimators using prognostic adjustment compared to without provides greater power (i.e., smaller standard errors) when the trial is small. Population shifts between historical and trial data attenuate benefits but do not introduce bias. We showcase our estimator using clinical trial data provided by Novo Nordisk A/S that evaluates insulin therapy for individuals with type II diabetes.

stat.ME↗

arXiv2023

Semiparametric Discovery and Estimation of Interaction in Mixed Exposures using Stochastic Interventions

This study introduces a nonparametric definition of interaction and provides an approach to both interaction discovery and efficient estimation of this parameter. Using stochastic shift interventions and ensemble machine learning, our approach identifies and quantifies interaction effects through a model-independent target parameter, estimated via targeted maximum likelihood and cross-validation. This method contrasts the expected outcomes of joint interventions with those of individual interventions. Validation through simulation and application to the National Institute of Environmental Health Sciences Mixtures Workshop data demonstrate the efficacy of our method in detecting true interaction directions and its consistency in identifying significant impacts of furan exposure on leukocyte telomere length. Our method, called InterXshift, advances the ability to analyze multi-exposure interactions within high-dimensional data, offering significant methodological improvements to understand complex exposure dynamics in health research. We provide peer-reviewed open-source software that employs or proposed methodology in the InterXshift R package.

stat.ME↗

arXiv2023

Discovery of Critical Thresholds in Mixed Exposures and Estimation of Policy Intervention Effects using Targeted Learning

Traditional regulations of chemical exposure tend to focus on single exposures, overlooking the potential amplified toxicity due to multiple concurrent exposures. We are interested in understanding the average outcome if exposures were limited to fall under a multivariate threshold. Because threshold levels are often unknown a priori, we provide an algorithm that finds exposure threshold levels where the expected outcome is maximized or minimized. Because both identifying thresholds and estimating policy effects on the same data would lead to overfitting bias, we also provide a data-adaptive estimation framework, which allows for both threshold discovery and policy estimation. Simulation studies show asymptotic convergence to the optimal exposure region and to the true effect of an intervention. We demonstrate how our method identifies true interactions in a public synthetic mixture data set. Finally, we applied our method to NHANES data to discover metal exposures that have the most harmful effects on telomere length. We provide an implementation in the CVtreeMLE R package.

stat.ME↗

arXiv2022

Prognostic Covariate Adjustment for Binary Outcomes Using Stratification

Covariate adjustment and methods of incorporating historical data in randomized clinical trials (RCTs) each provide opportunities to increase trial power. We unite these approaches for the analysis of RCTs with binary outcomes based on the Cochran-Mantel-Haenszel (CMH) test for marginal risk ratio (RR). In PROCOVA-CMH, subjects are stratified on a single prognostic covariate reflective of their predicted outcome on the control treatment (e.g. placebo). This prognostic score is generated based on baseline covariates through a model trained on historical data. We propose two closed-form prospective estimators for the asymptotic sampling variance of the log RR that rely only on values obtainable from observed historical outcomes and the prognostic model. Importantly, these estimators can be used to inform sample size during trial planning. PROCOVA-CMH demonstrates type I error control and appropriate asymptotic coverage for valid inference. Like other covariate adjustment methods, PROCOVA-CMH can reduce the variance of the treatment effect estimate when compared to an unadjusted (unstratified) CMH analysis. In addition to statistical methods, simulations and a case study in Alzheimer's Disease are given to demonstrate performance. Results show that PROCOVA-CMH can provide a gain in power, which can be used to conduct smaller trials.

stat.ME↗

arXiv2022

Lassoed Tree Boosting

Gradient boosting performs exceptionally in most prediction problems and scales well to large datasets. In this paper we prove that a ``lassoed'' gradient boosted tree algorithm with early stopping achieves faster than $n^{-1/4}$ L2 convergence in the large nonparametric space of cadlag functions of bounded sectional variation. This rate is remarkable because it does not depend on the dimension, sparsity, or smoothness. We use simulation and real data to confirm our theory and demonstrate empirical performance and scalability on par with standard boosting. Our convergence proofs are based on a novel, general theorem on early stopping with empirical loss minimizers of nested Donsker classes.

stat.ML↗