arXiv subjects

Mike Baiocchi

Publications and source records attributed to Mike Baiocchi.

7 recordsLinked to original sources

arXiv2026

Data Reuse and the Long Shadow of Error: Splitting, Subsampling, and Prospectively Managing Inferential Errors

When multiple investigators analyze a common dataset, the data reuse induces dependence across testing procedures, affecting the distribution of errors. Existing techniques of managing dependent tests require either cross-study coordination or post-hoc correction. These methods do not apply to the current practice of uncoordinated groups of researchers independently evaluating hypotheses on a shared dataset. We investigate the use of subsampling techniques implemented at the level of individual investigators to remedy dependence with minimal coordination. To this end, we establish the asymptotic joint normality of test statistics for the class of asymptotically linear test statistics, decomposing the covariance matrix as the product of a data overlap term and a test statistic association term. This decomposition shows that controlling data overlap is sufficient to control dependence, which we formalize through the notion of Expected Variance Ratio. This enables the closed form derivation of the variance of the joint rejection region under the global null as a function of pairwise correlations of test statistics. We adopt mean-variance portfolio theory to measure risk, defining the Expected Variance Ratio (EVR) as the ratio of the expected variance of the Type I error count to the independent baseline. We show that data splitting is asymptotically optimal among rules that ensure exact independence. We then use concentration inequalities to establish that subsampling techniques implementable by individual investigators can ensure an EVR close to $1$. Finally, we show that such subsampling techniques are able to simultaneously perform a number of tests while ensuring sufficient power and that the bounded EVR is $O\left(\frac{1}{r^2}\right)$ compared to data splitting's $O\left(\frac{1}{r}\right)$, where $r$ is the per-statistic fraction of data required.

math.ST↗

arXiv2026

Synthetic Data, Information, and Prior Knowledge: Why Synthetic Data Augmentation to Boost Sample Doesn't Work for Statistical Inference

The use of synthetic data to deidentify data and to improve predictive models is well-attested to. The augmentation of datasets using synthetically generated data is an alluring proposition: in the best case, it generates realistic data \textit{in silico} at a fraction of the cost of authentic data which may be found \textit{in vivo} or \textit{in vitro}. This poses novel epistemic challenges. We contend that synthetic data augmentation is best understood as a novel way of accounting for prior knowledge. In this manuscript, we propose a definition of synthetic distributions and analyze how synthetic data augmentation interplays with standard accounts of maximum likelihood and Bayesian estimation. We observe that the marginal Fisher information contributed by synthetic data processes is subject to fundamental bounds, and enumerate obstacles to the use of synthetic data augmentation to aid in inferential tasks. We then articulate a Bayesian formulation of the way that synthetic data augmentation can be coherently understood, but argue that naive approaches to the specification of the prior are epistemically unjustifiable. This suggests that enhanced scrutiny must be placed on identifying justifiable priors to warrant the use and inclusion of data drawn from specific synthetic distributions. While our analysis shows the challenges and limitations of using synthetic data augmentation to improve upon traditional statistical model reasoning, it does suggest that augmentation is the principal approach analysts using outcome reasoning (i.e. using train/test splits to justify the analysis) can constrain an otherwise high-dimensional model space, providing an alternative to trying to encode the constraints into the potentially complex architecture of the algorithm.

stat.ME↗

arXiv2025

Data Gluttony: Epistemic Risks, Dependent Testing and Data Reuse in Large Datasets

Large-scale registries have collected vast amounts of data which has enabled investigators to efficiently conduct studies of observational data. Common practice is for investigators to use all data meeting the inclusion criteria of their study to perform their analysis. We term this common practice data gluttony. It has apparent formal justification insofar as this approach maximizes per-study power. But this comes at a cost: data reuse affects the shape of the tail distribution of inferential errors. Using the theory of risk orderings we demonstrate how positively dependent testing procedures result in strictly riskier distributions of inferential error. We identify two remedies to this state of affairs: research portfolio optimization and what we term data temperance. Research portfolio optimization requires that we formulate the enterprise of inference in a utility theoretic framework: associated to each hypothesis to be evaluated is some utility dependent on its truth as well as the impact of the statistical decision rendered on the basis of the data. Under certain models of data governance, this approach can be used to optimally allocate data usage across multiple inferential tasks. On the other hand, data temperance is a more flexible strategy for managing the distribution of inferential errors. Data temperance is the principle that an investigator use only as much data as is necessary to perform the task at hand. This is possible due to the diminishing marginal returns in power and precision in sample size. We analyze the effectiveness of data temperance at reducing the dependence across testing and develop a theory of the capacity of a static database to sustain large numbers of inferential tasks with low probability of inducing pairwise dependent testing procedures.

math.ST↗

arXiv2022

Statistical matching and subclassification with a continuous dose: characterization, algorithm, and application to a health outcomes study

Subclassification and matching are often used in empirical studies to adjust for observed covariates; however, they are largely restricted to relatively simple study designs with a binary treatment and less developed for designs with a continuous exposure. Matching with exposure doses is particularly useful in instrumental variable designs and in understanding the dose-response relationships. In this article, we propose two criteria for optimal subclassification based on subclass homogeneity in the context of having a continuous exposure dose, and propose an efficient polynomial-time algorithm that is guaranteed to find an optimal subclassification with respect to one criterion and serves as a 2-approximation algorithm for the other criterion. We discuss how to incorporate dose and use appropriate penalties to control the number of subclasses in the design. Via extensive simulations, we systematically compare our proposed design to optimal non-bipartite pair matching, and demonstrate that combining our proposed subclassification scheme with regression adjustment helps reduce model dependence for parametric causal inference with a continuous dose. We apply the new design and associated randomization-based inferential procedure to study the effect of transesophageal echocardiography (TEE) monitoring during coronary artery bypass graft (CABG) surgery on patients' post-surgery clinical outcomes using Medicare and Medicaid claims data, and find evidence that TEE monitoring lowers patients' all-cause $30$-day mortality rate.

stat.ME↗

arXiv2021

Robust Designs for Prospective Randomized Trials Surveying Sensitive Topics

We consider the problem of designing a prospective randomized trial in which the outcome data will be self-reported, and will involve sensitive topics. Our interest is in misreporting behavior, and how respondents' tendency to under- or overreport a binary outcome might affect the power of the experiment. We model the problem by assuming each individual in our study is a member of one "reporting class": a truth-teller, underreporter, overreporter, or false-teller. We show that the joint distribution of reporting classes and "response classes" (characterizing individuals' response to the treatment) will exactly define the bias and variance of the causal estimate in our experiment. Then, we propose a novel procedure for deriving sample sizes under the worst-case power corresponding to a given level of misreporting. Our problem is motivated by prior experience implementing a randomized controlled trial of a sexual violence prevention program among adolescent girls in Nairobi, Kenya.

stat.ME↗

arXiv2020

A Causal Machine Learning Framework for Predicting Preventable Hospital Readmissions

Clinical predictive algorithms are increasingly being used to form the basis for optimal treatment policies--that is, to enable interventions to be targeted to the patients who will presumably benefit most. Despite taking advantage of recent advances in supervised machine learning, these algorithms remain, in a sense, blunt instruments--often being developed and deployed without a full accounting of the causal aspects of the prediction problems they are intended to solve. Indeed, in many settings, including among patients at risk of readmission, the riskiest patients may derive less benefit from a preventative intervention compared to those at lower risk. Moreover, targeting an intervention to a population, rather than limiting it to a small group of high-risk patients, may lead to far greater overall utility if the patients with the most modifiable (or preventable) outcomes across the population could be identified. Based on these insights, we introduce a causal machine learning framework that decouples this prediction problem into causal and predictive parts, which clearly delineates the complementary roles of causal inference and prediction in this problem. We estimate treatment effects using causal forests, and characterize treatment effect heterogeneity across levels of predicted risk using these estimates. Furthermore, we show how these effect estimates could be used in concert with the modeled "payoffs" associated with successful prevention of individual readmissions to maximize overall utility. Based on data taken from before and after the implementation of a readmissions prevention intervention at Kaiser Permanente Northern California, our results suggest that nearly four times as many readmissions could be prevented annually with this approach compared to targeting this intervention using predicted risk.

stat.AP↗

arXiv2017

The causal impact of bail on case outcomes for indigent defendants

We use near-far matching, a technique for estimating causal relationships, to explore whether bail causes a higher likelihood of conviction. We find evidence of a strong causal impact. This paper was compiled as a submission to the 2017 Fairness, Accountability, and Transparency in Machine Learning (FAT ML) workshop.

stat.AP↗