arXiv subjects

Donald P. Green

Publications and source records attributed to Donald P. Green.

9 recordsLinked to original sources

arXiv2026

Covariate Adjustment in Randomized Experiments: A Unified Framework for Decision and Practice

Should researchers adjust for covariates in randomized experiments, and if so, how? The literature offers three distinct prescriptions: do not adjust because randomization guarantees unbiasedness; adjust for outcome-prognostic covariates to improve precision; or adjust for covariates imbalanced between treatment arms. These competing prescriptions create confusion and uncertainty. We develop a unified framework for decision and practice. Given available information, we show that the optimal correction is what we call ex-post bias. The only relevant criterion for adjustment is prognosticity for ex-post bias; neither raw covariate imbalance nor outcome prognosticity is sufficient by itself. We also show that correcting imbalance and improving precision are two sides of the same decision problem. We develop two estimation approaches, one of which recovers familiar adjustment estimators and provides a new theoretical justification for them. Simulations compare alternative covariate-selection and adjustment strategies. Overall, our framework provides a unified foundation for covariate adjustment in randomized experiments.

stat.ME↗

arXiv2026

The Illusion of Learning from Observational Data: An Empirical Bayes Perspective

Randomized experiments have long been the gold standard for scientists seeking to learn about cause and effect. When randomized experiments are infeasible, scientists often resort to observational studies, which are widely available and often large but rely on untestable assumptions that, when violated, may result in biased estimates. Uncertainty about bias leads to a phenomenon known as the illusion of learning from observational research (Gerber, Green and Kaplan, 2004a): absent prior information about bias, observational results cannot meaningfully contribute to the estimation of a causal parameter. To shatter the illusion, we take an empirical Bayes perspective. We show that the distribution of observational biases can be learned from calibration studies-experiments that target a causal effect that is known a priori to be zero. Calibration identifies the distribution of observational bias and allows observational studies to inform the estimation of causal parameters via empirical Bayes shrinkage. We formalize the illusion phenomenon in an empirical Bayes setting and show that, with an increasing number of calibration and observation studies, both the bias distribution and the causal effect can be consistently recovered. We illustrate our method through a simulation study and a semi-synthetic application based on Ferraro and Miranda (2013)'s water-usage experiment.

stat.ME↗

arXiv2026

Nonparametric Identification and Estimation of Causal Effects on Latent Outcomes

How should researchers conduct causal inference when the outcome of interest is latent and measured imperfectly by multiple indicators? We develop a general nonparametric framework for identifying and estimating average treatment effects on latent outcomes in randomized experiments. We show that latent-outcome estimation faces two distinct noncomparability challenges. First, across studies, different measurement systems may cause estimators to target different empirical quantities even when the underlying latent treatment effect is the same. Second, within a study, different indicators may have different and possibly nonlinear relationships with the same latent outcome, making them not directly comparable. To address these challenges, we propose a design-based approach built around nonparametric bridge functions. We show that these bridge functions can be characterized and identified. Estimation relies on a debiasing procedure that permits valid inference even when the bridge functions are weakly identified. Simulations demonstrate that standard methods, such as principal components analysis and inverse covariance weighting, can generate spurious cross-study differences, whereas our approach recovers comparable latent treatment effects. Overall, the framework provides both a general strategy for causal inference with latent outcomes and practical guidance for designing measurements that support identification, comparability, and efficient estimation.

stat.ME↗

arXiv2026

Causal Inference for Experiments with Latent Outcomes: Key Results and Their Implications for Design and Analysis

How should researchers analyze randomized experiments in which the main outcome is latent and measured in multiple ways but each measure contains some degree of error? We first identify a critical study-specific noncomparability problem in existing methods for handling multiple measurements, which often rely on strong modeling assumptions or arbitrary standardization. Such approaches render the resulting estimands noncomparable across studies. To address the problem, we describe design-based approaches that enable researchers to identify causal parameters of interest, suggest ways that experimental designs can be augmented so as to make assumptions more credible, and discuss empirical tests of key assumptions. We show that when experimental researchers invest appropriately in multiple outcome measures, an optimally weighted scaled index of these measures enables researchers to obtain efficient and interpretable estimates of causal parameters by applying standard regression. An empirical application illustrates the gains in precision and robustness that multiple outcome measures can provide.

econ.EM↗

arXiv2025

The Sequential Nature of Science: Quantifying Learning from a Sequence of Studies

Scientific progress is inherently sequential: collective knowledge is updated as new studies enter the literature. We propose the sequential meta-analysis research trace (SMART), which quantifies the influence of each study at the time it enters the literature. In contrast to classical meta-analysis, our method can capture how new studies may cast doubt on previously held beliefs, increasing collective uncertainty. For example, a new study may present a methodological critique of prior work and propose a superior method. Even small studies, which may not materially affect a retrospective meta-analysis, can be influential at the time they appeared. To contrast SMART with classical meta-analysis, we re-analyze two meta-analysis datasets, from psychology and labor economics. One assembles studies using a single methodology; the other contains studies that predate or follow an important methodological innovation. Our formalization of sequential learning highlights the importance of methodological innovation that might otherwise be overlooked by classical meta-analysis.

stat.ME↗

arXiv2025

Sharp Bounds on the Variance of General Regression Adjustment in Randomized Experiments

A growing statistical literature focuses on causal inference in the context of experiments where the target of inference is the average treatment effect in a finite population and random assignment determines which subjects are allocated to one of the experimental conditions. In this framework, variances of average treatment effect estimators remain unidentified because they depend on the covariance between treated and untreated potential outcomes, which are never jointly observed. Conventional variance estimators are upwardly biased. Aronow, Green and Lee [Ann. Statist. 42(3): 850-871 (June 2014)] provide an estimator for the variance of the difference-in-means estimator that is asymptotically sharp. In practice, researchers often use some form of covariate adjustment, such as linear regression, when estimating the average treatment effect. Adapting propositions from empirical process theory, we extend the result in (Aronow et al., 2014), providing asymptotically sharp variance bounds for general regression adjustment. We apply these results to linear regression adjustment and show benefits both in a simulation and in three empirical applications drawn from different disciplines.

stat.ME↗

arXiv2025

Quantifying How Much Has Been Learned from a Research Study

How much does a research study contribute to a scientific literature? We propose a learning metric to quantify how much a research community learns from a given study. To do so, we adopt a Bayesian perspective and assess changes in the community's beliefs once updated with a new study's evidence. We recommend the Wasserstein-2 distance as a way to describe how the research community's prior beliefs change to incorporate a study's findings. We illustrate this approach through stylized examples and empirical applications, showing how it differs from more traditional evaluative standards, such as statistical significance. We then extend the framework to the prospective setting, offering a way for decision-makers to evaluate the expected amount of learning from a proposed study. While assessments about what has or could be learned from a research program are often expressed informally, our learning metric provides a principled tool for judging scientific contributions. By formalizing these judgments, our measure has the potential to allow for more transparent assessments of past and prospective research contributions.

stat.ME↗

arXiv2014

Combining List Experiment and Direct Question Estimates of Sensitive Behavior Prevalence

Survey respondents may give untruthful answers to sensitive questions when asked directly. In recent years, researchers have turned to the list experiment (also known as the item count technique) to overcome this difficulty. While list experiments may be less prone to bias than direct questioning, list experiments are also more susceptible to sampling variability. We show that researchers do not have to abandon direct questioning altogether in order to gain the advantages of list experimentation. We develop a nonparametric estimator of the prevalence of sensitive behaviors that combines list experimentation and direct questioning. We prove that this estimator is asymptotically more efficient than the standard difference-in-means estimator, and we provide a basis for inference using Wald-type confidence intervals. Additionally, leveraging information from the direct questioning, we derive two nonparametric placebo tests of the identifying assumptions for the list experiment. We demonstrate the effectiveness of our combined estimator and placebo tests with an original survey experiment.

stat.AP↗

arXiv2014

Sharp bounds on the variance in randomized experiments

We propose a consistent estimator of sharp bounds on the variance of the difference-in-means estimator in completely randomized experiments. Generalizing Robins [Stat. Med. 7 (1988) 773-785], our results resolve a well-known identification problem in causal inference posed by Neyman [Statist. Sci. 5 (1990) 465-472. Reprint of the original 1923 paper]. A practical implication of our results is that the upper bound estimator facilitates the asymptotically narrowest conservative Wald-type confidence intervals, with applications in randomized controlled and clinical trials.

math.ST↗