arXiv subjects

Guido Imbens

Publications and source records attributed to Guido Imbens.

At least 19 recordsLinked to original sources

arXiv2026

PLRD: Partially Linear Regression Discontinuity Inference

Regression discontinuity designs have become one of the most popular research designs in empirical economics. We argue, however, that the widely used approaches to building confidence intervals in regression discontinuity designs often exhibit suboptimal behavior in practice. We propose a new estimator, the partially linear regression discontinuity (PLRD) estimator that, in set of a simulation studies carefully calibrated to twelve high-profile applications of regression discontinuity designs, has substantially lower estimation error than available comparison methods. Throughout our experiments, the confidence intervals built using PLRD are both valid and short. We also provide large-sample guarantees for PLRD. Our simulation study serves as a general template for how new econometric methods can be credibly evaluated relative to the existing alternatives by constructing simulation designs that generate synthetic data indistinguishable from the original data using the Wasserstein generative adversarial network methodology.

econ.EM↗

arXiv2026

Demonstration Experiments

Adaptive experiments are used extensively in online platforms, healthcare and biotechnology, and the social sciences. Often, the primary goal is not to precisely estimate a treatment effect but to demonstrate that at least one candidate intervention yields a positive effect, for some subpopulation and on some measured outcome. We formalize this objective as testing the global null in a threshold bandit framework, and develop two inference procedures that are valid under general adaptive sampling: one that pools information across promising arms, and one based on time-uniform multiple testing of individual arm means. To support the latter, we establish a moderate-deviations principle for the sequential $t$-statistic, justifying asymptotic confidence sequences in settings where the number of arms is large relative to the sample size. To illustrate how adaptive designs can target the proposed statistics, we recast experimental design as bandit optimization with an arm's reward given by its signal-to-noise ratio, and analyze an allocation rule for which we establish a logarithmic regret bound. We apply the methods in a simulation study of targeting unconditional cash transfer programs.

math.ST↗

arXiv2026

Estimating Causal Effects from Data Generated by Stochastic Algorithms

Recommendation systems and chatbots present content to users, typically using stochastic algorithms that select the content based on user characteristics or context. Examples of content include chat responses, videos, or items available for purchase. Scientists and application developers are often interested in whether characteristics of content increase outcomes such as user engagement. Estimates of such causal effects may guide content providers to generate content that emphasize desirable features. However, in settings with a large content library or where content is generated uniquely for a given user, it can be difficult to use observational data to learn the causal effect of content features, because the content a user sees is tailored to that user, and because content varies in many dimensions. This paper proposes a new method for estimating the impact of content features using observational data, when the algorithm that determines user exposure incorporates some randomization, and when two additional data elements are logged for each user: $(i)$ the identity of at least one item that could have been exposed to the user, but was not (the unexposed item); $(ii)$ an estimate of the ratio of the probability that the unexposed item would have been shown to the probability that the exposed item was shown. We show that causal effects of features are identified in this setting, even in the presence of unobserved confounders that affect both user preferences and the identity of the considered pair of items (exposed and unexposed). Our estimator differs from prior approaches in terms of what data is used and how the estimator is constructed.

stat.ME↗

arXiv2026

Regression Adjustments for Double Randomization in Two-Sided Marketplaces

Multiple randomization designs (MRDs) are a class of experimental designs used to handle interference in two-sided marketplaces. We investigate regression adjustment strategies for estimating total, spillover, and direct effects in MRDs. We derive minimum asymptotic variance estimators among a broad class of linearly adjusted estimators, without assuming a linear model on the potential outcomes. Surprisingly, the optimal regression adjustments are estimable from data and are generally different from regression adjustments in classical randomized experiments. For example, one such optimal estimator for the direct effect corresponds to a weighted regression with interacted two-way fixed effects. We establish model-robustness properties, central limit theorems, and inferential methods for our estimators, relying on improved theoretical results for MRD experiments. Our results provide the analog of classical regression adjustments for marketplace experiments. Numerical simulations demonstrate a considerable increase in efficiency over simpler approaches, enabling better inference when running MRDs.

stat.ME↗

arXiv2026

Causal clustering: design of cluster experiments under network interference

This paper studies the design of cluster experiments to estimate the global treatment effect in the presence of network spillovers. We provide a framework to choose the clustering that minimizes the worst-case mean-squared error of the estimated global effect. We show that optimal clustering solves a novel penalized min-cut optimization problem computed via off-the-shelf semi-definite programming algorithms. Our analysis also characterizes simple conditions to choose between any two cluster designs, including choosing between a cluster or individual-level randomization. We illustrate the method's properties using unique network data from the universe of Facebook's users and existing data from a field experiment.

econ.EM↗

arXiv2026

Power Analysis is Essential: High-Powered Tests Suggest Minimal to No Effect of Rounded Shapes on Click-Through Rates

Underpowered studies (below 50% power) suffer from the winner's curse: A statistically significant positive estimate must exaggerate the true treatment effect to meet the significance threshold. A study by Dipayan Biswas, Annika Abell, and Roger Chacko published in the Journal of Consumer Research (2023) reported that in an A/B test, simply rounding the corners of square buttons increased the online click-through rate by 55% (p-value 0.037)$\unicode{x2014}$a striking finding with potentially wide-ranging implications for a digital industry that is seeking to enhance consumer engagement. Drawing on our experience with tens of thousands of A/B tests, many involving similar user interface modifications, we found this dramatic claim implausibly large. To evaluate the claim and provide a more accurate estimate of the treatment effect, we conducted three high-powered A/B tests, each involving over two thousand times more users than the original study. All three experiments yielded effect size estimates that were approximately two orders of magnitude smaller than initially reported, with 95% confidence intervals that include zero (i.e., not statistically significant at the 0.05 level). Two additional independent replications by Evidoo found similarly small effects. These findings underscore the critical importance of power analysis and experimental design in increasing trust and reproducibility of results.

stat.ME↗

arXiv2026

Estimating the Value of Evidence-Based Decision Making

In an era of data abundance, statistical evidence is increasingly critical for business and policy decisions. Yet, organizations lack empirical tools to assess the value of evidence-based decision making (EBDM), optimize statistical precision, and balance the costs of evidence-gathering strategies against their benefits. To tackle these challenges, this article introduces an empirical framework to estimate the value of EBDM and evaluate the return on investment in statistical precision and project ideation. The framework leverages parametric and nonparametric empirical Bayes methods to account for parameter heterogeneity and measure how statistical precision changes the value of evidence. The value extracted from statistical evidence depends critically on how organizations translate evidence into policy decisions. Commonly used decision rules based on statistical significance can leave substantial value unrealized and, in some cases, generate negative expected value.

stat.ME↗

arXiv2026

Triply Robust Panel Estimators

This paper studies estimation of causal effects in a panel data setting. We introduce a new estimator, the Triply RObust Panel (TROP) estimator, that combines (i) a flexible model for the potential outcomes based on a low-rank factor structure on top of a two-way-fixed effect specification, with (ii) unit weights intended to upweight units similar to the treated units and (iii) time weights intended to upweight time periods close to the treated time periods. We study the performance of the estimator in a set of simulations designed to closely match several commonly studied real data sets. We find that there is substantial variation in the performance of the estimators across the settings considered. The proposed estimator outperforms two-way-fixed-effect/difference-in-differences, synthetic control, matrix completion and synthetic-difference-in-differences estimators. We investigate what features of the data generating process lead to this performance, and assess the relative importance of the three components of the proposed estimator. We have two recommendations. Our preferred strategy is that researchers use simulations closely matched to the data they are interested in, along the lines discussed in this paper, to investigate which estimators work well in their particular setting. A simpler approach is to use more robust estimators such as synthetic difference-in-differences or the new triply robust panel estimator which we find to substantially outperform two-way fixed effect estimators in many empirically relevant settings.

stat.ME↗

arXiv2026

Scalable Decisions using a Bayesian Decision-Theoretic Approach

Randomized controlled experiments assess new policy impacts on performance metrics to inform launch decisions. Traditional approaches evaluate metrics independently despite correlations, and mixed results (e.g., positive revenue impact, negative customer experience) require manual judgment, hindering scalability. We propose a Bayesian decision-theoretic framework that systematically incorporates multiple objectives and trade-offs by comparing expected risks across decisions. Our approach combines experimenter-defined loss functions with observed evidence, using hierarchical models to leverage historical experiment learnings for prior information on treatment effects. Through real and simulated Amazon supply chain experiments, we demonstrate that compared to null hypothesis statistical testing, our method increases estimation efficiency via informative hierarchical priors and simplifies decision-making by systematically incorporating business preferences and costs for comprehensive, scalable decisions.

stat.AP↗

arXiv2026

Long-Term Causal Inference with Many Noisy Proxies

We propose a method for estimating long-term treatment effects with many short-term proxy outcomes: a central challenge when experimenting on digital platforms. We formalize this challenge as a latent variable problem where observed proxies are noisy measures of a low-dimensional set of unobserved surrogates that mediate treatment effects. Through theoretical analysis and simulations, we demonstrate that regularized regression methods substantially outperform naive proxy selection. We show in particular that the bias of Ridge regression decreases as more proxies are added, with closed-form expressions for the bias-variance tradeoff. We illustrate our method with an empirical application to the California GAIN experiment.

econ.EM↗

arXiv2025

Multiple Randomization Designs: Estimation and Inference with Interference

Classical designs of randomized experiments, going back to Fisher and Neyman in the 1930s still dominate practice even in online experimentation. However, such designs are of limited value for answering standard questions in settings, common in marketplaces, where multiple populations of agents interact strategically, leading to complex patterns of spillover effects. In this paper, we discuss new experimental designs and corresponding estimands to account for and capture these complex spillovers. We derive the finite-sample properties of tractable estimators for main effects, direct effects, and spillovers, and present associated central limit theorems.

stat.ME↗

arXiv2025

Multiple Randomization Designs: Estimation and Inference with Interference

Completely randomized experiments, originally developed by Fisher and Neyman in the 1930s, are still widely used in practice, even in online experimentation. However, such designs are of limited value for answering standard questions in marketplaces, where multiple populations of agents interact strategically, leading to complex patterns of spillover effects. In this paper, we derive the finite-sample properties of tractable estimators for "Simple Multiple Randomization Designs" (SMRDs), a new class of experimental designs which account for complex spillover effects in randomized experiments. Our derivations are obtained under a natural and general form of cross-unit interference, which we call "local interference". We discuss the estimation of main effects, direct effects, and spillovers, and present associated central limit theorems.

stat.ME↗

arXiv2025

Estimating Variances for Causal Panel Data Estimators

There has been a recent surge in research on causal panel data models, leading to many new estimators for average causal effects. However, researchers have paid less attention to quantifying the precision of these estimators. This paper addresses that gap by studying the problem of variance estimation in causal panel settings. We develop a unified framework for comparing the three main variance estimators used in these settings: regression-based, Unit-Placebo, and Time-Placebo estimators. We show that each relies on a distinct exchangeability assumption and, correspondingly, each targets a different conditional variance. We find that, under some assumptions, all three estimators are all valid, but that their statistical power differs substantially depending on the heteroskedasticity present in the data. Building on these insights, we propose a new variance estimator that flexibly accounts for heteroskedasticity across the unit and time dimensions, and delivers superior statistical power in realistic panel data settings.

econ.EM↗

arXiv2025

Challenges in Statistics: A Dozen Challenges in Causality and Causal Inference

Causality and causal inference have emerged as core research areas at the interface of modern statistics and domains including biomedical sciences, social sciences, computer science, and beyond. The field's inherently interdisciplinary nature -- particularly the central role of incorporating domain knowledge -- creates a rich and varied set of statistical challenges. Much progress has been made, especially in the last three decades, but there remain many open questions. Our goal in this discussion is to outline research directions and open problems we view as particularly promising for future work. Throughout we emphasize that advancing causal research requires a wide range of contributions, from novel theory and methodological innovations to improved software tools and closer engagement with domain scientists and practitioners.

stat.ME↗

arXiv2025

Causal Inference when Intervention Units and Outcome Units Differ

We study causal inference in settings characterized by interference with a bipartite structure. There are two distinct sets of units: intervention units to which an intervention can be applied and outcome units on which the outcome of interest can be measured. Outcome units may be affected by interventions on some, but not all, intervention units, as captured by a bipartite graph. Examples of this setting can be found in analyses of the impact of pollution abatement in plants on health outcomes for individuals, or the effect of transportation network expansions on regional economic activity. We introduce and discuss a variety of old and new causal estimands for these bipartite settings. We do not impose restrictions on the functional form of the exposure mapping and the potential outcomes, thus allowing for heterogeneity, non-linearity, non-additivity, and potential interactions in treatment effects. We propose unbiased weighting estimators for these estimands from a design-based perspective, based on the knowledge of the bipartite network under general experimental designs. We derive their variance and prove consistency for increasing number of outcome units. Using the Chinese high-speed rail construction study, analyzed in Borusyak and Hull [2023], we discuss non-trivial positivity violations that depend on the estimands, the adopted experimental design, and the structure of the bipartite graph.

stat.ME↗

arXiv2025

Robust and efficient multiple-unit switchback experimentation

User-randomized A/B testing has emerged as the gold standard for online experimentation. However, when this kind of approach is not feasible due to legal, ethical or practical considerations, experimenters have to consider alternatives like item-randomization. Item-randomization is often met with skepticism due to its poor empirical performance. To fill this gap, in this paper we introduce a novel and rich class of experimental designs, "Regular Balanced Switchback Designs" (RBSDs). At their core, RBSDs work by randomly changing treatment assignments over both time and items. After establishing the properties of our designs in a potential outcomes framework, characterizing assumptions and conditions under which corresponding estimators are resilient to the presence of carryover effects, we show empirically via both realistic simulations and real e-commerce data that RBSDs systematically outperform standard item-randomized and non-balanced switchback approaches by yielding much more accurate estimates of the causal effects of interest without incurring any additional bias.

stat.ME↗

arXiv2025

Admissibility of Completely Randomized Trials: A Large-Deviation Approach

When an experimenter has the option of running an adaptive trial, is it admissible to ignore this option and run a non-adaptive trial instead? We provide a negative answer to this question in the best-arm identification problem, where the experimenter aims to allocate measurement efforts judiciously to confidently deploy the most effective treatment arm. We find that, whenever there are at least three treatment arms, there exist simple adaptive designs that universally and strictly dominate non-adaptive completely randomized trials. This dominance is characterized by a notion called efficiency exponent, which quantifies a design's statistical efficiency when the experimental sample is large. Our analysis focuses on the class of batched arm elimination designs, which progressively eliminate underperforming arms at pre-specified batch intervals. We characterize simple sufficient conditions under which these designs universally and strictly dominate completely randomized trials. These results resolve the second open problem posed in Qin [2022].

stat.ML↗

arXiv2025

Using Experiments to Correct for Selection in Observational Studies

Researchers increasingly have access to two types of data: (i) large observational datasets where treatment (e.g., class size) is not randomized but several primary outcomes (e.g., graduation rates) and secondary outcomes (e.g., test scores) are observed and (ii) experimental data in which treatment is randomized but only secondary outcomes are observed. We develop a new method to estimate treatment effects on primary outcomes in such settings. We use the difference between the secondary outcome and its predicted value based on the experimental treatment effect to measure selection bias in the observational data. Controlling for this estimate of selection bias yields an unbiased estimate of the treatment effect on the primary outcome under a new assumption that we term latent unconfoundedness, which requires that the same confounders affect the primary and secondary outcomes. Latent unconfoundedness weakens the assumptions underlying commonly used surrogate estimators. We apply our estimator to identify the effect of third grade class size on students outcomes. Estimated impacts on test scores using OLS regressions in observational school district data have the opposite sign of estimates from the Tennessee STAR experiment. In contrast, selection-corrected estimates in the observational data replicate the experimental estimates. Our estimator reveals that reducing class sizes by 25% increases high school graduation rates by 0.7 percentage points. Controlling for observables does not change the OLS estimates, demonstrating that experimental selection correction can remove biases that cannot be addressed with standard controls.

stat.ME↗