Estimating treatment-effect heterogeneity across sites, in multi-site randomized experiments with few units per site

Abstract

In multi-site randomized trials with many sites and few randomization units per site, an Empirical-Bayes estimator can be used to estimate the variance of the treatment effect across sites. When this estimator indicates that treatment effects do vary, we propose estimators of the coefficients from regressions of site-level effects on site-level characteristics that are unobserved but can be unbiasedly estimated, such as sites' average outcome without treatment, or site-specific treatment effects on mediator variables. In experiments with imperfect compliance, we show that the sign of the correlation between local average treatment effects (LATEs) and site-level characteristics is identified, and we propose a partly testable assumption under which the variance of LATEs is identified. We use our results to revisit Behaghel et al (2014), who study the effect of counseling programs on job seekers' job-finding rate, in 200 job placement agencies in France. We find considerable treatment-effect heterogeneity, both for intention to treat and LATE effects, and the treatment effect is negatively correlated with sites' job-finding rate without treatment.