A Bracketing Relationship for Long-Term Policy Evaluation with Combined Experimental and Observational Data

Abstract

Combining short-term experimental data with observational data enables credible long-term policy evaluation. The literature offers two key but non-nested assumptions, namely the latent unconfoundedness (LU; Athey et al., 2020) and equi-confounding bias (ECB; Ghassami et al., 2022) conditions, to correct observational selection. Committing to the wrong assumption leads to biased estimation. To mitigate such risks, we provide a novel bracketing relationship (cf. Angrist and Pischke, 2009) repurposed for the setting with data combination: the LU-based estimand and the ECB-based estimand serve as the lower and upper bounds, respectively, with the true causal effect lying in between if either assumption holds. For researchers further seeking point estimates, our Lalonde-style exercise suggests the conservatively more robust LU-based lower bounds align closely with the hold-out experimental estimates for educational policy evaluation. We investigate the economic substantives of these findings through the lens of a nonparametric class of selection mechanisms and sensitivity analysis. We uncover as key the sub-martingale property and sufficient-statistics role (Chetty, 2009) of the potential outcomes of student test scores (Chetty et al., 2011, 2014).