Interpretational errors with instrumental variables

Abstract

Instrumental variables (IV) are often used to identify causal effects in observational settings and experiments subject to non-compliance. Under canonical assumptions, IVs allow us to identify a so-called local average treatment effect (LATE). The use of IVs is often accompanied by a pragmatic decision to abandon the identification of the causal parameter that corresponds to the original research question and target the LATE instead. This pragmatic decision presents a potential source of error: an investigator mistakenly interprets findings as if they had made inference on their original causal parameter of interest. We conducted a systematic review and meta-analysis of patterns of pragmatism and interpretational errors in the applied IV literature published in leading journals of economics, political science, epidemiology, and clinical medicine (n = 309 unique studies). We found that a large fraction of studies targeted the LATE, although specific interest in this parameter was rare. Of these studies, 61% contained claims that mistakenly suggested that another parameter was targeted -- one whose value likely differs, and could even have the opposite sign, from the parameter actually estimated. Our findings suggest that the validity of conclusions drawn from IV applications is often compromised by interpretational errors.