An Axiomatic Approach to Comparing Sensitivity Parameters

Abstract

Many methods are available for assessing the importance of omitted variables in linear regression. These methods typically make different, non-falsifiable assumptions. Hence the data alone cannot tell us which method is most appropriate. Since it is unreasonable to expect results to be robust against all possible robustness checks, researchers often use methods deemed ``interpretable,'' a subjective criterion with no formal definition. In contrast, we develop the first formal, axiomatic framework for comparing and selecting among these methods. Our framework is analogous to the standard approach for comparing estimators based on their sampling distributions. We propose that sensitivity parameters be selected based on their covariate sampling distributions, a design distribution of parameter values induced by an assumption on how covariates are assigned to be observed or unobserved. Using this idea, we define new concepts of parameter consistency and monotonicity, and argue that a reasonable sensitivity parameter should satisfy both properties. We prove that the literature's most popular approach is inconsistent and non-monotonic, while several alternatives satisfy both.