Numerical Analysis of Test Optimality

Abstract

In nonstandard testing environments, researchers often derive ad hoc tests with correct (asymptotic) size, but their optimality properties are typically unknown a priori and difficult to assess. This paper develops a numerical framework for determining whether an ad hoc test is effectively optimal - approximately maximizing a weighted average power criterion for some weights over the alternative and attaining a power envelope generated by a single weighted average power-maximizing test. Our approach uses nested optimization algorithms to approximate the weight function that makes an ad hoc test's weighted average power as close as possible to that of a true weighted average power-maximizing test, and we show the surprising result that the rejection probabilities corresponding to the latter form an approximate power envelope for the former. We provide convergence guarantees, discuss practical implementation and apply the method to the weak instrument-robust conditional likelihood ratio test and a recently-proposed test for when a nuisance parameter may be on or near its boundary.