On Asymptotic Behaviors of Stepwise Multiple Testing Procedures

Abstract

Stepwise multiple testing procedures have attracted several statisticians for decades and are also quite popular with statistics users because of their technical simplicity. The Bonferroni procedure has been one of the earliest and most prominent testing rules for controlling the familywise error rate (FWER). A recent article established that the FWER for the Bonferroni method asymptotically (i.e., when the number of hypotheses becomes arbitrarily large) approaches zero under any positively equicorrelated multivariate normal framework. However, similar results for the limiting behaviors of FWER of general stepwise procedures are nonexistent. The present work addresses this gap in a unified manner by studying the limiting behaviors of the FWER of several stepwise testing rules for correlated normal setups. Specifically, we show that the limiting FWER approaches zero for any step-down rule (e.g., Holm's method) provided the infimum of the correlations is strictly positive. We also establish similar limiting zero results on FWER of other popular multiple testing rules, e.g., Hochberg's and Hommel's procedures. We then extend these results to any configuration of true and false null hypotheses. It turns out that, within our chosen asymptotic framework, the Benjamini-Hochberg method can hold the FWER at a strictly positive level asymptotically under the equicorrelated normality. We finally discuss the limiting powers of various procedures.