Asymptotic false discovery control of the Benjamini-Hochberg procedure for pairwise comparisons

Abstract

In a one-way analysis-of-variance (ANOVA) model, the number of all pairwise comparisons can be large even when there are only a moderate number of groups. Motivated by this, we consider a regime with a growing number of groups, and prove that for testing pairwise comparisons the BH procedure can offer asymptotic control on false discoveries, despite that the t-statistics involved do not exhibit the well-known positive dependence structure called the PRDS to guarantee exact false discovery rate (FDR) control. Sharing Tukey's viewpoint that the difference in the means of any two groups cannot be exactly zero, our main result is stated in terms of the control on the directional false discovery rate and directional false discovery proportion. A key technical contribution is that we have shown the dependence among the t-statistics to be weak enough to induce a convergence result typically needed for establishing asymptotic FDR control. Our analysis does not adhere to stylized assumptions such as normality, variance homogeneity and a balanced design, and thus provides a theoretical grounding for applications in more general situations.