An alternative method of adjusting for multiple comparison in medical research

Abstract

Background Most methods of adjusting for multiplicity focus primarily on controlling type I errors and rarely consider type II errors. We propose a new method that considers controlling for false-positive findings while ensuring sufficient statistical power. Methods We proposed a new method for multiple corrections called (Beta-exponential Adjustment, BEA) that considered the statistical power to control for type I errors while also considering the probability of type II errors. We conducted simulation studies to evaluate the performance characteristic of multiple testing correction procedures. We calculated sensitivity, specificity, and power separately for different sample sizes and number of biomarkers and compared them with the Bonferroni, Holm, and Benjamini-Hochberg (BH) correction methods. Results The results demonstrated that our proposed BEA correction method exhibited the highest sensitivity at different sample sizes and biomarkers (e.g., sensitivity: BEA 0.8 versus BH 0.62 at sample size at 1000, tested biomarkers at 1000 and positive rate at 30%). With different sample sizes and number of biomarkers, the BEA correction method demonstrated comparable specificity compared with traditional methods. Moreover, we observed that the BEA-corrected had the highest statistical power than other methods, when the outcome was relatively rare. Conclusion We proposed the BEA multiple correction method to adjust for multiple comparisons while considering statistical power. The BEA method demonstrated a higher sensitivity, comparable specificity, and higher statistical power, compared with traditional correction methods in different conditions. The BEA correction method can be an alternative of traditional methods of adjusting for multiplicity, especially in studies with small sample size, rare outcomes, or substantial number of biomarkers.