A Formal Graphical Inference Framework for Combining Effect Size with Statistical Significance: Application to Multivariate and Functional Linear Models

Abstract

To address the critical need for statistical methods that evaluate practical magnitude alongside statistical significance, we introduce a formal graphical inference framework that intrinsically combines effect size with uncertainty. Built upon resampling and global envelopes, this approach provides universality and direct visual interpretability. While existing envelope tests provide only weak family-wise error rate (FWER) or false discovery rate control, we advance the methodology by introducing novel step-down global envelopes. We theoretically prove that several of these envelopes achieve exact strong FWER control under exchangeability, guaranteeing rigorous inference for each individual local hypothesis. Beyond yielding adjusted $p$-values, the framework provides adjusted subset $p$-values for blocks of hypotheses, such as the functional effect of a covariate within a specific category. By explicitly visualizing the size and direction of the effect relative to its variability under the null hypothesis, our approach overcomes the dichotomous nature of traditional testing and provides profound informational value. The proposed framework is applied to multivariate and functional linear models.