A bootstrap approach for testing invariance under parameterized group actions: orthogonal reflections and axial symmetry

Abstract

Testing whether a multivariate distribution is invariant under an orthogonal transformation is a classical problem when the transformation is fixed in advance. We address a harder situation: the transformation is unknown and must be inferred from the data. We frame this as invariance under a group action whose representation is indexed by an unknown parameter. We work in $\mathbb{R}^d$ under a simple-spectrum assumption on the covariance matrix $\Sigma$. Under this assumption, any orthogonal transformation that preserves the distribution must commute with $\Sigma$, and therefore must be a {reflection through a subspace spanned by a subset of the principal directions. This reduces the search over the orthogonal group to a finite family of candidate reflections, one for each subset of principal axes.} For each candidate, we construct a Kolmogorov--Smirnov-type statistic based on projected data and sample splitting. We derive its asymptotic distribution in a triangular-array framework and establish bootstrap validity under suitable regularity conditions. Axial symmetry about an unspecified direction (that is, invariance under reflection across an unknown one-dimensional subspace) and hyperplane (Householder) symmetry about an unspecified normal direction are the two leading particular cases: we treat them in detail, the former driving the simulation study and the latter the real-data application, to show how a concrete problem involving an unknown group representation can be successfully addressed.