arXiv subjects

Iain M. Johnstone

Publications and source records attributed to Iain M. Johnstone.

At least 19 recordsLinked to original sources

arXiv2026

Method of Moments Estimation of High-Dimensional Covariance Using a Parametric Model

We propose method-of-moments estimators for the eigenvalues of variance component covariance matrices in multivariate mixed effects models. Assuming a parametric form for the eigenvalue distribution, we focus on the high-dimensional regime where the number of predictors is large and comparable to the number of realizations of each random effect. In this setting, we show that the empirical moments of sum-of-squares matrices (e.g., MANOVA estimators of the covariance matrices) can be closely approximated by deterministic functions of the underlying parameters. This relationship enables the construction of consistent and asymptotically normal estimators via moment matching. Our approach is motivated by applications in quantitative genetics, where estimating genetic covariance components across multiple phenotypic traits is of central interest. We implement our method in a new python package mlmm-mom, and demonstrate how our method adapts to several common experimental designs in this domain.

stat.ME↗

arXiv2024

Spin glass to paramagnetic transition and triple point in Spherical SK model

This paper studies spin glass to paramagnetic transition in the Spherical Sherrington-Kirkpatrick model with ferromagnetic Curie-Weiss interaction with coupling constant $J$ and inverse temperature $β$. The disorder of the system is represented by a general Wigner matrix. We confirm a conjecture of \cite{Baik2016} and \cite{Baik2017}, that the critical window of temperatures for this transition is $β= 1 + bN^{-1/3} \sqrt{\log N}$ with $b\in\mathbb{R}$. The limiting distribution of the scaled free energy is Gaussian for negative $b$ and a weighted linear combination of independent Gaussian and Tracy-Widom components for positive $b$. In the special case where the Wigner matrix is from the Gaussian Orthogonal or Unitary Ensemble, we describe the triple point transition between spin glass, paramagnetic, and ferromagnetic regimes in a critical window for $(β, J)$ around the triple point $(1,1)$: the Tracy-Widom component is replaced by the one parameter family of deformations described by Bloemendal and Virag, \cite{BloVirI}.

math.PR↗

arXiv2022

Comparison of REML methods for the study of phenome-wide genetic variation

It is now well documented that genetic covariance between functionally related traits leads to an uneven distribution of genetic variation across multivariate trait combinations, and possibly a large part of phenotype-space that is inaccessible to evolution. How the size of this nearly-null genetic space translates to the broader phenome level is unknown. High dimensional phenotype data to address these questions are now within reach, however, incorporating these data into genetic analyses remains a challenge. Multi-trait genetic analyses, of more than a handful of traits, are slow and often fail to converge when fit with REML. This makes it challenging to estimate the genetic covariance ($\mathbf{G}$) underlying thousands of traits, let alone study its properties. We present a previously proposed REML algorithm that is feasible for high dimensional genetic studies in the specific setting of a balanced nested half-sib design, common of quantitative genetics. We show that it substantially outperforms other common approaches when the number of traits is large, and we use it to investigate the bias in estimated eigenvalues of $\mathbf{G}$ and the size of the nearly-null genetic subspace. We show that the high-dimensional biases observed are qualitatively similar to those substantiated by asymptotic approximation in a simpler setting of a sample covariance matrix based on i.i.d. vector observation, and that interpreting the estimated size of the nearly-null genetic subspace requires considerable caution in high-dimensional studies of genetic variation. Our results provide the foundation for future research characterizing the asymptotic approximation of estimated genetic eigenvalues, and a statistical null distribution for phenome-wide studies of genetic variation.

stat.AP↗

arXiv2022

An edge CLT for the log determinant of Wigner ensembles

We derive a Central Limit Theorem (CLT) for $\log \left\vert\det \left( W_{N}-E_{N}\right)\right\vert,$ where $W_{N}$ is a Wigner matrix, and $E_{N}$ is local to the edge of the semi-circle law. Precisely, $E_N=2+N^{-2/3}σ_N$ with $σ_N$ being either a constant (possibly negative), or a sequence of positive real numbers, slowly diverging to infinity so that $σ_N \ll \log^{2} N$. We also extend our CLT to cover spiked Wigner matrices. Our interest in the CLT is motivated by its applications to statistical testing in critically spiked models and to the fluctuations of the free energy in the spherical Sherrington-Kirkpatrick model of statistical physics.

math.PR↗

arXiv2020

Tracy-Widom at each edge of real covariance and MANOVA estimators

We study the sample covariance matrix for real-valued data with general population covariance, as well as MANOVA-type covariance estimators in variance components models under null hypotheses of global sphericity. In the limit as matrix dimensions increase proportionally, the asymptotic spectra of such estimators may have multiple disjoint intervals of support, possibly intersecting the negative half line. We show that the distribution of the extremal eigenvalue at each regular edge of the support has a GOE Tracy-Widom limit. Our proof extends a comparison argument of Ji Oon Lee and Kevin Schnelli, replacing a continuous Green function flow by a discrete Lindeberg swapping scheme.

math.PR↗

arXiv2019

Asymptotics of eigenstructure of sample correlation matrices for high-dimensional spiked models

Sample correlation matrices are employed ubiquitously in statistics. However, quite surprisingly, little is known about their asymptotic spectral properties for high-dimensional data, particularly beyond the case of "null models" for which the data is assumed independent. Here, considering the popular class of spiked models, we apply random matrix theory to derive asymptotic first-order and distributional results for both the leading eigenvalues and eigenvectors of sample correlation matrices. These results are obtained under high-dimensional settings for which the number of samples n and variables p approach infinity, with p/n tending to a constant. To first order, the spectral properties of sample correlation matrices are seen to coincide with those of sample covariance matrices; however their asymptotic distributions can differ significantly, with fluctuations of both the sample eigenvalues and eigenvectors often being remarkably smaller than those of their sample covariance counterparts.

math.ST↗

arXiv2018

Notes on asymptotics of sample eigenstructure for spiked covariance models with non-Gaussian data

These expository notes serve as a reference for an accompanying post Morales-Jimenez et al. [2018]. In the spiked covariance model, we develop results on asymptotic normality of sample leading eigenvalues and certain projections of the corresponding sample eigenvectors. The results parallel those of Paul [2007], but are given using the non-Gaussian model of Bai and Yao [2008]. The results are not new, and citations are given, but proofs are collected and organized as a point of departure for Morales-Jimenez et al. [2018].

math.ST↗

arXiv2018

Spiked covariances and principal components analysis in high-dimensional random effects models

We study principal components analyses in multivariate random and mixed effects linear models, assuming a spherical-plus-spikes structure for the covariance matrix of each random effect. We characterize the behavior of outlier sample eigenvalues and eigenvectors of MANOVA variance components estimators in such models under a high-dimensional asymptotic regime. Our results show that an aliasing phenomenon may occur in high dimensions, in which eigenvalues and eigenvectors of the MANOVA estimate for one variance component may be influenced by the other components. We propose an alternative procedure for estimating the true principal eigenvalues and eigenvectors that asymptotically corrects for this aliasing problem.

math.ST↗

arXiv2018

Testing in high-dimensional spiked models

We consider the five classes of multivariate statistical problems identified by James (1964), which together cover much of classical multivariate analysis, plus a simpler limiting case, symmetric matrix denoising. Each of James' problems involves the eigenvalues of $E^{-1}H$ where $H$ and $E$ are proportional to high dimensional Wishart matrices. Under the null hypothesis, both Wisharts are central with identity covariance. Under the alternative, the non-centrality or the covariance parameter of $H$ has a single eigenvalue, a spike, that stands alone. When the spike is smaller than a case-specific phase transition threshold, none of the sample eigenvalues separate from the bulk, making the testing problem challenging. Using a unified strategy for the six cases, we show that the log likelihood ratio processes parameterized by the value of the sub-critical spike converge to Gaussian processes with logarithmic correlation. We then derive asymptotic power envelopes for tests for the presence of a spike.

math.ST↗

arXiv2017

Eigenvalue distributions of variance components estimators in high-dimensional random effects models

We study the spectra of MANOVA estimators for variance component covariance matrices in multivariate random effects models. When the dimensionality of the observations is large and comparable to the number of realizations of each random effect, we show that the empirical spectra of such estimators are well-approximated by deterministic laws. The Stieltjes transforms of these laws are characterized by systems of fixed-point equations, which are numerically solvable by a simple iterative procedure. Our proof uses operator-valued free probability theory, and we establish a general asymptotic freeness result for families of rectangular orthogonally-invariant random matrices, which is of independent interest. Our work is motivated by the estimation of components of covariance between multiple phenotypic traits in quantitative genetics, and we specialize our results to common experimental designs that arise in this application.

math.ST↗

arXiv2017

Edgeworth correction for the largest eigenvalue in a spiked PCA model

We study improved approximations to the distribution of the largest eigenvalue $\hat{\ell}$ of the sample covariance matrix of $n$ zero-mean Gaussian observations in dimension $p+1$. We assume that one population principal component has variance $\ell > 1$ and the remaining `noise' components have common variance $1$. In the high dimensional limit $p/n \to γ> 0$, we begin study of Edgeworth corrections to the limiting Gaussian distribution of $\hat{\ell}$ in the supercritical case $\ell > 1 + \sqrt γ$. The skewness correction involves a quadratic polynomial as in classical settings, but the coefficients reflect the high dimensional structure. The methods involve Edgeworth expansions for sums of independent non-identically distributed variates obtained by conditioning on the sample noise eigenvalues, and limiting bulk properties \textit{and} fluctuations of these noise eigenvalues.

math.ST↗

arXiv2017

On Minimax Optimality of Sparse Bayes Predictive Density Estimates

We study predictive density estimation under Kullback-Leibler loss in $\ell_0$-sparse Gaussian sequence models. We propose proper Bayes predictive density estimates and establish asymptotic minimaxity in sparse models. A surprise is the existence of a phase transition in the future-to-past variance ratio $r$. For $r < r_0 = (\surd 5 - 1)/4$, the natural discrete prior ceases to be asymptotically optimal. Instead, for subcritical $r$, a `bi-grid' prior with a central region of reduced grid spacing recovers asymptotic minimaxity. This phenomenon seems to have no analog in the otherwise parallel theory of point estimation of a multivariate normal mean under quadratic loss. For spike-and-slab priors to have any prospect of minimaxity, we show that the sparse parameter space needs also to be magnitude constrained. Within a substantial range of magnitudes, spike-and-slab priors can attain asymptotic minimaxity.

math.ST↗

arXiv2017

Tail sums of Wishart and GUE eigenvalues beyond the bulk edge

Consider the classical Gaussian unitary ensemble of size $N$ and the real Wishart ensemble $W_N(n,I)$. In the limits as $N \to \infty$ and $N/n \to γ> 0$, the expected number of eigenvalues that exit the upper bulk edge is less than one, 0.031 and 0.170 respectively, the latter number being independent of $γ$. These statements are consequences of quantitative bounds on tail sums of eigenvalues outside the bulk which are established here for applications in high dimensional covariance matrix estimation.

math.ST↗

arXiv2017

Optimal Shrinkage of Eigenvalues in the Spiked Covariance Model

We show that in a common high-dimensional covariance model, the choice of loss function has a profound effect on optimal estimation. In an asymptotic framework based on the Spiked Covariance model and use of orthogonally invariant estimators, we show that optimal estimation of the population covariance matrix boils down to design of an optimal shrinker $η$ that acts elementwise on the sample eigenvalues. Indeed, to each loss function there corresponds a unique admissible eigenvalue shrinker $η^*$ dominating all other shrinkers. The shape of the optimal shrinker is determined by the choice of loss function and, crucially, by inconsistency of both eigenvalues and eigenvectors of the sample covariance matrix. Details of these phenomena and closed form formulas for the optimal eigenvalue shrinkers are worked out for a menagerie of 26 loss functions for covariance estimation found in the literature, including the Stein, Entropy, Divergence, Frechet, Bhattacharya/Matusita, Frobenius Norm, Operator Norm, Nuclear Norm and Condition Number losses.

math.ST↗

arXiv2015

Roy's Largest Root Test Under Rank-One Alternatives

Roy's largest root is a common test statistic in multivariate analysis, statistical signal processing and allied fields. Despite its ubiquity, provision of accurate and tractable approximations to its distribution under the alternative has been a longstanding open problem. Assuming Gaussian observations and a rank one alternative, or concentrated non-centrality, we derive simple yet accurate approximations for the most common low-dimensional settings. These include signal detection in noise, multiple response regression, multivariate analysis of variance and canonical correlation analysis. A small noise perturbation approach, perhaps underused in statistics, leads to simple combinations of standard univariate distributions, such as central and non-central $χ^2$ and $F$. Our results allow approximate power and sample size calculations for Roy's test for rank one effects, which is precisely where it is most powerful.

math.ST↗

arXiv2015

Exact minimax estimation of the predictive density in sparse Gaussian models

We consider estimating the predictive density under Kullback-Leibler loss in an $\ell_0$ sparse Gaussian sequence model. Explicit expressions of the first order minimax risk along with its exact constant, asymptotically least favorable priors and optimal predictive density estimates are derived. Compared to the sparse recovery results involving point estimation of the normal mean, new decision theoretic phenomena are seen. Suboptimal performance of the class of plug-in density estimates reflects the predictive nature of the problem and optimal strategies need diversification of the future risk. We find that minimax optimal strategies lie outside the Gaussian family but can be constructed with threshold predictive density estimates. Novel minimax techniques involving simultaneous calibration of the sparsity adjustment and the risk diversification mechanisms are used to design optimal predictive density estimates.

math.ST↗

arXiv2014

Local Asymptotic Normality of the spectrum of high-dimensional spiked F-ratios

We consider two types of spiked multivariate F distributions: a scaled distribution with the scale matrix equal to a rank-one perturbation of the identity, and a distribution with trivial scale, but rank-one non-centrality. The norm of the rank-one matrix (spike) parameterizes the joint distribution of the eigenvalues of the corresponding F matrix. We show that, for a spike located above a phase transition threshold, the asymptotic behavior of the log ratio of the joint density of the eigenvalues of the F matrix to their joint density under a local deviation from this value depends only on the largest eigenvalue $λ_{1}$. Furthermore, $λ_{1}$ is asymptotically normal, and the statistical experiment of observing all the eigenvalues of the F matrix converges in the Le Cam sense to a Gaussian shift experiment that depends on the asymptotic mean and variance of $λ_{1}$. In particular, the best statistical inference about a sufficiently large spike in the local asymptotic regime is based on the largest eigenvalue only. As a by-product of our analysis, we establish joint asymptotic normality of a few of the largest eigenvalues of the multi-spiked F matrix when the corresponding spikes are above the phase transition threshold.

math.ST↗

arXiv2014

Adaptation in a class of linear inverse problems

We consider the linear inverse problem of estimating an unknown signal $f$ from noisy measurements on $Kf$ where the linear operator $K$ admits a wavelet-vaguelette decomposition (WVD). We formulate the problem in the Gaussian sequence model and propose estimation based on complexity penalized regression on a level-by-level basis. We adopt squared error loss and show that the estimator achieves exact rate-adaptive optimality as $f$ varies over a wide range of Besov function classes.

math.ST↗