Number of relevant directions in Principal Component Analysis and Wishart random matrices

Abstract

We compute analytically, for large $N$, the probability $\mathcal{P}(N_+,N)$ that a $N\times N$ Wishart random matrix has $N_+$ eigenvalues exceeding a threshold $Nζ$, including its large deviation tails. This probability plays a benchmark role when performing the Principal Component Analysis of a large empirical dataset. We find that $\mathcal{P}(N_+,N)\approx\exp(-βN^2 ψ_ζ(N_+/N))$, where $β$ is the Dyson index of the ensemble and $ψ_ζ(κ)$ is a rate function that we compute explicitly in the full range $0\leq κ\leq 1$ and for any $ζ$. The rate function $ψ_ζ(κ)$ displays a quadratic behavior modulated by a logarithmic singularity close to its minimum $κ^\star(ζ)$. This is shown to be a consequence of a phase transition in an associated Coulomb gas problem. The variance $Δ(N)$ of the number of relevant components is also shown to grow universally (independent of $ζ)$ as $Δ(N)\sim (βπ^2)^{-1}\ln N$ for large $N$.