A universal expectation bound on empirical projections of deformed random matrices

Abstract

Let $C$ be a real-valued $M\times M$ matrix with singular values $λ_1\ge...\geλ_M$ and $E$ a random matrix of centered i.i.d. entries with finite fourth moment. In this paper we give a universal upper bound on the expectation of $||\hatπ_rX||_{S_2}^2-||π_rX||^2_{S_2}$, where $X:=C+E$ and $\hatπ_r$ (resp. $π_r$) is a rank-$r$ projection maximizing the Hilbert-Schmidt norm $||\tildeπ_rX||_{S_2}$ (resp. $||\tildeπ_rC||_{S_2}$) over the set $§_{M,r}$ of all orthogonal rank-$r$ projections. This result is a generalization of a theorem for Gaussian matrices due to Rohde (2012). Our approach differs substantially from the techniques of the mentioned article. We analyze $||\hatπ_rX||_{S_2}^2-||π_rX||^2_{S_2}$ from a rather deterministic point of view by an upper bound on $||\hatπ_rX||_{S_2}^2-||π_rX||^2_{S_2}$, whose randomness is totally determined by the largest singular value of $E$.