Gelfand-Tsetlin polytopes and random contractions away from the limiting shapes

Abstract

In this paper, we consider a sequence of selfadjoint matrices $A_n$ having a limiting spectral distribution as $n\to \infty$, and we consider a sequence of full flags $\{0\le p_1^n\le\ldots\le p_i^n\le\ldots\le 1_n\}$ chosen at random according to the uniform measure on full flag manifolds. We are interested in the behaviour of the extremal eigenvalues of $p_i^nA_np_i^n$. This problem is known to be equivalent to the study of uniform probability measures on Gelfand-Tsetlin polytopes. Our main results consist in explicit uniform estimates for extremal eigenvalues, and the fact that an outlier behavior has an exponentially small probability. This problem is of intrinsic interest in random matrix theory, but it has also a strong motivation and some applications in quantum information, which we discuss. The proofs rely on a reinterpretation of the problem with the help of determinantal point processes and the techniques are based on steepest descent analysis.