Quotient branching law for $p$-adic $(\mathrm{GL}_{n+1}, \mathrm{GL}_n)$ I: generalized Gan-Gross-Prasad relevant pairs

Abstract

Let $G_n=\mathrm{GL}_n(F)$ be the general linear group over a non-Archimedean local field $F$. We formulate and prove a necessary and sufficient condition on determining when \[ \mathrm{Hom}_{G_n}(\pi, \pi') \neq 0 \] for irreducible smooth representations $\pi$ and $\pi'$ of $G_{n+1}$ and $G_n$ respectively. This resolves the problem of the quotient branching law. We also prove that any simple quotient of a Bernstein-Zelevinsky derivative of an irreducible representation can be constructed by a sequence of derivatives of essentially square-integrable representations. This result transferred to affine Hecke algebras of type A gives a generalization of the classical Pieri's rule of symmetric groups. One key new ingredient is a characterization of the layer in the Bernstein-Zelevinsky filtration that contributes to the branching law, obtained by the multiplicity one theorem for standard representations, which also gives a refinement of the branching law. Another key new ingredient is constructions of some branching laws and simple quotients of Bernstein-Zelevinsky derivatives by taking certain highest derivatives.