Whittaker supports for representations of reductive groups

Abstract

Let $F$ be either $\mathbb{R}$ or a finite extension of $\mathbb{Q}_p$, and let $G$ be a finite central extension of the group of $F$-points of a reductive group defined over $F$. Also let $\pi$ be a smooth representation of $G$ (Frechet of moderate growth if $F=\mathbb{R}$). For each nilpotent orbit $\mathcal{O}$ we consider a certain Whittaker quotient $\pi_{\mathcal{O}}$ of $\pi$. We define the Whittaker support WS$(\pi)$ to be the set of maximal $\mathcal{O}$ among those for which $\pi_{\mathcal{O}}\neq 0$. In this paper we prove that all $\mathcal{O}\in\mathrm{WS}(\pi)$ are quasi-admissible nilpotent orbits, generalizing some of the results in [Moe96,JLS16]. If $F$ is $p$-adic and $\pi$ is quasi-cuspidal then we show that all $\mathcal{O}\in\mathrm{WS}(\pi)$ are $F$-distinguished, i.e. do not intersect the Lie algebra of any proper Levi subgroup of $G$ defined over $F$. We also give an adaptation of our argument to automorphic representations, generalizing some results from [GRS03,Shen16,JLS16,Cai] and confirming some conjectures from [Ginz06]. Our methods are a synergy of the methods of the above-mentioned papers, and of our preceding paper [GGS17].