Orthonormal Bases in the Orbit of Square-Integrable Representations of Nilpotent Lie Groups

Abstract

Let $G$ be a connected, simply connected nilpotent group and $π$ be a square-integrable irreducible unitary representation modulo its center $Z(G)$ on $L^2(\mathbf{R}^d)$. We prove that under reasonably weak conditions on $G$ and $π$ there exist a discrete subset $Γ$ of $G/Z(G)$ and some (relatively) compact set $F \subseteq \mathbf{R}^d$ such that $$\bigl \{ |F|^{-1/2} \hspace{2pt} π(γ) 1_F \mid γ\in Γ\bigr\}$$ forms an orthonormal basis of $L^2(\mathbf{R}^d)$. This construction generalizes the well-known example of Gabor orthonormal bases in time-frequency analysis. The main theorem covers graded Lie groups with one-dimensional center. In the presence of a rational structure, the set $Γ$ can be chosen to be a uniform subgroup of $G/Z$.