Rationality of twists of the Siegel modular variety of genus $2$ and level $3$

Abstract

Let $\overlineρ: G_{\mathbf{Q}} \rightarrow {\rm GSp}_4(\mathbf{F}_3)$ be a continuous Galois representation with cyclotomic similitude character -- or, what turns out to be equivalent, the Galois representation associated to the $3$-torsion of a principally polarized abelian surface $A/\mathbf{Q}$. We prove that the moduli space $\mathcal{A}_2(\overlineρ)$ of principally polarized abelian surfaces $B/\mathbf{Q}$ admitting a symplectic isomorphism $B[3] \simeq \overlineρ$ of Galois representations is never rational over $\mathbf{Q}$ when $\overlineρ$ is surjective, even though it is both rational over $\mathbf{C}$ and unirational over $\mathbf{Q}$ via a map of degree $6$.