Almost all primes satisfy the Atkin-Serre conjecture and are not extremal

Abstract

Let $f(z)=\sum_{n=1}^{\infty} a_f(n)e^{2\pi i n z}$ be a non-CM holomorphic cupsidal newform of trivial nebentypus and even integral level $k\geq 2$. Deligne's proof of the Weil conjectures shows that $|a_f(p)|\leq 2p^{\frac{k-1}{2}}$ for all primes $p$. We prove for 100% of primes $p$ that $2p^{\frac{k-1}{2}}\frac{\log\log p}{\sqrt{\log p}}<|a_f(p)|<\lfloor 2p^{\frac{k-1}{2}}\rfloor$. Our proof gives an effective upper bound for the size of the exceptional set. The lower bound shows that the Atkin-Serre conjecture is satisfied for 100% of primes, and the upper bound shows that $|a_f(p)|$ is as large as possible (i.e., $p$ is extremal for $f$) for 0% of primes. Our proofs use the effective form of the Sato-Tate conjecture proved by the second author, which relies on the recent proof of the automorphy of the symmetric powers of $f$ due to Newton and Thorne.