Almost all primes satisfy the Atkin–Serre conjecture and are not extremal

Abstract

Let \(f(z)=\sum _{n=1}^{\infty } a_f(n)e^{2\pi i n z}\) be a non-CM holomorphic cuspidal newform of trivial nebentypus and even integral weight \(k\ge 2\). Deligne’s proof of the Weil conjectures shows that \(|a_f(p)|\le 2p^{\frac{k-1}{2}}\) for all primes p. We prove for 100% of primes p that \( 2p^{\frac{k-1}{2}}{\log \log p}/{\sqrt{\log p}}<|a_f(p)|<\lfloor 2p^{\frac{k-1}{2}}\rfloor .\) Our proof gives an effective upper bound for the size of the exceptional set. The lower bound shows that the Atkin–Serre conjecture is satisfied for 100% of primes, and the upper bound shows that \(|a_f(p)|\) is as large as possible (i.e., p is extremal for f) for 0% of primes. Our proofs use the effective form of the Sato–Tate conjecture proved by the second author, which relies on the recent proof of the automorphy of the symmetric powers of f due to Newton and Thorne.