Asymptotic trace formula for the Hecke operators

Abstract

Given integers m, n and k, we give an explicit formula with an optimal error term (with square root cancelation) for the Petersson trace formula involving the mth and nth Fourier coefficients of an orthonormal basis of $$S_k\left( N\right) ^*$$ (the weight k newforms with fixed square-free level N) provided that $$|4 \pi \sqrt{mn}- k|=o\left( k^{\frac{1}{3}}\right) $$ . Moreover, we establish an explicit formula with a power saving error term for the trace of the Hecke operator $$\mathcal {T}_n^*$$ on $$S_k\left( N\right) ^*$$ averaged over k in a short interval. By bounding the second moment of the trace of $$\mathcal {T}_{n}$$ over a larger interval, we show that the trace of $$\mathcal {T}_n$$ is unusually large in the range $$|4 \pi \sqrt{n}- k| = o\left( n^{\frac{1}{6}}\right) $$ . As an application, for any fixed prime p coprime to N, we show that there exists a sequence $$\{k_n\}$$ of weights such that the error term of Weyl’s law for $$\mathcal {T}_p$$ is unusually large and violates the prediction of arithmetic quantum chaos. In particular, this generalizes the result of Gamburd et al. (J Eur Math Soc 1(1):51–85, 1999) [Theorem 1.4] with an improved exponent.