A transfer-operator-based relation between Laplace eigenfunctions and zeros of Selberg zeta functions

Abstract

Over the last few years Pohl (partly jointly with coauthors) has developed dual ‘slow/fast’ transfer operator approaches to automorphic functions, resonances, and Selberg zeta functions for a certain class of hyperbolic surfaces$\unicode[STIX]{x1D6E4}\backslash \mathbb{H}$with cusps and all finite-dimensional unitary representations$\unicode[STIX]{x1D712}$of$\unicode[STIX]{x1D6E4}$. The eigenfunctions with eigenvalue 1 of the fast transfer operators determine the zeros of the Selberg zeta function for$(\unicode[STIX]{x1D6E4},\unicode[STIX]{x1D712})$. Further, if$\unicode[STIX]{x1D6E4}$is cofinite and$\unicode[STIX]{x1D712}$is the trivial one-dimensional representation then highly regular eigenfunctions with eigenvalue 1 of the slow transfer operators characterize Maass cusp forms for$\unicode[STIX]{x1D6E4}$. Conjecturally, this characterization extends to more general automorphic functions as well as to residues at resonances. In this article we study, without relying on Selberg theory, the relation between the eigenspaces of these two types of transfer operators for any Hecke triangle surface$\unicode[STIX]{x1D6E4}\backslash \mathbb{H}$of finite or infinite area and any finite-dimensional unitary representation$\unicode[STIX]{x1D712}$of the Hecke triangle group $\unicode[STIX]{x1D6E4}$. In particular, we provide explicit isomorphisms between relevant subspaces. This solves a conjecture by Möller and Pohl, characterizes some of the zeros of the Selberg zeta functions independently of the Selberg trace formula, and supports the previously mentioned conjectures.