Generalized Whittaker Quotients of Schwartz Functions onG-Spaces

Abstract

AbstractLet $G$ be a reductive group over a local field $F$ of characteristic zero. Let $X$ be a $G$-space. In this paper we study the existence of generalized Whittaker quotients for the space of Schwartz functions on $X$, considered as a representation of $G$. We show that the set of nilpotent elements of the dual space to the Lie algebra such that the corresponding generalized Whittaker quotient does not vanish contains the nilpotent part of the image of the moment map and lies in the closure of this image. This generalizes recent results of Prasad and Sakellaridis. Applying our theorems to symmetric pairs $(G,H)$ we show that there exists an infinite-dimensional $H$-distinguished representation of $G$ if and only if the real reductive group corresponding to the pair $(G,H)$ is non-compact. For quasi-split $G$ we also extend to the Archimedean case the theorem of Prasad stating that there exists a generic $H$-distinguished representation of $G$ if and only if the real reductive group corresponding to the pair $(G,H)$ is quasi-split. In the non-Archimedean case our result also gives rather sharp bounds on the wave-front sets of distinguished representations. Finally, we deduce a corollary on vanishing of period integrals of automorphic forms with certain Whittaker supports. This corollary, when combined with the restrictions on the Whittaker support of cuspidal automorphic representations proven by Gomez–Gourevitch–Sahi, implies many of the vanishing results on periods of automorphic forms proved by Ash–Ginzburg–Rallis.