Direct integral decompositions and multiplicities for induced representations of nilpotent Lie groups

Abstract

Let K K be a Lie subgroup of the connected, simply connected nilpotent Lie group G G , and let k \mathfrak {k} , g \mathfrak {g} be the corresponding Lie algebras. Suppose that σ \sigma is an irreducible unitary representation of K K . We give an explicit direct integral decomposition of Ind k → G σ {\operatorname {Ind} _{k \to G}}\sigma into irreducibles. The description uses the Kirillov orbit picture, which gives a bijection between G ∧ G^\wedge and the coadjoint orbits in g ∗ {\mathfrak {g}^{\ast }} (and similarly for K ∧ , k ∗ K^\wedge ,\,{\mathfrak {k}^{\ast }} ). Let P : k ∗ → g ∗ P:{\mathfrak {k}^{\ast }} \to {\mathfrak {g}^{\ast }} be the canonical projection, let O σ ⊂ k ∗ {\mathcal {O}_\sigma } \subset {\mathfrak {k}^{\ast }} be the orbit corresponding to σ \sigma , and, for π ∈ G ∧ \pi \in G^\wedge , let O π ⊂ g ∗ {\mathcal {O}_\pi } \subset {\mathfrak {g}^{\ast }} be the corresponding orbit. The main result of the paper says essentially that π ∈ G ∧ \pi \in G^\wedge appears in the direct integral iff P − 1 ( O σ ) {P^{ - 1}}({\mathcal {O}_\sigma }) meets O π {\mathcal {O}_\pi } ; the multiplicity of π \pi is the number of Ad ∗ ( K ) {\operatorname {Ad} ^{\ast }}(K) -orbits in O π ∩ P − 1 ( O σ ) {\mathcal {O}_\pi } \cap {P^{ - 1}}({\mathcal {O}_\sigma }) . There is also a natural description of the measure class in the integral.