Some pathologies in the Mackey analysis for a certain nonseparable group

Abstract

Suppose G is a separable locally compact group and N is a closed normal subgroup. If the dual N̂ is smooth and the orbit space N ̂ G is smooth for the natural action of G on N ̂ (L x(n) = L(xnx −1)) , the method of G. W. Mackey ( Acta Math. 99 (1958), 265–311) gives a fairly simple procedure for constructing the dual Ĝ. In this paper we examine an example which shows that the nonseparable case is much more complicated. In the example, N is abelian, N ̂ G is finite and even when the stabilizer is N there are many irreducible representations of G associated with the same orbit.