The space \({\cal P}_n\) of bivariate generalised Hermite polynomials of degree n is invariant under rotations. We exploit this symmetry to construct an orthonormal basis for \({\cal P}_n\) which consists of the rotations of a single polynomial through the angles \({\ell\pi\over n+1}\), l=0,...n. Thus we obtain an orthogonal expansion which retains as much of the symmetry of \({\cal P}_n\) as is possible. Indeed we show that a continuous version of this orthogonal expansion exists.