The presence of a massive black hole has been invoked to match the observed rotation velocities and velocity dispersions at the centres of M31, M32, NGC 3115 and NGC 4594, Here we determine stellar line-of-sight velocity profiles of these galaxies, from high spatial resolution, high S/N spectra centred on the G-band, the Mg b triplet and the Ca II IR triplet. For all spectra we determine rotation velocities, V, velocity dispersions, |$\sigma$|, and deviations of the velocity profiles from a Gaussian shape, as quantified by the Gauss-Hermite moments |$h_3, \ldots, h_6$|. Significant deviations of the velocity profiles from a Gaussian are present in all objects. The strongest deviations are asymmetries quantified by |$h_3$| and are found primarily along the major axis. The amplitude of |$h_3$| is typically |$|h_3| \lesssim 0.15$| and is correlated with |$V/\sigma$|. Its sign always indicates an excess of stars at low rotation velocities, and results in the mean stellar streaming velocity being lower than the mean of the best-fitting Gaussian, the quantity conventionally used to characterize rotation. The difference is |$\sim 10$| per cent for M31, M32 and NGC 3115, and |$\sim 20$| per cent for NGC 4594. It is thus important that models take the observed velocity profile shapes into account. We demonstrate that the observed deviations from Gaussian velocity profiles are for the most part not the result of projection and seeing convolution. We calculate the velocity profiles predicted by Tonry's model for M32, taking seeing and projection into account, but assuming the local (unprojected) velocity distributions to be Gaussian. The predicted velocity profiles are nearly Gaussian, in conflict with our data. Self-consistent dynamical models must thus be constructed to fit the observed velocity profiles. Such models should constrain the central structure quite strongly, and will hence provide improved constraints on the mass and presence of a supermassive central object.