Abstract We address the effect of particle scattering on the energy spectra of solar energetic electron events using (i) an observational and (ii) a modeling approach. (i) We statistically study observations of the STEREO spacecraft, using directional electron measurements made with the Solar Electron and Proton Telescope in the range of 45–425 keV. We compare the energy spectra of the anti-Sunward propagating beam with that of the backward-scattered population and find that, on average, the backward-scattered population shows a harder spectrum with the effect being stronger at higher energies. (ii) We use a numerical solar energetic particle (SEP) transport model to simulate the effect of particle scattering (both in terms of pitch angle and perpendicular to the mean field) on the spectrum. We find that pitch-angle scattering can lead to spectral changes at higher energies (E > 100 keV) and further away from the Sun (r > 1 au), which are also often observed. At lower energies, and closer to the Sun, the effect of pitch-angle scattering is much more reduced, so that the simulated energy spectra still resemble the injected power-law functions. When examining pitch-angle-dependent spectra, we find, in agreement with the observational results, that the spectra of the backward-propagating electrons are harder than that of the forward (from the Sun) propagating population. We conclude that Solar Orbiter and Parker Solar Probe will be able to observe the unmodulated omnidirectional SEP electron spectrum close to the Sun at higher energies, giving a direct indication of the accelerated spectrum.