Context. The extreme contrast ratios between stars and their planets at optical wavelengths make it challenging to isolate the light reflected by exoplanet atmospheres. Yet, these reflective properties reveal key processes occurring in the atmospheres, and they also span wavelengths that include the potential O2 biosignature. High resolution cross-correlation spectroscopy (HRCCS) offers a robust avenue for developing techniques to extract exoplanet reflection spectra. Aims. We aimed to extract the optical reflected light spectrum of the non-transiting hot Jupiter 51 Pegasi b by adapting techniques designed to remove tellurics in infrared HRCCS to instead remove optical stellar lines. Importantly, we investigated the as of yet neglected impact of the broadening of the reflected host star spectrum due to the difference between the stellar rotation and the planet’s orbital velocity. Methods. We used 484, R = 115 000 optical spectra of 51 Pegasi b from HARPS-N and HARPS, which we aligned to the exact stellar rest frame, in order to effectively remove the contaminating host star. However, some stellar residuals remained, likely due to stellar activity. We cross-correlated with an appropriately broadened synthetic stellar model to search for the planet’s Doppler-shifting spectrum. Results. We detect no significant reflected light from 51 Pegasi b, and report a signal-to-noise (S∕N) = 3 upper limit on the contrast ratio of 76.0 ppm (7.60 × 10−5) when including broadening, and 24.0 ppm (2.40 × 10−5) without. These upper limits rule out radius and albedo combinations of previously claimed detections. Conclusions. Broadening can significantly impact the ability of HRCCS to extract reflected light spectra and it must be considered when determining the contrast ratio, radius, and albedo of the planet. Asynchronous systems (Prot,⋆≠Porb) are most affected, including most hot Jupiters as well as Earth-size planets in the traditional habitable zones of some M-dwarfs.