Many precision laser applications require active frequency stabilization. However, such stabilization loops operate by pushing noise to frequencies outside their bandwidth, leading to large ``servo bumps'' that can have deleterious effects for certain applications. The prevailing approach to filtering this noise is to pass the laser through a high-finesse optical cavity, which places constraints on the system design. Here, we propose and demonstrate a different approach where a frequency error signal is derived from a beat note between the laser and the light that passes through the reference cavity. The phase noise derived from this beat note is fed forward to an electro-optic modulator after the laser, carefully accounting for relative delay, for real-time frequency correction. With a hertz-line-width laser, we show $\ensuremath{\gtrsim}20\phantom{\rule{0.2em}{0ex}}\mathrm{dB}$ noise suppression at the peak of the servo bump (approximately $250\phantom{\rule{0.2em}{0ex}}\mathrm{kHz}$) and a noise-suppression bandwidth of approximately $5\phantom{\rule{0.2em}{0ex}}\mathrm{MHz}$---well beyond the servo bump. By simulating the Rabi dynamics of a two-level atom with our measured data, we demonstrate substantial improvements to the pulse fidelity over a wide range of Rabi frequencies. Our approach offers a simple and versatile method for obtaining a clean spectrum of a narrow-line-width laser, as required in many emerging applications of cold atoms, and is readily compatible with commercial systems that may even include wavelength conversion.