Quantum frequency conversion (QFC) will be an indispensable ingredient in future quantum technologies. For example, large-scale fibre-based quantum networks will require QFC to interconnect heterogeneous building blocks like emitters, channels, memories, and detectors. The performance of existing QFC devices – typically realized in periodically poled nonlinear crystals – is often severely limited by parasitic noise that arises when the pump wavelength lies between the inter-converted wavelengths. Here we comprehensively investigate the noise spectrum of a QFC device pumped by a CW 1064 nm laser. The converter was realized as a bulk periodically poled potassium titanyl phosphate (ppKTP) crystal quasi-phase-matched for conversion between 637 nm and 1587 nm, which was also polished and coated to resonantly enhance the pump field by a factor of 50. While Raman scattering dominates the noise background from 1140 nm to 1330 nm, at larger energy shifts (beyond 60 THz), parasitic spontaneous parametric down-conversion (SPDC) is the strongest noise source. Further, the noise spectrum was contaminated by a regular succession of narrow-band peaks, which we attribute to a heretofore unidentified higher-order counter-propagating SPDC processes – with quasi-phase-matching orders up to 44 evident in our measurements. This work provides a comprehensive overview of the limiting noise sources in QFC devices that use quasi-phase-matched crystals and will prove an invaluable resource in guiding their future development.