The GW approximation is nowadays being used to obtain accurate quasiparticle energies of atoms and molecules. In practice, the GW approximation is generally evaluated perturbatively, based on a prior self-consistent calculation within a simpler approximation. The final result thus depends on the choice of the self-consistent mean-field chosen as a starting point. Using a recently developed GW code based on Gaussian basis functions, we benchmark a wide range of starting points for perturbative GW, including Hartree-Fock, LDA, PBE, PBE0, B3LYP, HSE06, BH&HLYP, CAM-B3LYP, and tuned CAM-B3LYP. In the evaluation of the ionization energy, the hybrid functionals are clearly superior results starting points when compared to Hartree-Fock, to LDA, or to the semilocal approximations. Furthermore, among the hybrid functionals, the ones with the highest proportion of exact-exchange usually perform best. Finally, the reliability of the frozen-core approximation, that allows for a considerable speed-up of the calculations, is demonstrated.