Accurate prediction of the ionization potential and electron affinity energies of small molecules are important for many applications. Density functional theory (DFT) is computationally inexpensive, but can be very inaccurate for frontier orbital energies or ionization energies. The GW method is sufficiently accurate for many relevant applications, but much more expensive than DFT. Here we study how we can learn to predict orbital energies with GW accuracy using machine learning (ML) on molecular graphs and fingerprints using an interpretable delta-learning approach. ML models presented here can be used to predict quasiparticle energies of small organic molecules even beyond the size of the molecules used for training. We furthermore analyze the learned DFT-to-GW corrections by mapping them to specific localized fragments of the molecules, in order to develop an intuitive interpretation of the learned corrections, and thus to better understand DFT errors.