The performance of several standard and popular approaches for calculating X-ray absorption spectra at the carbon, nitrogen, and oxygen K-edges of 40 primarily organic molecules up to the size of guanine has been evaluated, focusing on the low-energy and intense 1s → π* transitions. Using results obtained with CVS-ADC(2)-x and fc-CVS-EOM-CCSD as benchmark references, we investigate the performance of CC2, ADC(2), ADC(3/2), and commonly adopted density functional theory (DFT)-based approaches. Here, focus is on precision rather than on accuracy of transition energies and intensities—in other words, we target relative energies and intensities and the spread thereof, rather than absolute values. The use of exchange–correlation functionals tailored for time-dependent DFT calculations of core excitations leads to error spreads similar to those seen for more standard functionals, despite yielding superior absolute energies. Long-range corrected functionals are shown to perform particularly well compared to our reference data, showing error spreads in energy and intensity of 0.2–0.3 eV and ∼10%, respectively, as compared to 0.3–0.6 eV and ∼20% for a typical pure hybrid. In comparing intensities, state mixing can complicate matters, and techniques to avoid this issue are discussed. Furthermore, the influence of basis sets in high-level ab initio calculations is investigated, showing that reasonably accurate results are obtained with the use of 6-311++G**. We name this benchmark suite as XABOOM (X-ray absorption benchmark of organic molecules) and provide molecular structures and ground-state self-consistent field energies and spectroscopic data. We believe that it provides a good assessment of electronic structure theory methods for calculating X-ray absorption spectra and will become useful for future developments in this field.