The interaction between light and matter is one of the oldest research areas of quantum mechanics, and a field that just keeps on delivering new insights and applications. With the arrival of cavity and circuit quantum electrodynamics we can now achieve strong light-matter couplings which form the basis of most implementations of quantum technology. But quantum information processing also has high demands requiring total error rates of fractions of percentage in order to be scalable (fault-tolerant) to useful applications. Since errors can also arise from modelling, this has brought into center stage one of the key approximations of quantum theory, the Rotating Wave Approximation (RWA) of the quantum Rabi model, leading to the Jaynes-Cummings Hamiltonian. While the RWA is often very good and incredibly useful to understand light-matter interactions, there is also growing experimental evidence of regimes where it is a bad approximation. Here, we ask and answer a harder question: for which experimental parameters is the RWA, although perhaps qualitatively adequate, already not good enough to match the demands of scalable quantum technology? For example, when is the error at least, and when at most, 1%? To answer this, we develop rigorous non-perturbative bounds taming the RWA. We find that these bounds not only depend, as expected, on the ratio of the coupling strength and the oscillator frequency, but also on the average number of photons in the initial state. This confirms recent experiments on photon-dressed Bloch-Siegert shifts. We argue that with experiments reporting controllable cavity states with hundreds of photons and with quantum error correcting codes exploring more and more of Fock space, this state-dependency of the RWA is increasingly relevant for the field of quantum computation, and our results pave the way towards a better understanding of those experiments.