In this article we discuss the invariance under general changes of reference frame of all the physical predictions of particle detector models in quantum field theory in general and, in particular, of those used in quantum optics to model atoms interacting with light. We find explicitly how the light-matter interaction Hamiltonians change under general coordinate transformations, and analyze the subtleties of the Hamiltonians commonly used to describe the light-matter interaction when relativistic motion is taken into account.