Motivated by the challenges of uncertainty quantification for coarse-grained (CG) molecular dynamics, we investigate the role of perturbation theory in model reduction of classical systems. In particular, we consider the task of coarse-graining rigid bodies in the context of generalized multipole potentials that have controllable levels of accuracy relative to their atomistic counterparts. We show how the multipole framework yields a hierarchy of models that systematically connects a CG "point molecule" approximation to the exact dynamics. We use these results to understand when and how the CG models fail to describe atomistic dynamics at the trajectory level and develop asymptotic error estimates for approximate molecular potential energies. Implications for other model-reduction strategies are also discussed. Key findings of this work are that (i) omitting rotational energy introduces significant error when coarse-graining and (ii) attention to symmetry can improve accuracy of "point-molecule" approximations. Analytical derivations and numerical results support these conclusions. Relevance to nonrigid bodies is also discussed.