Consistent interactions with electromagnetism and gravity for mass m particles of any spin are obtained. This is done by finding interactions which preserve the covariantized massive gauge symmetry present in recently constructed massive particle actions. This gauge principle is sufficient for finding consistent completions of minimal as well as non-minimal couplings of any type. For spins s ≥ 3/2, consistency requires infinitely many interaction terms in the action, including arbitrarily high order derivatives of electromagnetic and gravitational curvatures, with correspondingly high powers of 1/m. These interactions may be formally resummed and expressed in terms of non-local operators. Finally, although the interactions appear non-local, evidence is presented for the existence of a field redefinition which makes the interacting action local. This work provides the first explicit realization of an exactly gauge invariant formulation of massive particles interacting with electromagnetism and gravity.