Photonic materials are an emerging platform to explore quantum matter (Carusotto and Ciuti 2013 Rev. Mod. Phys. 85 299; Sommer et al 2015 arXiv:1506.00341) and quantum dynamics (Peyronel et al 2012 Nature 488 57). The development of Rydberg electromagnetically induced transparency (Weatherill et al 2008 J. Phys. B: At. Mol. Opt. Phys. 41 201002; Petrosyan et al 2011 Phys. Rev. Lett. 107 213601) provided a clear route to strong interactions between individual optical photons. In conjunction with carefully designed optical resonators, it is now possible to achieve extraordinary control of the properties of individual photons, introducing tunable gauge fields (Schine et al 2016 Nature 534 671) whilst imbuing the photons with mass and embedding them on curved spatial manifolds (Sommer et al 2016 New J. Phys. 18 035008). Building on work formalizing Rydberg mediated interactions between propagating photons (Gorshkov et al 2011 Phys. Rev. Lett. 107 133602; Gullans et al 2016 Phys. Rev. Lett. 117 113601), we develop a theory of interacting Rydberg polaritons in multimode optical resonators, where the strong interactions are married with tunable single particle properties to build and probe exotic matter. In the presence of strong coupling between the resonator field and a Rydberg-dressed atomic ensemble, a quasiparticle called the ‘cavity Rydberg polariton’ emerges. We investigate its properties, finding that it inherits both the fast dynamics of its photonic constituents and the strong interactions of its atomic constituents. We develop tools to properly renormalize the interactions when polaritons approach each other, and investigate the impact of atomic motion on the coherence of multi-mode polaritons, showing that most channels for atom-polariton cross-thermalization are strongly suppressed. Finally, we propose to harness the repeated diffraction and refocusing of the optical resonator to realize interactions which are local in momentum-space. This work points the way to efficient modeling of polaritonic quantum materials in properly renormalized strongly interacting effective theories, thereby enabling experimental studies of photonic fractional quantum Hall fluids and crystals (Sommer et al 2015 arXiv:1506.00341; Umucalılar et al 2014 Phys. Rev. A 89 023803; Grusdt and Fleischhauer 2013 Phys. Rev. A 87 043628), plus photonic quantum information processors and repeaters (Jaksch et al 2000 Phys. Rev. Lett. 85 2208; Saffman et al 2010 Rev. Mod. Phys. 82 2313; Han et al 2010 Phys. Rev. A 81, 052311).