In this paper, we perform the exact diagonalization of a light-matter strongly coupled system taking into account arbitrary losses via both energy dissipation in the optically active material and photon escape out of the resonator. This allows us to naturally treat the cases of couplings with structured reservoirs, which can strongly impact the polaritonic response via frequency-dependent losses or discrete-to-continuum strong coupling. We discuss the emergent gauge freedom of the resulting theory and provide analytical expressions for all the gauge-invariant observables in both the Power-Zienau-Woolley and the Coulomb representations. In order to exemplify the results, the theory is finally specialized to two specific cases. In the first one, both light and matter resonances are characterized by Lorentzian linewidths, and in the second one, a fixed absorption band is also present. The analytical expressions derived in this paper can be used to predict, fit, and interpret results from polaritonic experiments with arbitrary values of the light-matter coupling and with losses of arbitrary intensity and spectral shape in both the light and matter channels. A Matlab code implementing our results is provided.