We discuss the problem of gauge fixing for strongly correlated electrons coupled to quantum light, described by projected low-energy models such as those obtained within tight-binding methods. Drawing from recent results in the field of quantum optics, we present a general approach to write down quantum light-matter Hamiltonian in either dipole or Coulomb gauge which are explicitly connected by a unitary transformation, thus ensuring gauge equivalence even after projection. The projected dipole gauge Hamiltonian features a linear light-matter coupling and an instantaneous self-interaction for the electrons, similar to the structure in the full continuum theory. On the other hand, in the Coulomb gauge the photon field enters in a highly non-linear way, through phase factors that dress the electronic degrees of freedom. We show that our approach generalises the well-known Peierls approximation, to which it reduces when only local, on-site orbital contributions to light-matter coupling are taken into account. As an application, we study a two-orbital model of interacting electrons coupled to a uniform cavity mode, recently studied in the context of excitonic superradiance and associated no-go theorems. Using both gauges we recover the absence of superradiant phase in the ground state and show that excitations on top of it, described by polariton modes, contain instead non-trivial light-matter entanglement. Our results highlight the importance of treating the non-linear light-matter interaction of the Coulomb gauge non-perturbatively, to obtain a well-defined ultrastrong coupling limit and to not spoil gauge equivalence.