In this work, we investigate the 4d path integral for Euclidean quantum gravity on a hypercubic lattice, as given by the Spin Foam model by Engle, Pereira, Rovelli, Livine, Freidel and Krasnov (EPRL-FK). To tackle the problem, we restrict to a set of quantum geometries that reflects the large amount of lattice symmetries. In particular, the sum over intertwiners is restricted to quantum cuboids, i.e. coherent intertwiners which describe a cuboidal geometry in the large-$j$ limit. Using asymptotic expressions for the vertex amplitude, we find several interesting properties of the state sum. First of all, the value of coupling constants in the amplitude functions determines whether geometric or non-geometric configurations dominate the path integral. Secondly, there is a critical value of the coupling constant $\alpha$, which separates two phases. In both phases, the diffeomorphism symmetry appears to be broken. In one, the dominant contribution comes from highly irregular, in the other from highly regular configurations, both describing flat Euclidean space with small quantum fluctuations around them, viewed in different coordinate systems. On the critical point diffeomorphism symmetry is nearly restored, however. Thirdly, we use the state sum to compute the physical norm of kinematical states, i.e. their norm in the physical Hilbert space. We find that states which describe boundary geometry with high torsion have exponentially suppressed physical norm. We argue that this allows one to exclude them from the state sum in calculations.