In the context of Horava gravity, the most promising known scenarios to recover Lorentz invariance at low energy are the possibilities that (1) the renormalization group flow of the system leads to emergent infrared Lorentz invariance, and (2) that supersymmetry protects infrared Lorentz invariance. A third scenario proposes that a classically Lorentz invariant matter sector with controlled quantum corrections may simply co-exist with Horava gravity under certain conditions. However, for non-projectable Horava gravity in 3+1 dimensions it is known that, in the absence of additional structures, this mechanism is spoiled by unexpected power-law divergences. We confirm this same result in the projectable version of the theory by employing the recently found gauge-fixing term that renders the shift and graviton propagators regular. We show that the problem persists for all dimensions $D\geq 3$, and that the degree of fine tuning in squared sound speeds between a U(1) gauge field and a scalar field increases with $D$. In particular, this difference in the zero external momentum limit is proportional to $\Lambda^{D-1}$ for $D\geq 3$, where $\Lambda$ is the ultraviolet momentum cutoff for loop integrals, while the power-law divergences are absent for $D=1$ and $D=2$. These results suggest that not only the gravity sector but also the matter sector should exhibit a transition to Lifshitz scaling above some scale, and that there should not be a large separation between the transition scales in the gravity and matter sectors. We close with a discussion of other more promising scenarios, including emergent Lorentz invariance from supersymmetry/strong dynamics, and pointing out challenges where they exist.