A 'novel' pure theory of Einstein-Gauss-Bonnet gravity in four-spacetime dimensions can be constructed by rescaling the Gauss-Bonnet coupling constant, seemingly eluding Lovelock's theorem. Recently, however, the well-posedness of this model has been called into question. Here we apply a `dimensional regularization' technique, first used by Mann and Ross to write down a D→2 limit of general relativity, to the case of pure Einstein-Gauss-Bonnet gravity. The resulting four-dimensional action is a particular Horndeski theory of gravity matching the result found via a Kaluza-Klein reduction over a flat internal space. Some cosmological solutions of this four-dimensional theory are examined. We further adapt the technique to higher curvature Lovelock theories of gravity, as well as a low-energy effective string action with an α′ correction. With respect to the D→4 limit of the α′-corrected string action, we find we must also rescale the dilaton to have a non-singular action in four dimensions. Interestingly, when the conformal rescaling Φ is interpreted as another dilaton, the regularized string action appears to be a special case of a covariant multi-Galileon theory of gravity.