We investigate the D→4 limit of the D-dimensional Einstein-Gauss-Bonnet gravity, where the limit is taken with α˜=(D−4)α kept fixed and α is the original Gauss-Bonnet coupling. Using the ADM decomposition in D dimensions, we clarify that the limit is rather subtle and ambiguous (if not ill-defined) and depends on the way how to regularize the Hamiltonian or/and the equations of motion. To find a consistent theory in 4 dimensions that is different from general relativity, the regularization needs to either break (a part of) the diffeomorphism invariance or lead to an extra degree of freedom, in agreement with the Lovelock theorem. We then propose a consistent theory of D→4 Einstein-Gauss-Bonnet gravity with two dynamical degrees of freedom by breaking the temporal diffeomorphism invariance and argue that, under a number of reasonable assumptions, the theory is unique up to a choice of a constraint that stems from a temporal gauge condition.