We show that for classical Liouville field theory, diffeomorphism invariance, Weyl invariance and locality cannot hold together. This is due to a genuine Virasoro center, present in the theory, that leads to an energy-momentum tensor with non-tensorial conformal transformations, in flat space, and with a non-vanishing trace, in curved space. Our focus is on a field-independent term, proportional to the square of the Weyl gauge field, WμWμ, that makes the action Weyl-invariant and was disregarded in previous investigations of Weyl and conformal symmetry. We show this term to be related to the classical center of the Virasoro algebra. The mechanism uncovered here is a classical version of the quantum anomalous phenomenon: the generalization to curved space only allows to keep one of the two symmetries enjoyed by the flat space theory, either Lorentz (diffeomorphism) or conformal invariance.