Evidence from natural populations shows that changes in environmental conditions can cause rapid modifications in the evolutionary potential of phenotypes, partly through genotype-by-environment interactions (G×E). Therefore, the overall rate of microevolution should depend on fluctuations in environmental conditions, even when directional selection is sustained over several generations. We tested this hypothesis in a preindustrial human population that experienced a microevolutionary change in age at first reproduction (AFR) of mothers, using the annual infant mortality rate (IMR) as an indicator of environmental conditions during their early life. Using quantitative genetics analyses, we found that G×Es explained a nonnegligible fraction of the additive genetic variance in AFR and in relative fitness, as well as of the genetic covariance between AFR and fitness (i.e., the Robertson-Price covariance). The covariance was stronger for individuals exposed to unfavorable early-life environmental conditions. Our results unravel the presence of G×Es in an important life history trait and its impact on the rate of microevolution, which appears to have been sensitive to short-term fluctuations in local environmental conditions.