Exposure to infectious diseases in early life has been linked to increased mortality risk in later life in high-disease settings, such as eighteenth- and nineteenth-century Europe. Less is known about the long-term effects of early-life disease exposure in milder disease environments. This study estimates heterogeneous effects from disease exposure in infancy on later-life mortality in twentieth-century Sweden, by socioeconomic status at birth and sex. Using historical population data for southern Sweden, we study 11,515 individuals who were born in 1905–1929 from age 1 until age 85. We measure exposure to disease using the local post–early neonatal mortality rate in the first 12 months after birth and apply flexible parametric survival models. For females, we find a negative effect on life expectancy (scarring) at ages 1–85 following high disease exposure in infancy, particularly for those born to unskilled workers. For males, we find no negative effect on later-life survival, likely because stronger mortality selection in infancy outweighs scarring. Thus, even as the incidence of infectious diseases declined at the start of the twentieth century, early-life disease exposure generated long-lasting negative but heterogeneous population health effects.