To determine how physical activity at various ages over the life course is associated with cognitive impairment in late life. Cross-sectional study. Four U.S. sites. Nine thousand three hundred forty-four women aged 65 and older (mean 71.6) who self-reported teenage, age 30, age 50, and late-life physical activity. Logistic regression was used to determine the association between physical activity status at each age and likelihood of cognitive impairment (modified Mini-Mental State Examination (mMMSE) score >1.5 standard deviations below the mean, mMMSE score</=22). Models were adjusted for age, education, marital status, diabetes mellitus, hypertension, depressive symptoms, smoking, and body mass index. Women who reported being physically active had a lower prevalence of cognitive impairment in late life than women who were inactive at each time (teenage: 8.5% vs 16.7%, adjusted odds ratio (AOR)=0.65, 95% confidence interval (CI)=0.53-0.80; age 30: 8.9% vs 12.0%, AOR=0.80, 95% CI=0.67-0.96); age 50: 8.5% vs 13.1%, AOR=0.71, 95% CI=0.59-0.85; old age: 8.2% vs 15.9%, AOR=0.74, 95% CI=0.61-0.91). When the four times were analyzed together, teenage physical activity was most strongly associated with lower odds of late-life cognitive impairment (OR=0.73, 95% CI=0.58-0.92). However, women who were physically inactive as teenagers and became active in later life had lower risk than those who remained inactive. Women who reported being physically active at any point over the life course, especially as teenagers, had a lower likelihood of cognitive impairment in late life. Interventions should promote physical activity early in life and throughout the life course.