To examine the association of whole grain consumption and longitudinal change in global cognition, perceptual speed, and episodic memory by different race/ethnicity. We included 3,326 participants from the Chicago Health and Aging Project who responded to a Food Frequency Questionnaire (FFQ), with 2 or more cognitive assessments. Global cognition was assessed using a composite score of episodic memory, perceptual speed, and the Mini Mental State Examination (MMSE). Diet was assessed by a 144-item FFQ. Linear mixed-effects models were used to estimate the association of intakes of whole grains and cognitive decline. This study involved 3,326 participants (60.1% African American [AA], 63.7% female) with a mean age of 75 years at baseline and a mean follow-up of 6.1 years. Higher consumption of whole grains was associated with a slower rate of global cognitive decline. Among AA participants, those in the highest quintile of whole grain consumption had a slower rate of decline in global cognition (β = 0.024, 95% CI [0.008-0.039], p = 0.004), perceptual speed (β = 0.023, 95% CI [0.007-0.040], p = 0.005), and episodic memory (β = 0.028, 95% CI [0.005-0.050], p = 0.01) compared with those on the lowest quintile. Regarding the amount consumed, in AA participants, those who consumed >3 servings/d vs those who consumed <1 serving/d had a slower rate of decline in global cognition (β = 0.021, 95% CI [0.005-0.036], p = 0.0093). In White participants, with >3 servings/d, we found a suggestive association of whole grains with global cognitive decline when compared with those who consumed <1 serving/d (β = 0.025, 95% CI [-0.003 to 0.053], p = 0.08). Among AA participants, individuals with higher consumption of whole grains and more frequent consumption of whole grain had slower decline in global cognition, perceptual speed, and episodic memory. We did not see a similar trend in White adults.