ABSTRACT It is not well understood how neighborhood disadvantage is associated with specific domains of cognitive function and underlying brain health within older adults. Thus, the objective was to examine associations between neighborhood disadvantage, brain health, and cognitive performance, and examine whether associations were more pronounced among women. The study included 136 older adults who underwent cognitive testing and MRI. Neighborhood disadvantage was characterized using the Area Deprivation Index (ADI). Descriptive statistics, bivariate correlations, and multiple regressions were run. Multiple regressions, adjusted for age, sex, education, and depression, showed that higher ADI state rankings (greater disadvantage) were associated with poorer working memory performance (p < .01) and lower hippocampal volumes (p < .01), but not total, frontal, and white matter lesion volumes, nor visual and verbal memory performance. There were no significant sex interactions. Findings suggest that greater neighborhood disadvantage may play a role in working memory and underlying brain structure.