With the increasing aging population, dementia has become a significant socioeconomic burden. However, the effects of albumin on delayed recall (DR) impairment remain unclear, and there are limited reports on sex and race differences in this relationship. This study aimed to investigate the association between albumin levels and DR impairment in older adults. A total of 1507 normal cognitive function and 553 DR impairment from the National Health and Nutrition Examination Survey (NHANES) 2011-2014 were included in this cross-sectional analysis. Participants aged 60 years and above were assessed using the Consortium to Establish a Registry for Alzheimer's Disease DR (CERAD-DR) test to evaluate cognitive function. Participants were categorized into DR impairment and normal cognitive function groups according to their CERAD-DR scores. Logistic regression analyses, generalized additive models, and fitted smoothing curves were utilized for data analysis. After adjusting for potential confounders, a negative association was found between albumin levels and cognitive function (odds ratio [OR]=0.60, 95% confidence interval [CI] 0.41-0.87). Subgroup analysis stratified by sex, race/ethnicity, and age revealed that the negative association remained significant in men (OR=0.53, 95%CI 032-0.87), Blacks (OR=0.35, 95%CI 0.17-0.74), and the age group of 60-70 years (OR=0.48, 95%CI 0.28-0.81). However, no significant association was observed in women (OR=0.72, 95%CI 0.41-1.28), whites (OR=0.58, 95%CI 0.31-1.07), or Mexican Americans (OR=1.11, 95%CI 0.35-3.46), as well as the age group of 71-80 years (OR=0.62, 95%CI 0.37-1.03). Our study suggests that elevated albumin levels are associated with a decreased incidence of cognitive function impairment, particularly in older men and Blacks. This finding indicates that maintaining high levels of albumin may be beneficial for cognitive function in older adults.