Physical fitness (PF) has been associated with improved cognition in older age, but less is known about its effects on different cognitive domains in individuals diagnosed with dementia. We explored the associations between PF and cognitive performance in 40 healthy elderly and 30 individuals with dementia. Participants completed a battery of standardized cognitive tests (Mini-Mental State Exam, Verbal Fluency, Prospective and Retrospective Memory Questionnaire, Clock Drawing, and California Verbal Learning Test) and were classified into high versus low levels of PF based on their score on the Physical Fitness Questionnaire. Analyses took into account age, gender, education, occupation, head injury, Internet use, brain training, and past levels of exercise and revealed overall benefits of PF, in particular for the people with dementia. Discriminant analysis showed high accuracy of reclassification, with most errors being due to the misclassification of dementia cases as healthy when they had high PF. The first discriminant function accounted for 83% of the variance. Using individual estimates of this function, which reflected global cognitive performance, confirmed the beneficial role of PF in dementia, even when taking into account age, past level of exercise, and the number of years since the dementia diagnosis. Finally, univariate analyses confirmed the differential sensitivity of the cognitive tests, with MMSE and clock drawing showing reliable interaction effects. This work shows that PF is associated with a reduced level of cognitive deterioration expected with dementia, especially in executive functioning and provides empirical support for the cognitive benefits of interventions promoting PF for individuals with dementia.