Modifiable (physical activity) and non-modifiable (sex and genotype) risk factors interact to affect Alzheimer's disease (AD) risk. Further investigation is necessary to understand if these factors influence brain volume and cognition. The study aimed to assess the effect of physical activity, APOE genotype, and sex on AD risk, brain volume, and cognition. UK Biobank data from 2006 to 2023 was accessed. Physical activity was measured by accelerometers, and International Physical Activity Questionnaire. Outcomes were AD incidence; brain volume (ventricular cerebrospinal fluid and total brain); and cognition (executive function, memory, visuospatial ability, processing speed, and reaction time). Logistic and linear regression models were conducted. 69,060 participants met inclusion criteria (mean age: 62.28 years, SD: 7.84; 54.64% female). Higher self-reported (OR = 0.63, 95% CI [0.40, 1.00], p = 0.047) and accelerometer-assessed (OR = 0.96 [0.93, 0.98], p = 0.002) physical activity was associated with lower disease incidence. Smaller ventricular cerebrospinal fluid volume (β= - 65.43 [- 109.68, - 17.40], p = 0.007), and larger total brain volume (β= 4398.46 [165.11, 8631.82], p < 0.001) was associated with increased accelerometer-assessed and self-reported physical activity respectively. Both brain volume analyses were moderated by sex. Increased accelerometer-assessed physical activity levels were associated with faster reaction time (β= - 0.43 [- 0.68, - 0.18], p = 0.001); though poorer visuospatial ability (β= - 0.06 [- 0.09, - 0.03], p < 0.001), and executive function (β= 0.49 [0.31, 0.66], p < 0.001; β= 0.27 [0.10, 0.45], p = 0.002) was related to self-reported physical activity levels. Higher levels of physical activity reduce AD risk independently of non-modifiable risk factors. Moderation of sex on brain volume highlighted the importance of incorporating non-modifiable risk factors in analysis.