Sleep disturbances are common among older adults and have been associated with the development of Alzheimer disease (AD), such as amyloid-β (Aβ) pathology. For effective AD prevention, it is essential to pinpoint the specific disturbances in sleep and the underlying 24-hour activity rhythms that confer the highest risk of Aβ deposition. To determine the associations of 24-hour activity rhythms and sleep with Aβ deposition in adults without dementia, to evaluate whether disrupted 24-hour activity and sleep may precede Aβ deposition, and to assess the role of the apolipoprotein E ε4 (APOE4) genotype. This was an observational cohort study using data from the Rotterdam Study. Of 639 participants without dementia who underwent Aβ positron emission tomography (PET) from September 2018 to November 2021, 319 were included in the current study. Exclusion criteria were no APOE genotyping and no valid actigraphy data at the baseline visits from 2004 to 2006 or from 2012 to 2014. The mean (SD) follow-up was 7.8 (2.4) years. Data were analyzed from March 2023 to April 2024. Actigraphy (7 days and nights, objective sleep, and 24-hour activity rhythms), sleep diaries (self-reported sleep), Aβ42/40, phosphorylated tau (p-tau)181 and p-tau217 plasma assays, 18F-florbetaben PET (mean standard uptake value ratio [SUVR] in a large cortical region of interest), and APOE4 genotype. Association of objective and self-reported sleep and 24-hour activity rhythms at baseline with brain Aβ PET burden at follow-up. The mean (range) age in the study population was 61.5 (48-80) years at baseline and 69.2 (60-88) years at follow-up; 150 (47%) were women. Higher intradaily variability at baseline, an indicator of fragmented 24-hour activity rhythms, was associated with higher Aβ PET burden at follow-up (β, 0.15; bootstrapped 95% CI, 0.04 to 0.26; bootstrapped P = .02, false discovery rate [FDR] P = .048). APOE genotype modified this association, which was stronger in APOE4 carriers (β, 0.38; bootstrapped 95% CI, 0.05 to 0.64; bootstrapped P = .03) compared to noncarriers (β, 0.07; bootstrapped 95% CI, -0.04 to 0.18; bootstrapped P = .19). The findings remained largely similar after excluding participants with AD pathology at baseline, suggesting that a fragmented 24-hour activity rhythm may have preceded Aβ deposition. No other objective or self-reported measure of sleep was associated with Aβ. Among community-dwelling adults included in this study, higher fragmentation of the 24-hour activity rhythms was associated with greater subsequent Aβ burden, especially in APOE4 carriers. These results suggest that rest-activity fragmentation could represent a modifiable risk factor for AD.