Rapid eye movement sleep (REMS) is increasingly suggested as a discriminant sleep state for subtle signs of age-related neurodegeneration. While REMS expression is under strong circadian control and circadian dysregulation increases with age, the association between brain aging and circadian REMS regulation has not yet been assessed. Here, we measure the circadian amplitude of REMS through a 40-h in-lab multiple nap protocol in controlled laboratory conditions, and brain microstructural integrity with quantitative multi-parameter mapping (MPM) imaging in 86 older individuals. We show that reduced circadian REMS amplitude is related to lower magnetization transfer saturation (MTsat), longitudinal relaxation rate (R1) and effective transverse relaxation rate (R2*) values in several white matter regions mostly located around the lateral ventricles, and with lower R1 values in grey matter clusters encompassing the hippocampus, parahippocampus, thalamus and hypothalamus. Our results further highlight the importance of considering circadian regulation for understanding the association between sleep and brain structure in older individuals.