A decrease in global motion perception (GMP) has been reported in older adults, and this age-related decline in GMP varies with the speed of global motion. However, no studies have investigated whether the asynchronous age-related decline in GMP is related to degenerative changes in brain structure. In this study, the random dot kinematogram paradigm and structural magnetic resonance imaging were used to investigate the asynchronous aging of GMP at fast and slow speeds (called fast GMP and slow GMP, respectively) and their relationships with brain structure. Ninety-four older adults (65.74 ± 4.50 yrs) and 90 younger adults (22.83 ± 4.84 yrs) participated in the experiment. The results showed that older adults had higher motion coherence thresholds (MCT) than younger adults at both fast and slow speeds. Brain-behavior correlation analyses of younger adults revealed that none of the correlations between morphological measures and MCTs survived correction for multiple comparisons. For older adults, slow MCT was correlated with cortical thickness in the bilateral V4v, V5/MT+, left V7, V8, LO, and surface area in the right V7. Fast MCT was significantly correlated with gray matter volume in the right V7 and thickness in the left V5/MT+. These results support the view that global motion extraction occurs within two speed-tuned systems that are at least partially independent in terms of their neural substrates, which deteriorate with age at different speeds. Aging of GMP is also associated with morphological changes in the visual cortex. Age-related cerebral atrophy in the dorsal stream may impair both fast and slow GMP, whereas aging of the ventral stream specifically impairs slow GMP.