In the central nervous system, oligodendrocytes wrap around neuronal axons to form myelin, an insulating layer or sheath that allows for the efficient conductance of action potentials. In addition to structural insulation, myelin provides encased axons with nutrient, metabolic and defensive support. Demyelination, or myelin loss, can therefore cause axonal dysfunction, leading to neurological impairment and disease. In Alzheimer's disease (AD), progressive white matter demyelination is acknowledged as one of the earliest pathologies preceding symptom onset. Unfortunately, current pharmacotherapy for slowing demyelination or promoting remyelination in AD is nonexistent. Exercise is recognized for its wide-ranging benefits to human health, including improved mental health and the prevention of lifestyle-related diseases. Mounting evidence suggests the contribution of physical activity in delaying the progression of dementia in elderly populations. Recent mechanistic studies have shown that exercise facilitates myelination in the brain through the vitalization of intrinsic pro-myelination cues, such as increased neurotrophic factors and electrical activity. In this review, we summarize and discuss the potential of physical exercise on counteracting aging-associated white matter demyelination, which causes cognitive decline in AD. We highlight the need of further basic and clinical research investigations on this topic to establish novel approaches for healthy and improved brain aging.