Little is known about the effects of exercise training (ET) on lexical characteristics during fluency task and its association with cerebellum functional connectivity. The purposes of this study were (1) to investigate whether ET alters response patterns during phonemic and semantic fluency tasks and (2) to assess the association between ET-related changes in cerebellum functional connectivity (FC) and lexical characteristics during fluency tasks. Thirty-five older adults (78.0 ± 7.1years; 17 mild cognitive impairment (MCI) and 18 healthy cognition (HC)) underwent a 12-week treadmill ET. Before and after ET, cardiorespiratory fitness tests, phonemic and semantic fluency tests, and resting-state fMRI scans were administered. We utilized a seed-based correlation analysis to measure cerebellum FC and linear regression to assess the association of residualized ET-induced Δcerebellum FC with Δtask performance. Improved mean switches and frequency during the phonemic fluency task were observed following ET in all participants. There were significant associations between ET-induced increases in cerebellum FC and greater phonemic fluency task log frequency, increases in mean switches, and a reduction in the number of syllables in HC. Lastly, there was a significant interaction between group and cerebellar connectivity on phonemic fluency mean log frequency and number of syllables. A 12-week walking ET is related to enhanced phonemic fluency lexical characteristics in older adults with MCI and HC. The association between ET-induced increases in cerebellum FC and enhanced response patterns after ET suggests that the cerebellum may play an important role in ET-related improvement in phonemic fluency performance in cognitively healthy older adults.