Background: Atrophy of the deep gray matter (DGM) has been associated with a risk of conversion from mild cognitive impairment (MCI) to Alzheimer’s disease (AD) and the degree of cognitive impairment. However, specific knowledge of the associations between degenerative DGM changes and neurocognitive functions remains limited. Objective: To examine degenerative DGM changes and evaluate their association with neurocognitive functions. Method: We examined DGM volume changes with tensor-based morphometry (TBM) and analyzed the relationships between DGM changes and neurocognitive functions in control (n = 58), MCI (n = 38), and AD (n = 58) groups with multiple linear regression analyses. Results: In all DGM areas, the AD group had the largest changes in TBM volume. The differences in TBM volume changes were larger between the control group and the AD group than between the other pairs of groups. In the AD group, volume changes of the right thalamus were significantly associated with episodic memory, learning, and semantic processing. Significant or trend-level associations were identified between bilateral caudate nucleus changes and episodic memory as well as semantic processing. In the control and MCI groups, very few significant associations emerged. Conclusions: Atrophy of the DGM structures, especially the thalamus and caudate nucleus, is related to cognitive impairment in AD. DGM atrophy is associated with tests reflecting both subcortical and cortical cognitive functions.