Background: Mini-Mental State Exam (MMSE) is widely used for the cognitive assessment in Alzheimer’s disease (AD), but its interpretation could be affected by acute medical conditions, depression, or anxiety. Conversely, brain magnetic resonance imaging (MRI) is an independent diagnostic tool to evaluate specific cerebral pathology. Many studies have investigated the correlation between MRI visual scales such as hippocampal atrophy and baseline MMSE in AD patients. However, the correlation between a comprehensive MRI visual scale and baseline MMSE in people with AD remains less well known. Method: We retrospectively collected records of outpatients diagnosed with probable AD according to DSM -5 criteria. The comprehensive visual rating scale (CVRS) was used to semiquantitatively measure structural changes and vascular lesions. The relationship between MRI changes and baseline MMSE was evaluated using Bayesian model averaging (BMA). Results: A total of 65 patients, among whom 21 (32.31%) had early-onset AD, were included. Lateral temporal atrophy, level of education, and late age at onset were the strongest independent predictors of baseline MMSE. Furthermore, hippocampal atrophy was only correlated with delayed recall, while temporal atrophy was correlated with orientation, attention, language, and visual-spatial items of the MMSE. The cerebral atrophy and small vessel lesions scores of the late-onset AD group were significantly higher than those of the early-onset group, despite negligible differences in education and MMSE. Conclusion: Our study suggests lateral temporal atrophy correlates with baseline MMSE scores in people with AD better than the hippocampal atrophy. Age-related atrophy and silent small vessel disease lesions may have negligible impact on AD patients’ cognitive impairment.