The effect of liver fibrosis on mild cognitive impairment (MCI) and dementia risk in type 2 diabetes mellitus (T2DM) patients is unclear. Therefore, we performed a prospective cross-sectional study on 219 patients with T2DM and older than 60 years to evaluate the association between liver fibrosis, liver steatosis, and cognitive impairment. The Montreal Cognitive Assessment (MoCA) was used to screen for MCI or dementia. Liver fibrosis was estimated using the non-invasive Fibrosis-4 (FIB-4) score, and liver steatosis was assessed with the hepatic steatosis index. The mean age was 71 ± 6 years, 47% were women and according to MoCA cut-off values, 53.88% had MCI and 16.43% had dementia. A moderate or high risk of advanced fibrosis was significantly higher in patients with MCI or dementia compared to those with normal cognition (p < 0.001). After adjusting for confounders, a FIB-4 score greater than 1.54 was associated with MCI or dementia (p = 0.039). Multivariate analysis identified age over 70.5 years, antiplatelet medication use, and a FIB-4 score above 1.54 as the most relevant risk factors. Liver fibrosis, but not liver steatosis, is associated with MCI or dementia in older T2DM patients, suggesting that FIB-4 score might be a simple biomarker for the detection of cognitive impairment.