Cognitive impairment is a clinical manifestation that occurs in the course of dementia like Alzheimer's disease. The association between cognitive impairment and gut microbiota is unclear. We aimed to identify gut microbiota characteristics and key gut microbiota biomarkers associated with cognitive impairment in a relatively large cohort of older adults in China. A total of 229 adults aged ≥60 years from Shenzhen, China were recruited into this cross-sectional study. Participants were divided into cognitive impairment (CI) and no cognitive impairment (NCI) groups according to the results of the Mini-Mental State Examination. Diversity analysis and network analysis were used to characterize the gut microbiota between the two groups. The linear discriminant analysis effect size method and machine learning approaches were sequentially performed to identify gut microbiota biomarkers. The relationship between biomarkers and lifestyle factors was explored using Transformation-based redundancy analysis (tb-RDA). A total of 74 CI participants and 131 NCI participants were included in the analysis. The CI group demonstrated lower α-diversity compared to the NCI group (Shannon: 2.798 versus 3.152, p < 0.001). The density of the gut microbiota interaction network was lower in the CI group (0.074) compared to the NCI group (0.081). Megamonas, Blautia, Pseudomonas, Stenotrophomonas, and Veillonella were key biomarkers for CI. The tb-RDA revealed that increased fruit intake and exercise contribute to a higher abundance of Megamonas, Blautia, and Veillonella. We identified a significantly reduced abundance of certain beneficial gut microbiota in older Chinese adults with cognitive impairment.