Gut-brain axis might play an important role in cognitive impairments by various diseases including Alzheimer's disease (AD). To investigate the differences in gut microbial composition, intestinal barrier function, and systemic inflammation in patients with AD or mild cognitive impairment (MCI), and normal control (NC) cases. A total of 118 subjects (45 AD, 38 MCI, and 35 NC) were recruited. Cognitive function was assessed using Mini-Mental State Examination (MMSE), and Montreal Cognitive Assessment Scale (MoCA). Functional ability was assessed using Activity of Daily Living Scale (ADL). The composition of gut microbiome was examined by 16S rRNA high-throughput sequencing. Phylogenetic Investigation of Communities by Reconstruction of Unobserved States (PICRUSt) was used to predict functional transfer of gut microbiota. Gut barrier dysfunction was evaluated by measuring the levels of diamine oxidase (DAO), D-lactic acid (DA), and endotoxin (ET). The serum high-sensitivity C-reactive protein (hs-CRP) level was used to indicate systemic inflammation. Compared with normal controls, patients with cognitive impairments (AD and MCI) had lower abundance of Dorea and higher levels of DAO, DA, and ET. Kyoto Encyclopedia of Genes and Genomes (KEGG) results showed that the pathways related to glycan biosynthesis and metabolism increased in MCI patients, while the ones related to membrane transport decreased. The abundance of Bacteroides and Faecalibacterium was negatively correlated with the content of ET, and positively correlated with the scores of MMSE and MoCA. The hs-CRP levels were similar among the three groups. A significant negative correlation was observed between the severity of gut barrier dysfunction and cognitive function. Cognitive impairments might be associated with gut microbial dysbiosis and intestinal barrier dysfunction.