The littoral zones of lakes are potential hotspots for local algal blooms and biogeochemical cycles; however, the microbial communities within the littoral sediments of eutrophic plateau lakes remain poorly understood. Here, we investigated the taxonomic composition, co-occurrence networks, and potential functional roles of both abundant and rare taxa within bacterial and archaeal communities, as well as physicochemical parameters, in littoral sediments from Erhai Lake, a mesotrophic lake transitioning towards eutrophy located in the Yunnan-Guizhou Plateau. 16S rRNA gene sequencing revealed that bacterial communities were dominated by Proteobacteria, Bacteroidetes, and Chloroflexi, while Euryarchaeota was the main archaeal phylum. Co-occurrence network analysis revealed that keystone taxa mainly belonged to rare species in the bacterial domain, but in the archaeal domain, over half of keystone taxa were abundant species, demonstrating their fundamental roles in network persistence. The rare bacterial taxa contributed substantially to the overall abundance (81.52%), whereas a smaller subset of abundant archaeal taxa accounted for up to 82.70% of the overall abundance. Functional predictions highlighted a divergence in metabolic potentials, with abundant bacterial sub-communities enriched in pathways for nitrogen cycling, sulfur cycling, and chlorate reduction, while rare bacterial sub-communities were linked to carbon cycling processes such as methanotrophy. Abundant archaeal sub-communities exhibited a high potential for methanogenesis, chemoheterotrophy, and dark hydrogen oxidation. Spearman correlation analysis showed that genera such as Candidatus competibacter, Geobacter, Syntrophobacter, Methanocella, and Methanosarcina may serve as potential indicators of eutrophication. Overall, this study provides insight into the distinct roles that rare and abundant taxa play in the littoral sediments of mesotrophic plateau lakes.