A consensus is emerging on marine microbial biogeography. However, knowledge of the biodiversity and biogeographic patterns of deep-sea microbes and the drivers is far from adequate. In this study, we investigated the diversity and biogeography of prokaryotes at the Indian Ocean Ridges (IOR) using 16S rRNA gene and dsrB sequencing combined with bioinformatics analyses. A total of 39 hydrothermal field sediment (HFS) and non-hydrothermal field sediment (NHFS) samples were collected from the Carlsberg Ridge (CR) and Southwest Indian Ridge (SWIR). Seventy-four prokaryotic phyla were detected in the samples, of which 13 phyla were distributed across all samples. The composition of prokaryotic communities diverged between HFSs and NHFSs, but not between the CR and SWIR. Sixteen phyla were differentially enriched between HFSs and NHFSs. The community structure was significantly correlated to calcium, sulfur, total phosphorus, and total nitrogen, the first three of which were the main distinguishing factors of HFSs and NHFSs. Moreover, a weak distance–decay pattern was observed among the NHFS communities. PICRUSt predictions revealed functional discrepancies in carbon and nitrogen metabolism between the HFS and NHFS communities, and dsrB sequencing analysis indicated higher diversity of sulfate reduction genes and taxa in HFSs than in NHFSs. The diversity and relative abundance of corresponding functional taxa and genes suggested that the nitrogen cycle might be centralized in an energy-generating manner to support the microbial community in NHFSs while the sulfur cycle is more important for the HFS community. These findings provide new insights into microbial biogeography and metabolic differentiation along the IOR that are influenced by hydrothermal activity.