Long-term coal mining has created unique microbial communities in deep coal mines. Revealing the microbial community structure and metabolic functions in the underground environment can contribute to a better understanding of the coal mine ecosystem. In this study, we collected underground sediment samples from producing mines in eastern China at mining depths of −400 to −1100 m and performed high-throughput sequencing. Results showed that most of the genera in the underground sediment can degrade organic matter, such as polycyclic aromatic hydrocarbons (PAHs), benzene, toluene, and xylene, etc. The dominant genera in the underground sediment were Hydrogenophaga, Thauera, Pseudomonas, Rhodobacter, and Dietzia. Samples were divided into coal roadway (CR) and rock roadway (RR) groups according to the sampling location. The microbial community structure differed significantly (p < 0.05) between these two groups of samples, with the distribution of main genera in the CR group samples showing a negative correlation with Cu and a positive correlation with temperature. The samples from the CR and RR groups were significantly different (p < 0.05) in their metabolic functions, including membrane transport, metabolism of other amino acids, folding, sorting, and degradation. Microorganisms in the RR group samples showed high resistance to heavy metals, while microorganisms in the CR group had higher degradation functions of organic pollutants. Bugbase phenotypic predictions indicated a high potential pathogenicity of microorganisms in coal mine sediment, which was mainly contributed by the genera Hydrogenophaga, Pseudomonas, Geothermobacter, and Methylophaga, etc. This study deepens the understanding of microbial communities in deep coal mine environments; however, the organic contamination and biological health risks of underground environments require extensive attention.