Sediment in coral reef habitats has a tight interaction with coral reefs. Bacterial communities are a critical part of sediments and play a vital role in marine ecosystems, as well as coral reef habitats. To determine whether the bacterial structure of sediments differs between the inside and surrounding coral area, we compared the bacterial community on the surface of the dead coral skeleton (CS) and three soft sediment sites (SS, SS10, SS50) in and around the coral reef distribution of Dalajia Island in Daya Bay. Our results showed that bacterial diversity and composition of the coral skeleton had a significant difference with that of surrounding soft sediments. Proteobacteria and Actinobacteria were dominant in CS, and Proteobacteria and Bacteroidetes were dominant in SS, SS10, and SS50. Substrate type and environmental parameters are the main factors in the separation between coral skeletons and sediment. Salinity and pH were largely negatively correlated with CS, indicating they are primary factors that potentially impact coral mortality and revival. Heavy metals were positively related to sediments and they are potential threats to coral growth. Our work provides updated data on bacterial structure in the coastal coral area, contributing to policy-making and management of coastal activities to minimize the degradation of coral reefs.