The microorganisms in marine sediment are promising candidates for the treatment of the saline wastes due to their property of salt tolerance. However, the knowledge about the microbial community and property of the marine sediments is still limited. In the present study, the salt tolerance of the microorganisms in the marine sediment that was collected from a marine fish farm was investigated by being used as inoculum for anaerobic digestion. The microbial communities were analyzed by high-throughput sequencing. The inoculum from the wastewater plant (IWTP) was taken as a control. The inoculum from the marine sediment (IMS) showed excellent capacity for anaerobic digestion at salinities of 0.3%–6%. Even at a salinity of 9%, the methane yield remained 60% of the highest yield. IMS provides promising microbial resources for the treatment of both fresh-water and saliferous organic wastes. While the IWTP was sensitive to salt, the methane yield decreased to 56% of the highest yield at the salinity of 3%. The bacterial taxonomic richness of IMS was about half of that in IWTP. Eighty-one genera were identified only in IWTP but not in IMS. The IMS possessed fewer bacterial members related to the nitrogen cycle than IWTP, but more members related to the sulfur cycle. The members of animal parasites or symbionts in IMS were significantly fewer than those in IWTP. The archaeal compositions of IMS and IWTP were different. The relative abundance of the unidentified archaea in IMS was much higher than that in IWTP with 12.52% vs 0.06% at phylum level. The findings of this work expand our understanding of the microorganisms in marine sediments and will promote the application of them in waste treatment.