Marine sediments constitute the world’s most substantial long-term carbon repository. The microorganisms dwelling in these sediments mediate the transformation of fixed oceanic carbon, but their contribution to the carbon cycle is not fully understood. Previous culture-independent investigations into sedimentary microorganisms have underscored the significance of carbohydrates in the carbon cycle. In this study, we employ a metagenomic methodology to investigate the distribution and abundance of carbohydrate-active enzymes (CAZymes) in 37 marine sediments sites. These sediments exhibit varying oxygen availability and were isolated in diverse regions worldwide. Our comparative analysis is based on the metabolic potential for oxygen utilisation, derived from genes present in both oxic and anoxic environments. We found that extracellular CAZyme modules targeting the degradation of plant and algal detritus, necromass, and host glycans were abundant across all metagenomic samples. The analysis of these results indicates that the oxic/anoxic conditions not only influence the taxonomic composition of the microbial communities, but also affect the occurrence of CAZyme modules involved in the transformation of necromass, algae and plant detritus. To gain insight into the sediment microbial taxa, we reconstructed metagenome assembled genomes (MAG) and examined the presence of primary extracellular carbohydrate active enzyme (CAZyme) modules. Our findings reveal that the primary CAZyme modules and the CAZyme gene clusters discovered in our metagenomes were prevalent in the Bacteroidia, Gammaproteobacteria, and Alphaproteobacteria classes. We compared those MAGs to organisms from the same taxonomic classes found in soil, and we found that they were similar in its CAZyme repertoire, but the soil MAG contained a more abundant and diverse CAZyme content. Furthermore, the data indicate that abundant classes in our metagenomic samples, namely Alphaproteobacteria, Bacteroidia and Gammaproteobacteria, play a pivotal role in carbohydrate transformation within the initial few metres of the sediments.