Macroalgae stand out for their high content of dietary fiber (30–75%) that include soluble, sulfated (fucoidan, agaran, carrageenan, and ulvan) and non-sulfated (laminaran and alginate) polysaccharides. Many studies indicate that these compounds exert varied biological activities and health-promoting effects and for this reason, there is a growing interest for using them in food products. The aim of this review was to critically evaluate prebiotic properties of algal polysaccharides, i.e., their ability to exert biological activities by modulating the composition and/or diversity of gut microbiota (GM). Pre-clinical studies show that the non-sulfated alginate and laminaran are well-fermented by GM, promoting the formation of short chain fatty acids (SCFAs) including butyrate, and preventing that of harmful putrefactive compounds (NH3, phenol, p-cresol, indole and H2S). Alginate increases Bacteroides, Bifidobacterium, and Lactobacillus species while laminaran mostly stimulates Bacteroides sp. Results with sulfated polysaccharides are more questionable. Agarans are poorly fermentable but agarose-oligosaccharides exhibit an interesting prebiotic potential, increasing butyrate-producing bacteria and SCFAs. Though carrageenan-oligosaccharides are also fermented, their use is currently limited due to safety concerns. Regarding fucoidan, only one study reports SCFAs production, suggesting that it is poorly fermented. Its effect on GM does not indicate a clear pattern, making difficult to conclude whether it is beneficial or not. Notably, fucoidan impact on H2S production has not been evaluated, though some studies report it increases sulfate-reducing bacteria. Ulvan is badly fermented by GM and some studies show that part of its sulfate is dissimilated to H2S, which could affect colonic mitochondrial function. Accordingly, these results support the use of laminaran, alginate and agaro-oligosaccharides as prebiotics while more studies are necessary regarding that of fucoidan, carrageenan and ulvan. However, the realization of clinical trials is necessary to confirm such prebiotic properties in humans.