A large number of compounds mimicking the structures of monosaccharides or oligosaccharides have been discovered from natural sources. Such sugar mimics inhibit carbohydrate-degrading enzymes because of a structural resemblance to the sugar moiety of the natural substrate. Carbohydrate-degrading enzymes are involved in a wide range of important biological processes, such as intestinal digestion, posttranslational processing of the sugar chain of glycoproteins, their quality control mechanisms, lysosomal catabolism of glycoconjugates, and some viral infections. It has now been realized that inhibitors of the enzymes have enormous therapeutic potential in diabetes and lysosomal storage disorders. In this review, the general bioactivity, current applications, and the prospects for new therapeutic applications are described.