Current work proposes elm sawdust, poplar sawdust, pine sawdust, and cotton straw with different lignocellulose compositions and structures as the research objects to investigate the relationship between the hypoglycemic activity of mycelium polysaccharides from Inonotus obliquus and lignocellulose biodegradation. Four kinds of lignocellulosic materials could significantly increase the exopolysaccharide content and α-glucosidase inhibition rate and advance the occurrence time of α-glucosidase inhibition activity. Among all groups, the polysaccharide synthesis promotion effect of the cotton straw group was the best, which exopolysaccharide yield was 92.05% higher than that of the control group after 11-day fermentation. Meanwhile, the highest α-glucosidase inhibitory activity was found in the elm sawdust group on the 11th day (30.99%, which was 137.47% higher than control), and the exopolysaccharide in the elm sawdust group showed its effectiveness on glucose consumption of insulin resistant HepG2 cells at the concentration of 20 µg/mL, significantly higher than that of the metformin group (P < 0.05). The cellulose in the non-crystalline region of elm and pine and the hemicellulose of poplar were mainly used in the fermentation of I. obliquus, while the cellulose in the crystalline zone and amorphous zone of cotton straw was degraded to improve the exopolysaccharide content of I. obliquus. This paper revealed the necessity of different kinds of lignocellulose for the synthesis of active polysaccharide from I. obliquus and provided a new idea for the regulation of polysaccharide synthesis pathway.