Cibotium barometz, an important traditional Chinese medicine, is used in strengthening bones and tendons. We found that C. barometz crude polysaccharides (CB70) could alleviate bone loss and markedly improve the biomechanical properties of OVX rats. Thus, to clarify biological active ingredient(s) of CB70, two homogeneous polysaccharides (CBP70-1-1 and CBP70-1-2) were purified from CB70. A combination of monosaccharide composition, FT-IR, GC–MS and NMR analysis indicated that CBP70-1-1 was composed of →6)-D-Galp-(1→, D-Glcp-(1→, →3,6)-D-Manp-(1→, →4)-D-Glcp-(1→ and →6)-D-Glcp-(1→ with relative molecular weights of 12,724 Da, and CBP70-1-2 was composed of →4)-D-Glcp-(1→, D-Glcp-(1→, →3,6)-D-Manp-(1→, →6)-D-Galp-(1→, →4,6)-D-Glcp-(1→ and →3)-L-Araf-(1→ with relative molecular weights of 3611 Da. Morphological analyses revealed that CBP70-1-1 and CBP70-1-2 appeared as a sheet that were irregular in size and shape, while the surface of CBP70-1-1 was full of sharp protuberances and CBP70-1-2 was smooth. Furthermore, the effects of CBP70-1-1 and CBP70-1-2 on the proliferation, differentiation and mineralization of mouse pre-osteoblastic MC3T3-E1 cells were assessed via CCK-8 assay, alkaline phosphatase activity assay, and alizarin red-based assay, respectively. These results revealed that CBP70-1-1 and CBP70-1-2 significantly promoted the proliferation, differentiation and mineralization of MC3T3-E1 cells, even better than E2. More importantly, quantitative real-time PCR and Western blot analysis indicated that CBP70-1-2 pronouncedly promoted the expression of osteogenic-related marker genes (Runx2, Osx, Ocn and Opn) and proteins (BMP2, RUNX2, OSX and p-SMAD1), which implies that the osteogenic activity of CBP70-1-2 is accomplished mainly by activating the BMP2/SMAD1 signaling pathway. These findings suggest CBP70-1-2 as a potential natural anti-osteoporotic agent for pharmacotherapy.