A generic strategy based on chemical fingerprinting is proposed for the differentiation of polysaccharides from Astragalus membranaceus (AEP) and A. mongholicus (AOP), using multiple chromatographic and mass spectrometric techniques. Complete and mild acid hydrolysis and Smith degradation were employed for the depolymerization of polysaccharides. The corresponding digested products were efficiently separated and detected using GC–MS, HILIC-ELSD and HR-ESI−-MS. The resulting bottom-up fingerprinting reflected the variations in native polysaccharides, which may be attributed to the three structural levels, which are monosaccharide compositions, glycosidic linkages, and skeletal structure. Similarity analysis from GC–MS and HILIC-ELSD fingerprinting showed that the correlation coefficients from homologous species were more than 0.914, whereas those from heterologous species were less than 0.796. It also noted that characteristic peak area ratio of Man/Gal in Smith fingerprinting could be used a feasible parameter for direct discrimination of AEP and AOP. Principal component analysis from m/z fingerprinting data resulted in clear clustering of AEP and AOP based on changes of oligosaccharides in mild acid hydrolyzates. By combining the accurate m/z, ESI−-MS/MS and methylation assays, the structures of a series of hexose glycopolymers in mild acid hydrolyzates were characterized by predominant 1,6 glycosidic linkages along with minor 1,4 glycosidic linkages. The established chemical fingerprinting is not only an interconnected structure mapping but also a powerful approach for the evaluation of polysaccharides from traditional Chinese medicines.