Cyclocarya paliurus is an edible and medicinal plant exhibiting significant hypoglycemic effect. However, its active components are still unclear and need further elucidation. In this research, the active components of the leaves of C. paliurus responsible for the α-glucosidase inhibitory activity were screened and identified based on a spectrum-effect relationship study in combination with ultra-performance liquid chromatography tandem mass spectrometry (UPLC-MS/MS) analysis. The 70% ethanol eluate fraction of the leaves of C. paliurus with the strongest α-glucosidase inhibitory activity was obtained after extraction and purification with macroporous resin. Their chromatographic fingerprints (15 batches) were established by UPLC analysis and 32 common peaks were specified by similarity analysis. Their IC50 values for α-glucosidase inhibition were measured by an enzymatic reaction. Several multivariate statistical analysis methods including hierarchical cluster analysis, principal component analysis, partial least square analysis and gray relational analysis were applied to explore the spectrum-effect relationship between common peaks and IC50 values, and the chromatographic peaks making a large contribution to efficacy were screened out. To further elucidate the active components of leaves of C. paliurus, the 70% ethanol eluate fraction was characterized by UPLC-MS/MS analysis, and 10 compounds were identified. This study provides a valuable reference for further research and development of hypoglycemic active components of C. paliurus.