Most stochastic problems are based on the independent and symmetrically distributed random variables. Using this principle, in this study, we propose an efficient calculation scheme to obtain the statistical moments based on high-order stochastic perturbation (SP) expansion. This solves the long-existing problem of the SP method, namely, the low calculation efficiency of statistical moments using high-order perturbation expansion, and the complexity of the calculation format. The scheme involves adopting the divided-difference method to approximate the partial derivative items of the SP method. The proposed method, which is similar to the collocation point method, only requires the deterministic calculation result for each node in the divided-difference method. In addition, as the adoption of high-order perturbation expansion is limited in multidimensional random problems due to the curse of dimensionality, an adaptive partial derivative items selection method is proposed in this paper to efficiently and reasonably ignore the expansion items that have an insignificant influence on statistical moments. This considerably improves the calculation efficiency of the statistical moments. Finally, the adaptive divided-difference perturbation (ADDP) method proposed in this paper is compared with the quasi-Monte Carlo method and the adaptive sparse grid methods using three numerical examples. The results prove that the ADDP method not only has high efficiency but also leads to significant improvements in the accuracy of statistical moments.