Soil quality assessment, as a means to assess the impact of human activities on soil, is of great significance to achieve sustainable development. Proximal sensing offers a rapid and cost-effective means for soil spectral acquisition, which resolves the shortcomings of traditional laboratory analysis and provides a new way for soil quality assessment. However, using a singular spectroscopic technique may not encompass adequate information needed for soil quality assessment. Therefore, in this research, the visible-near-infrared (vis-NIR) and portable X-ray fluorescence (pXRF) spectral data were fused, and the continuous wavelet transform (CWT) was combined with the parallel input 2D convolutional neural network (PI-2D-CNN) algorithm, to enhance the prediction accuracy of soil quality index (SQI). A total of 287 soil samples were collected from the Yarkand-Kashgar River Basin in Xinjiang, China for proximal sensing in the laboratory, and 21 soil physicochemical properties were measured for SQI calculation simultaneously. After preprocessing the vis-NIR and pXRF spectral data with the CWT, partial least squares regression (PLSR) and CNN models were constructed to predict SQI. The results showed that the SQI prediction accuracy based on vis-NIR spectra (coefficient of determination for validation set (R2) = 0.32–0.54) was higher than that based on pXRF spectra (R2 = 0.17–0.36). The prediction accuracy of both vis-NIR and pXRF spectra were improved by CWT combined with CNN. Compared with the traditional spectral data fusion methods (concatenation, sequential and orthogonalised-partial least squares (SO-PLS)), the PI-CNN algorithm demonstrated superior SQI prediction accuracy, and the constructed PI-2D-CNN model obtained the highest prediction accuracy (R2 = 0.67). Overall, this study proved that the combination of CWT and PI-CNN could increase the SQI prediction accuracy. This research will provide a rapid and low-cost approach for large-scale soil quality assessment.