The PROSAIL model is widely used to retrieve vegetation parameters from remote sensing data in agriculture regions. Soil reflectance is a key input to the PROSAIL model, but its influence on vegetation parameters retrieval accuracy of crops is rarely discussed. Therefore, this study investigated the influence of different soil reflectance schemes on leaf area index (LAI) and fractional vegetation cover (FVC) retrieval accuracy based on the PROSAIL model in an agriculture region. Firstly, three sources of soil reflectance including ICRAF-ISRIC soil spectral library (SR_SSL), general spectral vectors model (SR_GSV), and ASD spectroradiometer measurement (SR_ASD) and two reflectance extraction schemes were used to generate input soil reflectance for the PROSAIL model to simulate canopy reflectance. Then, the LAI and FVC retrieval models were developed using the random forest algorithm and validated using field survey data. Determination coefficient (R2), Root Mean Square Error (RMSE) and normalized RMSE (NRMSE) were used to evaluate the accuracy of LAI and FVC retrieval. Under the direct extraction scheme, the LAI retrieval based on SR_ASD (R2 = 0.78, RMSE = 0.613, NRMSE = 0.269) achieved better performance than SR_GSV (R2 = 0.73, RMSE = 0.671, NRMSE = 0.294) and SR_SSL (R2 = 0.71, RMSE = 0.762, NRMSE = 0.334), whereas the performances of FVC retrieval were comparable for SR_ASD (R2 = 0.91, RMSE = 0.084, NRMSE = 0.136), SR_GSV (R2 = 0.90, RMSE = 0.086, NRMSE = 0.139) and SR_SSL (R2 = 0.89, RMSE = 0.091, NRMSE = 0.147). The influence of soil reflectance on LAI retrieval is larger than that of FVC in this study. Furthermore, soil reflectance was more important for canopies characterized by low LAI and FVC. In addition, the combination of SR_SSL and multiplication coefficients scheme could be conveniently used for large areas vegetation parameters retrieval. While SR_GSV and SR_ASD with direct extraction scheme was suitable for small areas with field survey data and more homogeneous region.