Estimating leaf chlorophyll content (LCC) in a timely manner and accurately is of great significance for the precision management of rape. The spectral index derived from UAV images has been adopted as a non-destructive and efficient way to map LCC. However, soil background impairs the performance of UAV-based LCC estimation, limiting the accuracy and applicability of the LCC estimation model, and this issue remains to be addressed. Thus, this research was conducted to study the influence of soil pixels in UAV RGB images on LCC estimation. UAV campaigns were conducted from overwintering to flowering stages to cover the process of soil background being gradually covered by rapeseed plants. Three planting densities of 11.25, 18.75, and 26.26 g/m2 were chosen to further enrich the different soil background percentage levels, namely, the rape fractional vegetation coverage (FVC) levels. The results showed that, compared to the insignificant difference observed for the ground measured LCC at a certain growth stage, a significant difference was found for most of the spectral indices extracted without soil background removal, indicating the influence of soil background. Removing soil background during the extraction of the spectral index enhanced the LCC estimation accuracy, with the coefficient of determination (R2) increasing from 0.58 to 0.68 and the root mean square error (RMSE) decreasing from 5.19 to 4.49. At the same time, the applicability of the LCC estimation model for different plant densities (FVC levels) was also enhanced. The lower the planting density, the greater the enhancement. R2 increased from 0.53 to 0.70, and the RMSE decreased from 5.30 to 4.81 under a low planting density of 11.25 g/m2. These findings indicate that soil background removal significantly enhances the performance of UAV-based rape LCC estimation, particularly under various FVC conditions.