Quantifying and mapping heavy metals’ concentrations in the soil are important in monitoring and managing heavy metal pollution in the mining areas. However, the cover on the soil acts as a balk when retrieving information from soil. In order to retrieve heavy metal pollution precisely and quickly from hyperspectral images, this study presents a new method to remove non-soil information based NDVI from hyper-spectral and multi-spectral images. The method assumed that the mixed objects in each pixel of remote sensing images are composed only of soil and vegetation-based non-soil end-generational endmembers, then, the soil information of each pixel can be compensated with the non-soil information removed based on its NDVI. Thus, the soil DN value can be corrected to retrieve soil information more precisely. The method has been used on the Hyperion image in June 8, 2002 and the Gaofen-2 (GF-2) image in February 14, 2016 to retrieve the heavy metals’ contents in Bai-ma and De-sheng mining areas, Miyi County, Sichuan Province. From the non-soil information removed images, the R2 and RMSE of the models of estimating Cr, Ag, Cu and Ba in soil are 0.68, 0.724, 0.71, 0.695 and 75.96, 0.03, 52.88, 284.70 respectively. From the original images, the R2 and RMSE of the models of estimating Cr, Ag, Cu and Ba in soil are 0.67, 0.385, 0.425, 0.406 and 80.11, 0.18, 53.43, 396.49 respectively. The retrieval results show that the non-soil information removed images are superior to original images in soil heavy metals’ contents retrieval. This indicates that this method is feasible, and it can be used in soil information retrieval.