Heavy metals pollution is a notable threat to environment and human health. This study evaluated the potential ecological and health risks of heavy metals (Cu, Cr, Cd, Pb, Zn, Ni, and As) and their accumulation in a peanut-soil system based on 34 soil and peanut kernel paired samples across China. Soil As and Cd posed the greatest pollution risk with 47.1% and 17.6% of soil samples exceeding the risk screen levels, respectively, with 26.5% and 20.6% of the soil sites at relatively strong potential ecological risk level, respectively, and with the geo-accumulation levels at several soil sites in the uncontaminated to moderately contaminated categories. About 35.29% and 2.94% of soil sites were moderately and severely polluted based on Nemerow comprehensive pollution index, respectively, and a total of 32.4% of samples were at moderate ecological hazard level based on comprehensive potential ecological risk index values. The Cd, Cr, Ni, and Cu contents exceeded the standard in 11.76, 8.82, 11.76 and 5.88% of the peanut kernel samples, respectively. Soil metals posed more health risks to children than adults in the order As > Ni > Cr > Cu > Pb > Zn > Cd for non-carcinogenic health risks and Ni > Cr ≫ Cd > As > Pb for carcinogenic health risks. The soil As non-cancer risk index for children was greater than the permitted limits at 14 sites, and soil Ni and Cr posed the greatest carcinogenic risk to adults and children at many soil sites. The metals in peanut did not pose a non-carcinogenic risk according to standard. Peanut kernels had strong enrichment ability for Cd with an average bio-concentration factor (BCF) of 1.62. Soil metals contents and significant soil properties accounted for 35–74% of the variation in the BCF values of metals based on empirical prediction models.