The LouShao Basin in Hunan Province is covered with two sets of black rock series containing various potential toxic heavy metal elements. The rock series are rich in various potential toxic heavy metal elements, and the weathered black shale is the main formation of local farmland soil, making it a typical area with high geological background in China. The purpose of this study is to assess the regional ecological and health risks by studying the correlation between the content of heavy metal elements enriched in rhizosphere soil, non-rhizosphere soil, and crops, and the concentration of heavy metal elements in the main source, black shale rocks. The results show that the elements in the black shale far exceed the world average of rock, severely affecting soil element concentrations. Se element is the most severely polluted in rhizosphere soil, exceeding the background value by 7 times. The concentration of heavy metals between rock-non-rhizosphere soil-rhizosphere soil-crops decreases linearly. The distribution of pollution has strong spatial autocorrelation, and the high geological background is the main reason. Pollution index results show significant pollution of Se, Cr, Hg, and Cd in crops, with the pollution of heavy metal element Cr in corn being extremely serious with Igeo = 6.71. The ecological risk order is Se > Cd > Cr > As > Pb > Hg. Humans mainly contact elements through skin contact and food chain intake. The HI value of heavy metal element Cd is greater than 1, indicating a high carcinogenic risk. Human exposure to Cd through skin contact and food chain intake may lead to related diseases such as kidney stones, and pose greater harm to children.