Coal gangue releases heavy metals and other pollutants through weathering and rain conditions during the natural accumulation process, which causes serious ecological risks to the surrounding soil environment. This study aimed to analyze the content of heavy metals in the soil around a coal gangue dump and to explore the phytotoxic effects induced by the exposure of soil samples to comprehensively assess the ecological risks of the soil around the coal gangue accumulation area. The results showed that the average contents of Cd, Pb, and Zn in the soil around the coal gangue accumulation area exceeded the background values of soil elements in Shanxi Province, and the content of heavy metals in the soil first increased and then decreased with the distance from the coal gangue dump. The enrichment factor results also showed that the pollution of Cd, Pb, and Zn in the soil was serious. The growth of barley seedlings was inhibited after exposure to the soil at different distances from the gangue dump, and some soils could induce the decrease in chlorophyll and the increase in MDA content; the increase in SOD, POD, and GSH activities; the significant decrease in CAT activities; and the decrease in mitotic index (MI) and significant increase in micronucleus rate (MN) in root tip cells. The Pearson correlation coefficient showed that Cr, As, and Zn were significantly positively correlated with the shoot weight, root weight, and micronucleus rate of barley, whereas Cu and Pb were significantly negatively correlated with the chlorophyll and micronucleus rate, respectively.