Heavy metal contamination of clay waste through the phosphate beneficiation process is a serious problem faced by scientists and regulators worldwide. Through the beneficiation process, heavy metals naturally present in the phosphate rocks became concentrated in the clay waste. This study evaluated the concentration of heavy metals and their fractions in the clay waste in order to assess the risk of environmental contamination. A five-step sequential extraction method, the risk assessment code (RAC), effects range low (ERL), effects range medium (ERM), the lowest effect level (LEL), the severe effect level (SEL), the redistribution index (U tf), the reduced partition index (I), residual partition index (I R), and the Nemerow multi-factor index (PC) were used to assess for clay waste contamination. Heavy metals were analyzed using high-resolution inductively coupled plasma mass spectrometry (HR-ICP-MS) and inductively coupled plasma optical emission spectroscopy (ICP-OES). Correlation analyses were carried out to better understand the relationships between the chemical characteristics and the contents of the different phase fractions. Concentrations of Cd and Cu confirmed that both were bound to the exchangeable fraction (F1) and the carbonate fraction (F2), presenting higher mobility, whereas Pb was most abundant in the Fe-Mn oxide fraction (F3) and organic matter fraction (F4). The residual fraction (F5) contained the highest concentrations (>60%) of As, Cr, Mo, V, and Zn, with lower mobility. Application of the RAC index showed that Cd and Cu should be considered a moderate risk, whereas As, Cr, Mo, Pb, and Zn presented a low risk. Cadmium and Cu contents in mobile fractions F1 and F2 were higher than ERL but lower than ERM. On the other hand, As, Pb, and Zn contents of mobile fractions F1 and F2 were lower than ERL and ERM guideline values. Moreover, total Pb concentrations in the clay waste were below the lowest effect level (LEL) threshold value period, Cr and Zn values in the clay waste were determined to have exceeded the severe effect level (SEL) limit values, whereas Cd and Cu level ranges between LEL and SEL indicate moderate contamination. I R values of heavy metals in the clay waste confirmed that Cd and Cu were bound to the exchangeable and carbonate fractions and presented higher mobility, whereas As, Cr, Mo, Pb, V, and Zn were bound to organic or residual fractions and consequently exhibit lower mobility. A Nemerow multi-factor index revealed that the mine site contains high levels of Cd, Cu, V, and Zn pollution. As and Cr were found at a moderate level of contamination, whereas Pb was present at a safe level of contamination. The order of the comprehensive contamination indices was Cd > Cu > Mo > Zn > V > Cr > As > Pb, indicating that the assessment of clay waste, especially with Cd and Cu, should be undertaken to control heavy metal contamination in adjacent urban and mine areas at the Eshidiya mines.