To address the time-consuming identification and poor timeliness of environmental pollution tracing problems of traditional chemical detection methods for hazardous solid waste, a rapid identification and characterization methodology for hazardous waste containing heavy metals is established by using the EDXRF spectroscopy principle combined with an optical intelligent optimization algorithm. Two years of long-time serial optical spectrum information are collected from nine typical types of hazardous waste containing heavy metals produced in the zinc smelting process using EDXRFs, online analytical instruments installed on different process lines. Then, different machine learning models are applied to identify the key spectral factors that are first screened after noise reduction and standardization to pretreat all spectrum information data to distinguish different characteristics of each type of hazardous waste. Finally, the random forest model is selected and optimized to improve its identification accuracy. The results show that the spectral line signals corresponding to heavy metal elements such as Fe, Zn, Cu, Cd, and Ag in zinc smelting materials can be used as important factors to distinguish different types of zinc smelting solid waste. Because different smelting processes lead to different heavy metal contents in different types of zinc smelting hazardous waste, the optimized random forest model can be implemented to accurately identify and characterize nine typical types of hazardous waste containing heavy metals, and an accuracy rate of 100% is achieved. This method provides an efficient tool for the environmental management of hazardous waste and the rapid tracing of environmental risks in the future. Especially in tracking soil or surface water pollution caused by the illegal dumping of solid waste, the rapid identification and characterization of solid waste are of great significance for the quality supervision of regional ecological environments.