In many developing countries, the existence of the uncertified recycler seriously hinders the healthy development of the waste electrical and electronic equipment (WEEE or e-waste) recycling industry. As a result, how the government can regulate the uncertified recycler to improve environment and public health during the recycling processes has become a critical issue. To help tackle this issue, we build an evolutionary game model to study the interactions between the government and the uncertified recycler. We conduct stability analysis of each participant and obtain four asymptotically stable states. Furthermore, we conduct numerical simulations for comparative analysis based on the current situation of the Chinese e-waste recycling industry. Our results are as follows. First, there exist multiple asymptotically stable states for the government and the uncertified recycler, namely (no-governance, maintaining status quo), (governance, maintaining status quo), (governance, industrial upgrading), and (no-governance, industrial upgrading). Then, we verify the validity of the evolutionary game model through numerical simulations and find that penalty, supervision cost, additional investment cost, and financial subsidy can significantly influence the behavioral strategy of the government and the uncertified recycler. Finally, we find that the government should adopt the reward-penalty-supervision mechanism to promote the healthy development of the e-waste recycling industry and protect the environment and public health. Specifically, first, the government’s subsidy for the uncertified recycler has upper and lower limits. Exceeding the upper limit will result in an excessive financial burden to the government, while falling below the lower limit will hinder the uncertified recycler from technology upgrading. Second, the government should strengthen the supervision of the uncertified recycler and increase the punishment for violations. Third, the government should focus on controlling the supervision cost. Fourth, according to the asymptotically stable state (no-governance, industrial upgrading), the government should prepare to withdraw from the market when the uncertified recycler chooses industrial upgrading.