The purification of zinc sulfate electrolytes in zinc hydrometallurgy has always been a vexing problem. The aggregation of zinc powder and the removal of recalcitrant Co(II) are the most challenging. Here, we propose a new method for removing Co(II) from a zinc sulfate solution using ultrasonic-enhanced zinc powder replacement. The parameters affecting Co(II) removal are investigated to obtain the optimum conditions. The Co(II) removal efficiency increases remarkably by 26.6% and the required amount of zinc powder was correspondingly reduced by 50% after introducing ultrasound and activators. The optimal dosage of activators Sb2O3 and Cu(II) is CCu(II): CSb(III): CCo(II) = 2: 1: 1. Moreover, the addition of Cu(II) does not deteriorate the quality of zinc sulfate solution because of its low residual concentration (0.84 mg/L). The removal kinetics show that the activation energy of Co-precipitation decreases by 32.27 kJ/mol, and the conversion range of the kinetic mechanism is increased to CCo(II),0 = 60–80 mg/L. The particle diameter of the purified slags decreases from 136.22 to 19.81 µm compared with that of conventional experiments. The cobalt removal mechanism mainly involves that ultrasound promotes the dispersion of zinc powder, opens the package and reduces the activation energy; the Co-Sb-Cu alloys reduce the Co-precipitation overpotential and inhibit the redissolution of Co. The proposed method presents a promising solution to the challenging problem of Co(II) removal. This study opens new possibilities for the application of ultrasound in the metallurgical fields for more efficient and energy-saving production.