Iron hydroxyl sulfate minerals (IHSMs, including schwertmannite and jarosite) are a promising material for environmental applications with excellent adsorption of metal oxygen anions. The acid mine drainage (AMD) abundant in iron and sulfate ions holds potential for the production of valuable IHSMs, thereby achieving resource recycling whilst simultaneously mitigating water contamination, which is important for the sustainable remediation of AMD. Conventional mineralization, which promotes the generation of minerals from Fe3+ and SO42− through the energy provided by chemical or biological oxidation, can only partially mineralize iron in AMD containing substantial quantities of Fe2+. In this study, an improved method for mineralizing AMD containing iron of a different valence into IHSMs under the induction of KHCO3 was proposed. For AMD containing Fe2+, the combination of KHCO3 and H2O2 was used to promote the hydrolysis of iron (92.7%) and sulfate (11.1%) into IHK minerals, which resulted in a significant increase in iron removal of 75.2% and a minor increase in sulfate removal of 4.9%, compared with the formation of schwertmannite from oxidation solely involving H2O2. For the AMD containing Fe3+, the energy generated by the acid–base reaction in water could also directly promote the formation of IK minerals from 97.2% iron and 6.9% sulfate. XRD and FTIR analyses confirmed the identification of the IHK and IK minerals as IHSMs transitioning from schwertmannite to jarosite. SEM and elemental analyses indicated that the mineral exhibited denser aggregate spheres with the incorporation of KHCO3 in mineralization yet displayed enhanced mineralization abilities for the contaminant ions in AMD. Moreover, despite the SSA of the modified minerals being diminished (2.02, 1.83 and 1.83 m2/g for IH, IHK and IK, respectively), the presence of more sulfate in the mineral enhanced the adsorption capacity of Cr(VI). Furthermore, the water quality results also illustrated that the removal ratios of iron and sulfate in AMD notably increased with the involvement of KHCO3 in mineralization. In conclusion, the KHCO3-induced mineralization of iron-containing (either divalent or trivalent) AMD into IHSMs not only improved the mineralization ratios and contaminant removal ratios for better remediation of AMD but also obtained mineral resources with better adsorption of Cr(VI), thereby fostering the sustainable advancement of the remediation of AMD. Therefore, this innovative strategy employing KHCO3-assisted chemical mineralization to form IHSMs holds ample potential and promises to be an efficacious methodology for the sustainable remediation of iron-rich AMD.