With the increasing accumulation of alkaline industrial solid waste, the mineralization of CO2 using alkaline industrial solid waste has broad application prospects. Carbide slag is highly alkaline and contains a large amount of calcium elements, making it an excellent material for CO2 mineralization. Our idea was to acquire qualified products and fast kinetics by integrating carbide slag utilization and carbon reduction. The reaction route was divided into two steps: calcium extraction and carbonization. In order to achieve efficient extraction of utilizable calcium, we selected NH4Ac as the extraction agent, which has the advantage of buffer protection and environmental friendliness due to being an acetate radical. The extraction efficiency of utilizable calcium exceeded 90% under the conditions of L/S 20:1 and NH4+/Ca2+ 2:1. In the carbonization process, the crystal forms of CaCO3 synthesized by direct carbonation, acid extraction, and ammonium salt were characterized. The formation mechanism of vaterite in ammonium solution and the influence of impurities (Al3+, Mg2+) on the crystal transformation were revealed. This study provides technical support for using alkaline industrial waste to prepare high-purity vaterite. Therefore, alkaline industrial waste can be efficiently and sustainably utilized through CO2 mineralization.