This study focused on the synthesis of aragonite whiskers through a synergistic wet carbonation technology utilizing waste magnesia slag (MS) and magnesium sulfate (MgSO4), aiming to improve the microstructure and mechanical properties of ordinary Portland cement (OPC) paste. The influence of MgSO4 concentration on the wet carbonation process, phase composition, and microstructure of MS was investigated. Furthermore, the effect of incorporating carbonated MS (C-MS) on the mechanical properties and microstructure of Portland cement paste was evaluated. Results showed that appropriate MgSO4 concentrations favored aragonite whisker formation. A concentration of 0.075 M MgSO4 yielded 86.6% aragonite with high aspect ratio nanofibers. Incorporating 5% of this C-MS into OPC increased the seven-day compressive strength by 37.5% compared to the control OPC paste. The improvement was attributed to accelerated hydration and reduced porosity by the filling effect and microfiber reinforcement of aragonite whiskers. MS demonstrated good CO2 sequestration capacity during carbonation. This study provides an effective method to synthesize aragonite whiskers from waste MS and use it to enhance cementitious materials while reducing CO2 emissions, which is valuable for the development of a sustainable cement industry.