In recent years, magnesium slag has been used as a raw material for solid waste treatment using the carbonization method and has proven to be promising in reducing carbon emissions. In this study, the alkali activation reaction was introduced to promote the carbonization of magnesium slag. The resulting mechanical properties, microstructural attributes, and carbonization mechanism were studied by varying the sodium hydroxide content, temperature, and carbon dioxide concentration during the reaction process. The results showed that the amounts of calcium hydroxide, C-S-H, and calcium carbonate in the reaction products increased with the sodium hydroxide content, which enhanced the compressive strength of the composite. However, it does not influence the carbonization mechanism with the increasing reaction temperature, which only elevates the reaction rate. With the increase in the carbon dioxide concentration during alkali activation, the carbonization reaction is dominated by the amount of CO2 dissolved in the reaction medium, and the carbonization mechanism is changed. Thus, a significant decrease in the calcium hydroxide content and a sharp increase in the calcium carbonate content in the products occurred, which significantly improved the compressive strength of the resulting magnesium slag composite. Among them, the maximum compressive strength is 6.83 MPa.