In this study, we replaced 30% of the cement with steel slag to prepare binding material; additionally, small amounts of CaCO3 and CaSO4·2H2O were added. This was done to study the compound effect of CaCO3 and CaSO4·2H2O on the strength of steel slag-cement binding materials. The hydration degree of the steel slag cementitious material was analyzed by XRD, TG and SEM. The results showed that the optimum proportions of CaCO3 and CaSO4·2H2O were 3% and 2%, respectively. Compared with the steel slag-cement binders without adding CaCO3 and CaSO4·2H2O, the compressive strength increased by 59.9% at 3 days and by 17.8% at 28 days. Acting as the nucleation matrix, CaCO3 could accelerate the hydration of C3S. In addition, CaCO3 was involved in the hydration reaction, generating a new hydration product, which could stably exist in a slurry. Meanwhile, CaSO4·2H2O could increase the number of AFt. The compound effect of CaCO3 and CaSO4·2H2O enhanced the intensity of steel slag-cement binding materials and improved the whole hydration behavior.