In this paper, low-cost dihydrate gypsum (CaSO4⋅2H2O) was used as calcium source together with sodium metasilicate solution to prepare calcium silicate hydrate (C-S-H) gels through a mechanochemical activation strategy by using planetary and vibrational ball-milling. The influences of the prepared C-S-H gels on hydrating and hardening properties of super-retarded cement-based materials with sucrose were investigated. The results showed that CaSO4⋅2H2O reacted with sodium metasilicate solution under mechanochemical effect during both planetary and vibrational ball-milling, although CaSO4⋅2H2O was just slightly dissolved in water. It was also interestingly found that the C-S-H gels can continuously form while resting for a certain time after ball-milling. Under the mechanochemical activation, the prepared C-S-H gels had complete characteristic diffusion peaks and were composed of layered, flocculent and fibrous structures. While incorporating the prepared C-S-H gels into the super-retarded cement paste with sucrose, the cement hydration was greatly accelerated and the compressive strength of the hardened cement-based materials was also effectively improved at early age as expected. Through X-ray diffraction (XRD) analysis, it was found that C-S-H gels can effectively promote the growth of CH crystals along the (001) and (101) crystal planes. The scanning electron microscopy (SEM) results supported that the C-S-H gels inhibited the refinement of ettringite (AFt) crystals in super-retarded HCP with sucrose, which was more obvious at 28 d.