Photocatalysis is a surface catalytic process in which photogenerated electrons are transferred to the surface of the catalyst for water reduction. In this paper, we described a novel two-step calcination strategy that not only realizes the integrating of N-doped carbon dots (NCDs) with g-C3N4 nanosheets, but also can the g-C3N4/NCDs heterojunction provide more catalytic active sites and shortens the distance of charges transport to the surface of materials. The first thermal treatment in the air lead to the formation of bulk g-C3N4 while NCDs was loaded in it. The subsequent secondary calcination was conducive to the exfoliation of bulk g-C3N4 into thinner nanosheets with thickness of 1 nm. Meanwhile, more NCDs insetted in the pristine g-C3N4 were exposed to the surface of materials and more catalytic active site were formed, thus promoting the hydrogen evolution. Electron paramagnetic resonance (EPR) tests certified that the g-C3N4 nanosheet/NCDs composites can generate more OH radical compared with bulk g-C3N4 and bulk g-C3N4/NCDs, which further validated that photogenerated electron-hole was effective separated due to more exposure of NCDs and shorter distance of electron transport to the surface. Impressively, results showed that modification of g-C3N4 with NCDs (1.0 wt % loading) exhibited the highest photocatalytic hydrogen production rate (3319.3 μmol g−1 h−1) and an apparent quantum yield of 29.8 % at 420 nm, which was 13 folds of bulk g-C3N4. Our work illuminates a new method for high activity g-C3N4/carbon dots photocatalyst in the field of photocatalysis.