Graphene oxide (GO) is one of the nanomaterials that have attracted the most attention of researchers in recent years. Because of its excellent mechanical, thermal and electrical properties, it is widely used in the fields of energy, photoelectric, catalysis and materials, and it has shown broad development prospects. For the high demand of cement-based materials in buildings, as a new cement composites reinforcement agent, GO has been widely studied by researchers. Based on previous studies, cement, silica fume (SF) and fly ash (FA) were used as composite cementitious materials in this experiment, and the effects of different GO contents (0.00%, 0.01%, 0.03%, 0.05%, 0.07%) on the mechanical properties of mortar were analyzed. In addition, the impermeability and sulfate resistance of the modified mortar were studied. The results showed that under the same curing conditions and ages, when the GO content is 0.05%, the flexural strength of the specimens after 28 days of curing is the highest, reaching 10.8 MPa, and the improvement rate is 16.1%. When the GO content is 0.03%, the compressive strength of the specimens after 28 days of curing is the highest, reaching 64.5 MPa, and the improvement rate is 12.4%. In addition, when the dosage of GO is 0.03%, the durability of the mortar is also extremely improved. The impermeability of the mortar is 80% higher than the reference specimens. The strength retention rate under the action of sulfate corrosion is also the highest, which is 11.3% and 6.7% higher than the compressive strength and flexural strength of the reference specimens that corroded for 3 months. GO was proved to be an excellent reinforcement agent for cement-based materials. Through SEM and XRD analysis, it was found that GO promoted the hydration reaction, and it played a filling effect and template role in the mortar matrix. Its two-dimensional lamella structure regulates the morphology of hydration products and enhances the interface adhesion between the matrix and the aggregate. In addition, its nano-filling effect eliminates a large number of micro-pores in the mortar and increases the density of the matrix.