Graphene oxide/cement composite was prepared using a graphene oxide aqueous solution. The workability and mechanical properties of graphene oxide/cement composite with different concentrations for graphene oxide and the ratio of water to cement were investigated. The results observed were the fluidity of cement pastes decreased noticeably with the addition of graphene oxide and increased with the increase in the ratio of water to cement for all tested samples of different graphene oxide contents. It is indicated that a noticeable inverse correlation between the concentration of graphene oxide and fluidity was observed, and a positive linear relationship between the ratio of water to cement and fluidity was also obtained. The compressive strength of cement pastes significantly improved in the presence of an appropriate concentration of graphene oxide as compared to that of the cement paste without graphene oxide; this difference was due to the denser microstructure of graphene oxide/cement composite than that of the control specimens. With the combined analysis of X-ray diffraction and scanning electron microscopy with energy-dispersive spectrometry, the results showed that graphene oxide could promote and regulate the formation and connection of calcium hydroxide and calcium silicate hydrate during the hydration reaction, forming numerous regular and extremely compact plate-shaped crystals, and the compact plate-shaped microstructures constituted of not only calcium hydroxide and calcium silicate hydrate but also wrapped ettringite. This investigation will provide a flexible way to preparation of graphene oxide/cement composite with wanted fluidity and optimized compressive strength that promote the industry application of graphene oxide/cement composite.