Bone regeneration in large bone defects remains one of the major challenges in orthopedic surgery. Calcium polyphosphate (CPP) scaffolds possess excellent biocompatibility and exhibits good bone ingrowth. However, the present CPP scaffolds lack enough osteoinductive activity to facilitate bone regeneration at bone defects that exceed the critical size threshold. To endow CPP scaffolds with improved osteoinductive activity for better bone regeneration, in this study, a self-assembled coating with chitosan-grafted reduced graphene oxide (CS-rGO) sheets was successfully constructed onto the surface of CPP scaffolds through strong electrostatic interaction and hydrogen bonds. Our results showed that the obtained CPP/CS-rGO composite scaffolds exhibited highly improved biomineralization and considerable antibacterial activity. More importantly, CPP/CS-rGO composite scaffolds could drive osteogenic differentiation of BMSCs and significantly up-regulate the expression of osteogenesis-related proteins in vitro. Meanwhile, the CS-rGO coating could inhibit aseptic loosening and improve interfacial osseointegration through stimulating bone marrow mesenchymal stem cells (BMSCs) to secrete more osteoprotegerin (OPG) and lesser receptor activator of nuclear factor-κB ligand (RANKL). Overall, the CS-rGO coating adjusts CPP scaffolds’ biological environment interface and endows CPP scaffolds with more bioactivity.