Developing highly active and cost-efficient electrocatalyst for the oxygen evolution reaction (OER) is crucial for practical water electrolysis, which is an ideal technique for obtaining high-purity hydrogen without causing environmental pollution. In this study, we propose a composite material of bimetallic CoFe-layered double hydroxide (LDH) and oxygen vacancy-rich Co3O4 on carbon support (CoFe-LDH/Vo-Co3O4@C), which is synthesized using co-precipitation, thermal treatment, and facile ultrasonic treatment methods. By efficiently modulating surface electronic structure and introducing abundant structural defect sites, the CoFe-LDH/Vo-Co3O4@C exhibits remarkable OER activity and long-term stability. Specifically, the CoFe-LDH/Vo-Co3O4@C has a low overpotential of 296 mV to achieve a current density of 10 mA cm−2, and a very small Tafel slope of 60.2 mV dec−1, which are much superior values than those of the precious metal-based RuO2 (331 mV and 73.7 mV dec−1). Furthermore, the excellent electrocatalytic durability of the CoFe-LDH/Vo-Co3O4@C for over 100 h in alkaline solution suggests its potential for industrial applications of water electrolysis.