Development of low-cost electrocatalysts toward oxygen evolution (OER) and hydrogen evolution reactions (HER) is crucial for large-scale and clean hydrogen production. Cost-effective transition metal oxide-based catalysts are superbly active for OER; however, their applications in catalyzing HER remain challenging due to unsatisfactory activity and intrinsically poor electronic conductivity. Here, we report the synthesis of NiO nanorods (NRs) with abundant oxygen (O) vacancies via a facile cation exchange strategy. Based on the experimental studies and density functional theory calculations, we demonstrate that the chemical and electronic property of NiO NRs is successfully optimized through O-vacancy engineering; the O-vacancies on the surface of NiO NRs remarkably enhance their electronic conduction and promote HER reaction kinetics simultaneously. The resulting NiO NRs exhibit excellent alkaline HER catalytic activity and durability. Furthermore, these specific designed NiO NRs in situ on carbon fiber paper substrates were directly employed as both HER and OER catalysts for overall water splitting, affording better performance than benchmark Pt and RuO2 catalysts. The successful synthesis of these metal oxides nanomaterials with abundant O-vacancies may pave a new path for rationally fabricating efficient HER/OER bi-functional catalysts.