The development of low-cost, efficient and robust non-precious metal electrocatalysts for acidic oxygen evolution (OER) is a major challenge. A novel self-supported N-doped hierarchical Co3O4 (3D Co3O4/NC-250) electrocatalyst with abundant oxygen vacancies was prepared by a facile carbonization-oxidation process. XAFS and EPR results clarified that N doping regulated electronic state of central Co atom to generate abundant oxygen vacancies, which activated lattice oxygen in the water oxidation process. The key intermediate O*O was observed via in-situ ATR-FTIR, providing direct evidence for LOM mechanism on active oxygen vacancy sites. DFT calculations showed that LOM mechanism on oxygen vacancy sites was conducive to reduce adsorption free energy of oxygen-containing intermediates in limiting steps of acidic OER process. The as-prepared 3D Co3O4/NC-250 electrocatalyst exhibited ultra-high OER performance with a low overpotential of 225 mV at 10 mA cm−2 in acidic media. This study provided a new insight to rational design of efficient transition metal-based electrocatalysts for acidic OER.