Co-based catalysts are the most promising catalysts in catalytic oxidation of volatile organic compounds (VOCs). Precious metal doping is adopted to improve the catalytic activity of toluene on Co3O4 catalysts, but greatly increases its cost along with it. It is found that doping a small amount of rare earth (Ce, Pr, Sm and Nd) can dramatically promote the catalytic activity of Co3O4. Especially, the Nd-doped Co3O4 catalyst exhibits excellent catalytic activity with a toluene removal rate of 90% at 162.1 °C, which is even better than that of Pt-doped Co3O4. Compared with other rare earth metal doping, the Nd doping leads to a higher ratio of Co3+/Co2+ and has more oxygen vacancies. The in situ diffuse reflectance infrared Fourier transform spectroscopy (DRIFTS) experiments show that the lattice oxygen of Nd–Co sample can be utilized at a quite low temperature, while that of pure Co3O4 cannot engage in oxidation reaction when the temperature is below 200 °C, which visually demonstrates the main reason for the improved catalytic performance of Nd–Co catalyst.