In this work, the Z-scheme layered double hydroxide (LDH)/reduced graphene oxide (RGO)/indium vanadate (InVO4) heterojunction was synthesized by a simple hydrothermal synthesis. RGO, with excellent electron mobility, endowed rapid charge migration and separation between the CoAl-LDH and InVO4. Various characterizations and tests showed that the LDH/RGO/InVO4 heterojunction exhibited superior optical properties, and its carriers’ transfer pathway conformed to Z-scheme mechanism. Under visible light illumination, CO was observed to be the main product during photocatalytic CO2 reduction, and the LDH/RGO/InVO4-15 exhibited the highest CO yield of 204.86 μmol g−1 within 2 h, approximately 2.62 times of pristine CoAl-LDH (78.2 μmol g−1). In addition, LDH/RGO/InVO4-15 showed favorable structure stability and performance stability. In situ fourier infrared spectroscopy (ISFT-IR) analysis demonstrated that the predominant intermediates of CO2 reduction process were carboxylate (COOH*) and bicarbonate (HCO3*). Accordingly, possible pathways of CO2 reduction to CO were deduced. This work can advance the development of efficient Z-scheme heterojunction photocatalysts for CO2 reduction.