The search for efficient approaches to realize local switching of magnetic moments in spintronic devices has attracted extensive attention. One of the most promising approaches is the electrical manipulation of magnetization through electron-mediated spin torque. However, the Joule heat generated via electron motion unavoidably causes substantial energy dissipation and potential damage to spintronic devices. Here, all-oxide heterostructures of SrRuO3 /NiO/SrIrO3 are epitaxially grown on SrTiO3 single-crystal substrates following the order of the ferromagnetic transition metal oxide SrRuO3 with perpendicular magnetic anisotropy, insulating and antiferromagnetic NiO, and metallic transition metal oxide SrIrO3 with strong spin-orbit coupling. It is demonstrated that instead of the electron spin torques, the magnon torques present in the antiferromagnetic NiO layer can directly manipulate the perpendicular magnetization of the ferromagnetic layer. This magnon mechanism may significantly reduce the electron motion-related energy dissipation from electron-mediated spin currents. Interestingly, the threshold current density to generate a sufficient magnon current to manipulate the magnetization is one order of magnitude smaller than that in conventional metallic systems. These findings suggest a route for developing highly efficient all-oxide spintronic devices operated by magnon current.