In this paper, we propose a protocol to realize non-adiabatic holonomic quantum computation (NHQC) of cavity modes via invariant-based reverse engineering. Coupling cavity modes with an auxiliary atom trapped in a cavity, we derive effective Hamiltonians with the help of laser pulses. Based on the derived Hamiltonians, invariant-based reverse engineering is used to find proper evolution paths for NHQC. Moreover,the systematic-error-sensitivity nullified optimal control method is considered in the parameter selections, making the protocol insensitive to the influence of systematic errors of pulses. We also estimate the imperfections induced by random noise and decoherence. Numerical results show that the protocol holds robustness against these imperfections. Therefore, the protocol may provide useful perspectives to quantum computation with optical qubits in cavity quantum electrodynamics systems. This article is part of the theme issue 'Shortcuts to adiabaticity: theoretical, experimental and interdisciplinary perspectives'.