Abstract Photon-Induced Near-field Electron Microscopy (PINEM), Kapitza-Dirac (KD) gratings, and ponderomotive phase plates are examples of techniques in which the wave function of an electron in free space is manipulated using light fields: Free Electron Quantum Optics (FEQO). These effects are usually treated in separate theoretical frameworks. In this paper we present a unified, two-pronged framework that can be used to describe and numerically evaluate the performance of a number of FEQO-based electron-optical elements. The first part is an extension of an existing analytical treatment of PINEM, based on solving a relativistically corrected Schrödinger equation. The extension covers both second-order contributions to PINEM and the Kapitza-Dirac effect. The second, novel element of the approach is based on electron wavefront reconstruction by evaluating the quantum mechanical phase along a bundle of classical electron trajectories. The quasi-classical (but fully relativistic) approach lends itself to simulating a wide variety of FEQO devices, including the examples mentioned. We apply both approaches to a few specific experimental configurations: mirror-based first-order PINEM, second-order PINEM in very high laser intensity, and Kapitza-Dirac diffraction. The results show excellent agreement between the analytical results and the quasi-classical simulations. Finally, we propose a setup that combines KD and PINEM to allow for simultaneous coherent energy and transverse momentum shaping of an electron beam, and present simulation results thereof.