Weak gravitational lensing changes the angular power spectra of the cosmic microwave background (CMB) temperature and polarization in a characteristic way containing valuable information for cosmological parameter estimation and weak lensing reconstructions. So far, analytical expressions for the lensed CMB power spectra assume the probability density function (PDF) of the lensing excursion angle to be Gaussian. However, coherent light deflection by nonlinear structures at low redshifts causes deviations from a pure Gaussian PDF. Working in the flat-sky limit we develop a method for computing the lensed CMB power spectra which takes these non-Gaussian features into account. Our method does not assume any specific PDF but uses instead an expansion of the characteristic function of the lensing excursion angle into its moments. Measuring these in the CMB lensing deflection field obtained from the Millennium Simulation we show that the change in the lensed power spectra is only at the 0.1% - 0.4% level on very small scales (below 4 arcmin) and demonstrate that the assumption of a Gaussian lensing excursion angle PDF is well applicable.