Deep cosmic microwave background polarization experiments allow a very precise internal reconstruction of the gravitational lensing signal in principle. For this aim, likelihood-based or Bayesian methods are typically necessary, where very large numbers of lensing and delensing remappings on the sphere are sometimes required before satisfactory convergence. We discuss here an optimized piece of numerical code in some detail that is able to efficiently perform both the lensing operation and its adjoint (closely related to delensing) to arbitrary accuracy, using nonuniform fast Fourier transform technology. Where applicable, we find that the code outperforms current widespread software by a very wide margin. It is able to produce high-resolution maps that are accurate enough for next-generation cosmic microwave background experiments on the timescale of seconds on a modern laptop. The adjoint operation performs similarly well and removes the need for the computation of inverse deflection fields. This publicly available code enables de facto efficient spherical harmonic transforms on completely arbitrary grids, and it might be applied in other areas as well.