We show that when a solid plasma foil with a density gradient on the front surface is irradiated by an intense laser pulse at a grazing angle,${\sim}80^{\circ }$, a relativistic electron vortex is excited in the near-critical-density layer after the laser pulse depletion. The vortex structure and dynamics are studied using particle-in-cell simulations. Due to the asymmetry introduced by non-uniform background density, the vortex drifts at a constant velocity, typically$0.2{-}0.3$times the speed of light. The strong magnetic field inside the vortex leads to significant charge separation; in the corresponding electric field initially stationary protons can be captured and accelerated to twice the velocity of the vortex (100–200 MeV). A representative scenario – with laser intensity of$10^{21}~\text{W}~\text{cm}^{-2}$– is discussed: two-dimensional simulations suggest that a quasi-monoenergetic proton beam can be obtained with a mean energy 140 MeV and an energy spread of${\sim}10\,\%$. We derive an analytical estimate for the vortex velocity in terms of laser and plasma parameters, demonstrating that the maximum proton energy can be controlled by the incidence angle of the laser and the plasma density gradient.