We present an ab initio time-dependent density functional theory study of the ultrafast demagnetization of elemental ferromagnets subject to terahertz (THz) laser pumping compared to optical laser pumping. We find that THz radiation can cause as much demagnetization as optical radiation for significantly lower-energy transfer to the sample. We show that the demagnetization efficiency of the THz pulse is related to a coherent coupling of the radiation to the electron charge dynamics. In fact, our simulations reveal that the low-frequency THz electric field enforces in-phase electron charge oscillations around the atomic centers, leading to a highly efficient spin-orbit torque switching of the magnetization. Unlike for optical pumping, at the end of the pulse a major fraction of the excited electrons is driven back to equilibrium acquiring a thermalized Fermi-Dirac distribution.