The prospect of controlling the magnetization (M) of a material solely with an external electric field (E) could enable the development of low-power spintronics. Although there has been some success towards this end, most approaches involve controlling interactions at the interface between two different materials rather than switching of a single bulk phase. Here we report the ability to exert complete control over the generation and reversal of the bulk spontaneous M of the single-component multiferroics RFeO3 (R = Dy0.70Tb0.30, Dy0.75Gd0.25) with an E alone. We achieve this by controlling the anisotropic character of rare-earth magnetism and exploiting the competition between different magnetoelectric phases. We also show that whether M is reversed or retained on the E-induced polarization reversal depends on the E modulation speed. This is ascribed to the different dynamical characteristics of ferroelectric and multiferroic domain walls governed by the reversal dynamics of rare-earth moments and iron spins, respectively. The ability to modify a material’s magnetization with an electric field could enable lower-power electronic devices. Such ‘magnetoelectric’ behaviour is usually only seen at the interface between magnetostrictive and electrostrictive materials, but has now been observed in the bulk of single-component rare-earth ferrites.