A common scenario of magnetoelectric coupling in multiferroics is the electric polarization induced by spatially modulated spin structures. It is shown in this paper that the same mechanism works in magnetic dielectrics with inhomogeneous magnetization distribution: the domain walls and magnetic vortexes can be the sources of electric polarization. The electric field driven magnetic domain wall motion is observed in iron garnet films. The electric field induced nucleation of vortex state of magnetic nanodots is theoretically predicted and numerically simulated. From the practical point of view the electric field control of micromagnetic structures suggests a low-power approach for spintronics and magnonics.