This paper presents a compact two-dimensional varifocal-scanning imaging device, with the capability of continuously variable focal length and a large scanning range, actuated by artificial muscle material. The varifocal function is realized by the principle of laterally shifting cubic phase masks and the scanning function is achieved by the principle of the decentered lens. One remarkable feature of these two principles is that both are based on the lateral displacements perpendicular to the optical axis. Artificial muscle material is emerging as a good choice of soft actuators capable of high strain, high efficiency, fast response speed, and light weight. Inspired by the artificial muscle, the dielectric elastomer is used as an actuator and produces the lateral displacements of the Alvarez lenses and the decentered lenses. A two-dimensional varifocal scanning imaging device prototype was established and validated through experiments to verify the feasibility of the proposed varifocal-scanning device. The results showed that the focal length variation of the proposed varifocal scanning device is up to 4.65 times higher (31.6 mm/6.8 mm), and the maximum scanning angle was 26.4°. The rise and fall times were 110 ms and 185 ms, respectively. Such a varifocal scanning device studied here has the potential to be used in consumer electronics, endoscopy, and microscopy in the future.