Abstract
With the continuous development of deep-learning technology, ever more advanced face-swapping methods are being proposed. Recently, face-swapping methods based on generative adversarial networks (GANs) have realized many-to-many face exchanges with few samples, which advances the development of this field. However, the images generated by previous GAN-based methods often show instability. The fundamental reason is that the GAN in these frameworks is difficult to converge to the distribution of face space in training completely. To solve this problem, we propose a novel face-swapping method based on pretrained StyleGAN generator with a stronger ability of high-quality face image generation. The critical issue is how to control StyleGAN to generate swapped images accurately. We design the control strategy of the generator based on the idea of encoding and decoding and propose an encoder called ShapeEditor to complete this task. ShapeEditor is a two-step encoder used to generate a set of coding vectors that integrate the identity and attribute of the input faces. In the first step, we extract the identity vector of the source image and the attribute vector of the target image; in the second step, we map the concatenation of the identity vector and attribute vector onto the potential internal space of StyleGAN. Extensive experiments on the test dataset show that the results of the proposed method are not only superior in clarity and authenticity than other state-of-the-art methods but also sufficiently integrate identity and attribute.
Highlights
As one of the main contents of deepfake, face swapping declares to the world today that seeing is not always believing
We propose a many-to-many face-swapping method based on the pretrained StyleGAN model (Karras et al, 2019), which strives to ensure the clarity and fidelity of the results while fusing identity and attribute information
We creatively combine the advantages of the above two fields to improve the performance of face swapping, and make possible the more complex control of generative adversarial networks (GANs)’s potential space
Summary
As one of the main contents of deepfake, face swapping declares to the world today that seeing is not always believing. Face swapping refers to transferring the identity of a source image to the face of another target image while keeping unchanged the illumination, head posture, expression, dress, background, and other attribute information of the target image. Face swapping has received widespread attention since its birth, catering to the affluent needs of social life, such as hairstyle simulation, film and television shooting, privacy protection, and so on (Ross and Othman, 2010). Face swapping is accompanied by its interesting and operational application prospects and by various challenges between reality and vision. The early face-swapping methods (Bitouk et al, 2008; Korshunova et al, 2017) require many images of source and target characters to provide sufficient facial information.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have