TransFS: Face Swapping Using Transformer

Abstract

This paper proposes a Transformer based face swapping model, namely, TransFS. The proposed model mainly solves two current problems of face swapping: 1) the face swapping result does not fully preserve pose and expression of the target face as expected; 2) most of the existing models fail to accomplish high-quality face swapping on high-resolution images. To address these two challenges, we first propose a Cross- Window Face Encoder based on Swin Transformer that learns rich facial features including poses and expressions. Then, we devise an Identity Generator to reconstruct high-resolution images of specific identity with high quality while utilizing the Transformer attention mechanism to increase identity information retention. Finally, a Face Conversion Module is proposed to transform the source identity reconstructed image into the target face image to synthesize the final face swapping result while maintaining the details of pose and expression of the target face. Through extensive experiments, our method not only accomplishes face swapping for low-resolution images with arbitrary identities, but also accomplishes face swapping for high-resolution images. Furthermore, our method achieves the state-of-the-art performance in pose and expression controls compared to other methods.