Three-classification face manipulation detection using attention-based feature decomposition

Abstract

Face manipulation detection has become a recent research hotpot, and many detection methods have been proposed. Most existing detection methods treat face manipulation detection as a vanilla binary classification problem. However, according to the visual effects of facial manipulation, face manipulation approaches are divided into face replacement and face reenactment. In this study, we propose a three-classification face manipulation detection method(TFMD). To implement three-classification face manipulation detection, we introduce a new face forgery feature representation, where the face forgery features are jointly represented by the identity-changing features and the face real-fake features. To decompose the face forgery features into the identity-changing features and the face real-fake features, we design an attention-based feature decomposition module (AFDM). Moreover, we extract high-frequency noise features from the shallow features of the RGB stream to enrich high-frequency information. Through extensive experiments on two datasets, FaceForensics++ and Celeb-DF, we achieve state-of-the-art performance and demonstrate the effectiveness and robustness of our proposed method.