Contrastive learning-based general Deepfake detection with multi-scale RGB frequency clues

Abstract

Deepfake is a type of image and video face manipulation methods which could cause security and society threats. Although some related databases and detection models have been proposed for detecting face forgery media, achieving a generalizable detector for both known and unknown manipulations remains challenging. In this study, a novel deepfake detection model with high generalizability is proposed to tackle this issue. We employ supervised contrastive learning to enhance the generalizability to unknown manipulations and datasets. In addition, we design a cross-modality data augmentation method by combining SRM and RGB features to extract detection clues comprehensively. Furthermore, we propose a multi-scale feature enhancement module to enhance textural and semantic information. Extensive experiments have demonstrated that our method improves model generalization in both intra- and cross- dataset scenarios.