FDML: Feature Disentangling and Multi-view Learning for face forgery detection

Abstract

Recent advances in realistic facial manipulation techniques have led to a growing interest in forgery detection due to security concerns. The presence of source-dependent information in both the forged images and the learned representations inevitably confuses the detector. To alleviate this issue, we present a Feature Disentangling and Multi-view Learning (FDML) framework to distill forgery-relevant intrinsic features from entangled information in a progressive manner, i.e., from image-level to feature-level. Towards image-level, the input image is first transformed into two complementary views, one using learnable filters to adaptively mine subtle frequency-aware clues, and the other using a novel data augmentation operation called SceneMix to weaken source-specific factors. The intermediate features output from these two branches are fully integrated through a trainable two-branch Hybrid Attention Module (HAM), guiding the effective performance of fused features. For feature-level, to automatically separate forgery-relevant features from source-relevant features and reduce the interference of irrelevant factors in decision making, two feature disentangling schemes are proposed, and finally only forgery-relevant features are used for prediction, which greatly improves the detection performance. Extensive experiments show that our framework achieves competitive performances compared with state-of-the-art works.