Patch-DFD: Patch-based end-to-end DeepFake discriminator

Abstract

Facial forgery by DeepFake has recently attracted more public attention. Face image contains sensitive personal information, abuse of such technology will grow into a menace. Since the difference between real and fake faces is usually subtle and local, the general detection framework of applying the backbone network to capture the global features of the entire face and then feeding it into the binary classifier is not optimal. In addition, patch-based schemes are widely used in various computer vision tasks, including image classification. However, how to extract features for location-specific and arbitrary-shaped patches while preserving their original information and spoof patterns as much as possible requires further exploration. In this paper, a novel deep forgery detector called Patch-DFD is proposed, which applies a patch-based solution of Facial Patch Mapping (FPM) to obtain several part-based feature maps, preserving original details of each facial patch to the greatest extent. Besides, the BM-pooling module aims to fix the size of the feature maps while reducing quantization errors. The local voting strategy is finally used to fuse the results of parts detectors, so as to more accurately identify the fake faces generated by deep generative models. Compared to typical patch-wise framework that takes patch inputs, our scheme is more efficient due to the absence of repeated convolution operations. Moreover, extensive experiments conducted on publicly available face forensics datasets have proved that the effectiveness of our framework.