A Novel Counterfeit Feature Extraction Technique for Exposing Face-Swap Images Based on Deep Learning and Error Level Analysis.

Yuxing Li,Weiguo Zhang,Chenggang Zhao

doi:10.3390/e22020249

Abstract

The quality and efficiency of generating face-swap images have been markedly strengthened by deep learning. For instance, the face-swap manipulations by DeepFake are so real that it is tricky to distinguish authenticity through automatic or manual detection. To augment the efficiency of distinguishing face-swap images generated by DeepFake from real facial ones, a novel counterfeit feature extraction technique was developed based on deep learning and error level analysis (ELA). It is related to entropy and information theory such as cross-entropy loss function in the final softmax layer. The DeepFake algorithm is only able to generate limited resolutions. Therefore, this algorithm results in two different image compression ratios between the fake face area as the foreground and the original area as the background, which would leave distinctive counterfeit traces. Through the ELA method, we can detect whether there are different image compression ratios. Convolution neural network (CNN), one of the representative technologies of deep learning, can extract the counterfeit feature and detect whether images are fake. Experiments show that the training efficiency of the CNN model can be significantly improved by the ELA method. In addition, the proposed technique can accurately extract the counterfeit feature, and therefore achieves outperformance in simplicity and efficiency compared with direct detection methods. Specifically, without loss of accuracy, the amount of computation can be significantly reduced (where the required floating-point computing power is reduced by more than 90%).

Highlights

Nowadays, with the popularization of smartphones and various face-swap applications, the manipulation of visual content is becoming increasingly common, which has been one of the most heated issues in the digital society
We proposed a novel counterfeit feature extraction technique to expose face-swap images based on the deep learning and error level analysis (ELA), which can effectively distinguish facial images generated by deep learning
Coupled with the experiments showing that the training efficiency of the Convolution neural network (CNN) model can be significantly improved by using the ELA method, this indicates that our method is feasible in simplicity and efficiency

Summary

Introduction

With the popularization of smartphones and various face-swap applications, the manipulation of visual content is becoming increasingly common, which has been one of the most heated issues in the digital society. The core problem, more importantly, lies in the new generation of generative deep neural networks [1], which are capable of synthesizing videos from a large volume of training data with minimum manual editing. DeepFake replaces the face in an original video with the face of another person using generative adversarial networks (GANs) [3]. As the GAN models were trained with tens of thousands of images, it is more likely to generate realistic faces that can be spliced into the original video more precisely. With the advancement of deep learning, face-swap technology has been applied in numerous scenes including privacy protection, video synthesis as well as other innovative applications. Utilizing DeepFake or other technologies, one GPU and a large number

Objectives

Methods

Findings

Conclusion