Deepfake detection via inter-frame inconsistency recomposition and enhancement

Abstract

Due to the remarkable progress in face manipulation technology, malicious applications of these technologies may pose a great threat to the social stability. Therefore, it is essential to carry out the research of deepfake detection. In this paper, we assumed that the illumination on frames that skip a certain space is basically consistent in real videos, but tends to be inconsistent in fake videos. From this point, a network which contains a learnable Image Decomposition Module (IDM) and multi-level feature enhancement is proposed. IDM decomposes frames into illumination and reflection, and frame recomposition is followed to highlight the frame-level illumination inconsistency. Multi-level feature enhancement is proposed to enhance the illumination inconsistency at feature level. In addition, considering the computational complexity and human vision perception mechanism, we train the network in logarithm domain. Experimental results show that the proposed method is effective and superior compared with other state-of-the-art deepfake detection methods on mainstream deepfake datasets.