Detecting deepfake videos based on spatiotemporal attention and convolutional LSTM

Abstract

Fake face detection is in dilemma with the rapid development of face manipulation technology. One way to improve the effectiveness of detector is to make full use of intra and inter frame information. In this paper, a novel Xception-LSTM algorithm is proposed by using our new spatiotemporal attention mechanism and convolutional long short-term memory (ConvLSTM). In the algorithm, the spatiotemporal attention mechanism, including spatial and temporal attention mechanism, is proposed to capture and enhance spatiotemporal correlations before dimension reduction of Xception. Thereafter, the ConvLSTM is introduced to consider frame structure information while modeling temporal information. The experimental results on three widely used datasets demonstrate that the proposed algorithms perform better than the state-of-the-art algorithms. In addition, the effectiveness of the spatiotemporal attention mechanism and ConvLSTM are illustrated in ablation experiments.