DeepFake Detection with Remote Heart Rate Estimation Using 3D Central Difference Convolution Attention Network

Abstract

Objective: As GAN-based deepfakes have become increasingly mature and realistic, the demand for effective deepfake detectors has become essential. We are inspired by the fact that normal pulse rhythms present in real-face video can be decreased or even completely interrupted in a deepfake video; thus, we have introduced a new deepfake detection approach based on remote heart rate estimation using the 3D Central Difference Convolution Attention Network (CDCAN). Methods: Our proposed fake detector is mainly composed of a 3D CDCAN with an inverse attention mechanism and LSTM architecture. It utilizes 3D central difference convolution to enhance the spatiotemporal representation, which can capture rich physiological-related temporal context by gathering the time difference information. The soft attention mechanism is to focus on the skin region of interest, while the inverse attention mechanism is to further denoise rPPG signals. Results: The performance of our approach is evaluated on the two latest Celeb-DF and DFDC datasets, for which the experiment results show that our proposed approach achieves an accuracy of 99.5% and 97.4%, respectively. Conclusion: Our approach outperforms the state-of-art methods and proves the effectiveness of our DeepFake detector.