A dual descriptor combined with frequency domain reconstruction learning for face forgery detection in deepfake videos

Abstract

Conventional face forgery detectors have primarily relied on image artifacts produced by deepfake video generation models. These methods have performed well when the training and test sets were derived from the same deepfake algorithm, but accuracy and generalizability remain a challenge for diverse datasets. In this study, both supervised and unsupervised approaches are proposed for more accurate detection in in-domain and cross-domain experiments. Specifically, two descriptors are introduced to extract rich information in the spatial domain to achieve higher accuracy. A frequency domain reconstruction module is then included to expand the representation space for facial features. A reconstruction method based on an auto-encoder was also applied to obtain a frequency domain coding vector. In this process, reconstruction learning was sufficient for extracting unknown information, while a combination with classification learning provided essential high-frequency pixel differences between real and fake samples, thus facilitating forgery identification. A series of validation experiments with large-scale benchmark datasets demonstrated that the proposed technique was superior to existing methods.