Safeguarding media integrity: A hybrid optimized deep feature fusion based deepfake detection in videos

Abstract

The surge in deepfake techniques allows even non-experts to create deceptive videos using only source and target images, posing risks such as impersonation, character defamation, and spreading misinformation on various platforms. Existing methods often prove insufficient, particularly when faced with videos generated through diverse deepfake techniques that feature variations in lighting conditions and ethnicities. This paper introduces a novel approach called Hybrid Optimized Deep Feature Fusion-based Deepfake Detection (HODFF-DD) in videos, employing a spotted hyena optimizer. The HODFF-DD framework is designed to be resilient across ethnicities and varying lighting conditions, capable of detecting deepfake videos created by different techniques. The architecture comprises two integral components: a customized model incorporating InceptionResNetV1 and InceptionResNetV2, and a Bidirectional Long-Short Term Memory (BiLSTM). In this framework, faces extracted from videos undergo feature extraction at the frame level using our custom model comprising InceptionResNetV1 and InceptionResNetV2. The resulting sequences of features are then utilized to train a temporally aware BiLSTM for binary classification between real and fake videos. To optimize network weights, a bio-inspired spotted hyena optimizer is applied during the training process. Evaluation on diverse datasets, including the Kaggle dataset, FaceForensics++ (FF++) encompassing videos manipulated using various techniques (DeepFakes, FaceSwap, Face2Face, FaceShifter, and NeuralTextures), and the FakeAVCeleb dataset, demonstrates the effectiveness of our presented approach. Our method achieves an accuracy exceeding 90 % on subsets like DeepFakes, FaceSwap, and Face2Face.