Deepfake Videos Research Articles

Malware Detection and Classification using Generative Adversarial Network

The Generative Adversarial Networks (GANs) are playing a crucial role in deep-learning-based malware classification to overcome the dataset imbalance and unseen malware. The Generative AI is preferably used in many applications, such as improving image resolution and generating audio, video, and text. The cybercriminals are also using the Generative AI for generating the malware and deepfake videos to harm the targeted person or device. By generating the synthetic data, it makes the deep learning model more robust to detect such types of unseen and adversarial attacks. This work utilizes GANs for generating adversarial malware samples to train a classification and detection model, improving the model’s ability to identify sophisticated malware variants. The performance of the proposed Conditional Generative Adversarial Network (CGAN) model is evaluated on a multiclass malware grayscale image dataset and a binary class malware RGB image dataset. The performance of proposed model is compared with current state-of-the-art. Results indicate a significant improvement in classification accuracy and a reduction in training time and false positives, showcasing GAN’s potential in the dynamic cybersecurity landscape.

AVENUE: A novel deep fake detection method based on temporal convolution network & rPPG information

In Deep Learning (DL), an adversary creates Deepfakes by manipulating facial features to fool someone. The Deepfakes pose a security threat to anyone's privacy and a primary concern for our society. It can be detected by utilizing the texture and physiological properties of the face, like eye and lip movements; however, such methods are incompetent when Deepfakes are created using recent generative adversarial networks (GAN). Alternatively, remote Photoplethysmography (rPPG) information can be used for Deepfake detection because GANs neglect human physiological information for Deepfake generation. Such detection can be inaccurate when rPPG signals are affected by the noises induced by facial deformation and illumination variations. Furthermore, the exiting Deepfake detections are usually performed using sequential models, and such models fail to process the long sequence of temporal information. These issues are mitigated by our proposed method AVENUE , that is, \(A\) no \(V\) el d \(E\) epfake detectio \(N\) method based on temporal convol \(U\) tion n \(E\) twork & rPPG information. For mitigating the noise issues in the rPPG signals, the proposed method detects and employs relatively stable clips of the input video for Deepfake detection. The stable clips are those clips that are least affected by facial deformations. Also, we use a modified Temporal convolution network to model the long sequence of Deepfake information rather than the sequential architectures. We performed the experimental result on publically available datasets of Deepfake videos. It demonstrates that our proposed method performs better than the existing rPPG-based Deepfake detection methods.

GCS-YOLOv8: A Lightweight Face Extractor to Assist Deepfake Detection.

To address the issues of target feature blurring and increased false detections caused by high compression rates in deepfake videos, as well as the high computational resource requirements of existing face extractors, we propose a lightweight face extractor to assist deepfake detection, GCS-YOLOv8. Firstly, we employ the HGStem module for initial downsampling to address the issue of false detections of small non-face objects in deepfake videos, thereby improving detection accuracy. Secondly, we introduce the C2f-GDConv module to mitigate the low-FLOPs pitfall while reducing the model's parameters, thereby lightening the network. Additionally, we add a new P6 large target detection layer to expand the receptive field and capture multi-scale features, solving the problem of detecting large-scale faces in low-compression deepfake videos. We also design a cross-scale feature fusion module called CCFG (CNN-based Cross-Scale Feature Fusion with GDConv), which integrates features from different scales to enhance the model's adaptability to scale variations while reducing network parameters, addressing the high computational resource requirements of traditional face extractors. Furthermore, we improve the detection head by utilizing group normalization and shared convolution, simplifying the process of face detection while maintaining detection performance. The training dataset was also refined by removing low-accuracy and low-resolution labels, which reduced the false detection rate. Experimental results demonstrate that, compared to YOLOv8, this face extractor achieves the AP of 0.942, 0.927, and 0.812 on the WiderFace dataset's Easy, Medium, and Hard subsets, representing improvements of 1.1%, 1.3%, and 3.7% respectively. The model's parameters and FLOPs are only 1.68 MB and 3.5 G, reflecting reductions of 44.2% and 56.8%, making it more effective and lightweight in extracting faces from deepfake videos.

Artificial Intelligence and Political Deepfakes: Shaping Citizen Perceptions Through Misinformation

In the post-truth age, political conspiracies circulate rapidly on social media, cultivating false narratives, while challenging the public’s ability to distinguish truth from fiction. ‘Deepfakes’ represent the most recent type of misinformation. They display deceitful representations of events to lead audiences to believe in fabricated realities. There has been limited research on deepfakes in political communications. As this technology progresses, deepfakes look deceptively authentic; thus, it is necessary to explore their effects on public perceptions. This study examines viewers’ comments on an Instagram-published deepfake video of Hillary Clinton to understand the impact of this technology. The results demonstrate that individuals struggle to identify deepfake videos and that their opinions are affected by this persuasive type of misinformation. This study also explores different ethical concerns posed by political deepfakes. By offering insights into public reactions to manipulated content, this study contributes to our understanding of the political effects of AI-fabricated content.

Deep fake Video Face Recognition Using Supervised Contrastive Learning for Scalability and Interpretability

Deep fake Video Face Recognition Using Supervised Contrastive Learning for Scalability and Interpretability

Mining Generalized Multi-timescale Inconsistency for Detecting Deepfake Videos

Mining Generalized Multi-timescale Inconsistency for Detecting Deepfake Videos

Possible Health Benefits and Risks of DeepFake Videos: A Qualitative Study in Nursing Students.

"DeepFakes" are synthetic performances created by AI, using neural networks to exchange faces in images and modify voices. Due to the novelty and limited literature on its risks/benefits, this paper aims to determine how young nursing students perceive DeepFake technology, its ethical implications, and its potential benefits in nursing. This qualitative study used thematic content analysis (the Braun and Clarke method) with videos recorded by 50 third-year nursing students, who answered three questions about DeepFake technology. The data were analyzed using ATLAS.ti (version 22), and the project was approved by the Ethics Committee (code UCV/2021-2022/116). Data analysis identified 21 descriptive codes, classified into four main themes: advantages, disadvantages, health applications, and ethical dilemmas. Benefits noted by students include use in diagnosis, patient accompaniment, training, and learning. Perceived risks include cyberbullying, loss of identity, and negative psychological impacts from unreal memories. Nursing students see both pros and cons in DeepFake technology and are aware of the ethical dilemmas it poses. They also identified promising healthcare applications that could enhance nurses' leadership in digital health, stressing the importance of regulation and education to fully leverage its potential.

MeST-Former: Motion-enhanced Spatiotemporal Transformer for generalizable Deepfake detection

The rise of Deepfake technology has sparked significant concerns due to its potential for misuse and malicious manipulation of multimedia content. Various detection approaches aimed at detecting Deepfake videos have been proposed, mostly relying on the identification of spatial and temporal artifacts. However, due to the different contexts of source images and the variety of generation techniques, current Deepfake detection methods usually perform well on training datasets, and yet generalize poorly to those unseen identities in new datasets. This issue is widely known as the generalization challenge of Deepfake detection. To address this challenge, this paper proposes an advanced spatiotemporal Deepfake video detector, named Motion-enhanced Spatiotemporal Transformer (MeST-Former). MeST-Former is based on the spatiotemporal modeling capacity of the video Swin Transformer. The spatial and temporal features are obtained from the RGB and motion images, respectively. To enhance the generalization ability of MeST-Former to unseen identities in unseen datasets, the ID-related components in the spatial and temporal features are detached. Specifically, MeST-Former adopts the newly proposed Identity-Decoupling Attention (IDC-Att) module to disentangle the ID-related and ID-unrelated components. Only the ID-unrelated components are used to construct more generalizable spatiotemporal representations. This process makes the constructed spatiotemporal features identity-agnostic and more generalizable to unseen identities. We conducted extensive experiments to evaluate the performance of the MeST-Former. Our results indicate that MeST-Former achieves accurate and generalizable Deepfake detection performance. Notably, MeST-Former also demonstrates high efficacy in detecting AI-animated talking-head videos.

Local Region Frequency Guided Dynamic Inconsistency Network for Deepfake Video Detection

Local Region Frequency Guided Dynamic Inconsistency Network for Deepfake Video Detection

Combating deepfakes: a comprehensive multilayer deepfake video detection framework

AbstractDeepfakes represent a class of synthetic media crafted with the aid of advanced deep learning techniques that exhibit an unparalleled degree of authenticity. The rapid advancement in Artificial Intelligence (AI) has contributed to an increase in the prevalence of deepfakes on the internet, consequently amplifying the spread of misinformation among the public. Consequently, the detection of deepfakes has become a pressing concern. In this context, we put forth a comprehensive framework for deepfake video detection, which is built upon three distinct layers. The first layer, termed as the RGB features extraction layer, is designed to identify potential signs of forgery within the spatial domain of analogous video frames. The second layer, known as the GAN features extraction layer, focuses on the extraction of forgery fingerprints in the high-frequency region. This layer is specifically engineered to detect the fingerprints left by the Generative Adversarial Network (GAN) process in fake videos and the traces of the imaging process in genuine videos. The third and final layer, referred to as the facial region intra-frame inconsistency feature extraction layer, is dedicated to uncovering the anomalies associated with the manipulation process. This is achieved by extracting features from both the inner and outer regions of the manipulated portion of a frame. The extensive experimental evaluations have underscored the superior performance of proposed approach in comparison to existing state-of-the-art methods.

Development and Validation of the Perceived Deepfake Trustworthiness Questionnaire (PDTQ) in Three Languages

Exposure to false information is becoming a common occurrence in our daily lives. New developments in artificial intelligence are now used to produce increasingly sophisticated multimedia false content, such as deepfakes, making false information even more challenging to detect and combat. This creates expansive opportunities to mislead individuals into believing fabricated claims and negatively influence their attitudes and behavior. Therefore, a better understanding of how individuals perceive such content and the variables related to the perceived trustworthiness of deepfakes is needed. In the present study, we developed and validated the Perceived Deepfake Trustworthiness Questionnaire (PDTQ) in English, Italian, and Slovene. This was done in three phases. First, we developed the initial pool of items by reviewing previous studies, generating items via interviews and surveys, and employing artificial intelligence. Second, we shortened and adapted the questionnaire according to experts’ evaluation of content validity and translated the questionnaire into Italian and Slovene. Lastly, we evaluated the psychometric characteristics via a cross-sectional study in three languages (N = 733). The exploratory factor analyses suggested a two-factor solution, with the first factor measuring the perceived trustworthiness of the content and the second measuring the perceived trustworthiness of the presentation. This factorial structure was replicated in confirmatory factor analyses. Moreover, our analyses provided support for PDTQ’s reliability, measurement invariance across all three languages, and its construct and incremental validity. As such, the PDTQ is a reliable, measurement invariant, and valid tool for comprehensive exploration of individuals’ perception of deepfake videos.

Multi-domain awareness for compressed deepfake videos detection over social networks guided by common mechanisms between artifacts

The viral spread of massive deepfake videos over social networks has caused serious security problems. Despite the remarkable advancements achieved by existing deepfake detection algorithms, deepfake videos over social networks are inevitably influenced by compression factors. This causes deepfake detection performance to be limited by the following challenging issues: (a) interfering with compression artifacts, (b) loss of feature information, and (c) aliasing of feature distributions. In this paper, we analyze the common mechanism between compression artifacts and deepfake artifacts, revealing the structural similarity between them and providing a reliable theoretical basis for enhancing the robustness of deepfake detection models against compression. Firstly, based on the common mechanism between artifacts, we design a frequency domain adaptive notch filter to eliminate the interference of compression artifacts on specific frequency bands. Secondly, to reduce the sensitivity of deepfake detection models to unknown noise, we propose a spatial residual denoising strategy. Thirdly, to exploit the intrinsic correlation between feature vectors in the frequency domain branch and the spatial domain branch, we enhance deepfake features using an attention-based feature fusion method. Finally, we adopt a multi-task decision approach to enhance the discriminative power of the latent space representation of deepfakes, achieving deepfake detection with robustness against compression. Extensive experiments show that compared with the baseline methods, the detection performance of the proposed algorithm on compressed deepfake videos has been significantly improved. In particular, our model is resistant to various types of noise disturbances and can be easily combined with baseline detection models to improve their robustness.

Fully Unsupervised Deepfake Video Detection Via Enhanced Contrastive Learning.

Nowadays, Deepfake videos are widely spread over the Internet, which severely impairs the public trustworthiness and social security. Although more and more reliable detectors have recently sprung up for resisting against that new-emerging tampering technique, some challengeable issues still need to be addressed, such that most of Deepfake video detectors under the framework of the supervised mechanism require a large scale of samples with accurate labels for training. When the amount of the training samples with the true labels are not enough or the training data are maliciously poisoned by adversaries, the supervised classifier is probably not reliable for detection. To tackle that tough issue, it is proposed to design a fully unsupervised Deepfake detector. In particular, in the whole procedure of training or testing, we have no idea of any information about the true labels of samples. First, we novelly design a pseudo-label generator for labeling the training samples, where the traditional hand-crafted features are used to characterize both types of samples. Second, the training samples with the pseudo-labels are fed into the proposed enhanced contrastive learner, in which the discriminative features are further extracted and continually refined by iteration on the guidance of the contrastive loss. Last, relying on the inter-frame correlation, we complete the final binary classification between real and fake videos. A large scale of experimental results empirically verify the effectiveness of our proposed unsupervised Deepfake detector on the benchmark datasets including FF++, Celeb-DF, DFD, DFDC, and UADFV. Furthermore, our proposed well-performed detector is superior to the current unsupervised method, and comparable to the baseline supervised methods. More importantly, when facing the problem of the labeled data poisoned by malicious adversaries or insufficient data for training, our proposed unsupervised Deepfake detector performs its powerful superiority.

Predicting manipulated regions in deepfake videos using convolutional vision transformers

Deepfake technology, which uses artificial intelligence to create and manipulate realistic synthetic media, poses a serious threat to the trustworthiness and integrity of digital content. Deepfakes can be used to generate, swap, or modify faces in videos, altering the appearance, identity, or expression of individuals. This study presents an approach for deepfake detection, based on a convolutional vision transformer (CViT), a hybrid model that combines convolutional neural networks (CNNs) and vision transformers (ViTs). The proposed study uses a 20-layer CNN to extract learnable features from face images, and a ViT to classify them into real or fake categories. The study also employs MTCNN, a multi-task cascaded network, to detect and align faces in videos, improving the accuracy and efficiency of the face extraction process. The method is assessed using the FaceForensics++ dataset, which comprises 15,800 images sourced from 1600 videos. With an 80:10:10 split ratio, the experimental results show that the proposed method achieves an accuracy of 92.5% and an AUC of 0.91. We use Gradient-Weighted Class Activation Mapping (Grad-CAM) visualization that highlights distinctive image regions used for making a decision. The proposed method demonstrates a high capability of detecting and distinguishing between genuine and manipulated videos, contributing to the enhancement of media authenticity and security.

Detection of Deepfake Videos Using Long-Distance Attention.

With the rapid progress of deepfake techniques in recent years, facial video forgery can generate highly deceptive video content and bring severe security threats. And detection of such forgery videos is much more urgent and challenging. Most existing detection methods treat the problem as a vanilla binary classification problem. In this article, the problem is treated as a special fine-grained classification problem since the differences between fake and real faces are very subtle. It is observed that most existing face forgery methods left some common artifacts in the spatial domain and time domain, including generative defects in the spatial domain and interframe inconsistencies in the time domain. And a spatial-temporal model is proposed which has two components for capturing spatial and temporal forgery traces from a global perspective, respectively. The two components are designed using a novel long-distance attention mechanism. One component of the spatial domain is used to capture artifacts in a single frame, and the other component of the time domain is used to capture artifacts in consecutive frames. They generate attention maps in the form of patches. The attention method has a broader vision which contributes to better assembling global information and extracting local statistic information. Finally, the attention maps are used to guide the network to focus on pivotal parts of the face, just like other fine-grained classification methods. The experimental results on different public datasets demonstrate that the proposed method achieves state-of-the-art performance, and the proposed long-distance attention method can effectively capture pivotal parts for face forgery.

An Approach to Deepfake Video Detection Based on ACO-PSO Features and Deep Learning

The rapid advancement of deepfake technology presents significant challenges in detecting highly convincing fake videos, posing risks such as misinformation, identity theft, and privacy violations. In response, this paper proposes an innovative approach to deepfake video detection by integrating features derived from ant colony optimization–particle swarm optimization (ACO-PSO) and deep learning techniques. The proposed methodology leverages ACO-PSO features and deep learning models to enhance detection accuracy and robustness. Features from ACO-PSO are extracted from the spatial and temporal characteristics of video frames, capturing subtle patterns indicative of deepfake manipulation. These features are then used to train a deep learning classifier to automatically distinguish between authentic and deepfake videos. Extensive experiments using comparative datasets demonstrate the superiority of the proposed method in terms of detection accuracy, robustness to manipulation techniques, and generalization to unseen data. The computational efficiency of the approach is also analyzed, highlighting its practical feasibility for real-time applications. The findings revealed that the proposed method achieved an accuracy of 98.91% and an F1 score of 99.12%, indicating remarkable success in deepfake detection. The integration of ACO-PSO features and deep learning enables comprehensive analysis, bolstering precision and resilience in detecting deepfake content. This approach addresses the challenges involved in facial forgery detection and contributes to safeguarding digital media integrity amid misinformation and manipulation.

Deepfake Video Detection Using Facial Feature Points and Ch-Transformer

With the development of Metaverse technology, the avatar in Metaverse has faced serious security and privacy concerns. Analyzing facial features to distinguish between genuine and manipulated facial videos holds significant research importance for ensuring the authenticity of characters in the virtual world and for mitigating discrimination, as well as preventing malicious use of facial data. To address this issue, the Facial Feature Points and Ch-Transformer (FFP-ChT) deepfake video detection model is designed based on the clues of different facial feature points distribution in real and fake videos and different displacement distances of real and fake facial feature points between frames. The face video input is first detected by the BlazeFace model, and the face detection results are fed into the FaceMesh model to extract 468 facial feature points. Then the Lucas-Kanade (LK) optical flow method is used to track the points of the face, the face calibration algorithm is introduced to re-calibrate the facial feature points, and the jitter displacement is calculated by tracking the facial feature points between frames. Finally, the Class-head (Ch) is designed in the transformer, and the facial feature points and facial feature point displacement are jointly classified through the Ch-Transformer model. In this way, the designed Ch-Transformer classifier is able to accurately and effectively identify deepfake videos. Experiments on open datasets clearly demonstrate the effectiveness and generalization capabilities of our approach.

Deepfake Video Detection Using Res-Next CNN and LSTM

Abstract: Deepfake videos are becoming more and more common in the digital age, which has led to major worries about how they can undermine the trustworthiness of visual media. With deep learning algorithms' rising processing power, producing lifelike human-synthesized films or deepfakes has never been easier. Disinformation and political unrest can be caused by these videos. A novel deep learning-based method has been created to distinguish between actual and AI-generated fraudulent videos to address this problem. The suggested technique uses Attention-based networks (Res-Next CNN), a kind of deep learning architecture that can selectively focus on significant features in a video, to fine-tune the transformer module to search for new sets of feature space to detect false images.

Video Detection Method Based on Temporal and Spatial Foundations for Accurate Verification of Authenticity

With the rapid development of deepfake technology, it is finding applications in virtual movie production and entertainment. However, its potential for malicious use, such as generating false information, fake news, or synthetic pornography, poses significant threats to national and social security. Various research disciplines are actively engaged in developing deepfake video detection technologies to mitigate the risks associated with malicious deepfake content. Therefore, the importance of deepfake video detection technology cannot be overemphasized. This study addresses the challenge posed by images in nonexistent datasets by analyzing deepfake video detection methods. Using temporal and spatial detection techniques and employing 68 facial landmarks for alignment and feature extraction, this research integrates the attention-guided data augmentation (AGDA) strategy to enhance generalization capabilities. The detection performance is evaluated on four datasets: UADFV, FaceForensics++, Celeb-DF, and DFDC, with superior results compared to alternative approaches. To evaluate the study’s ability to accurately discriminate authenticity, detection experiments are conducted on both genuine and deepfake videos synthesized using the DeepFaceLab and FakeApp frameworks. The experimental results show better performance in detecting deepfake videos than other methods compared.

Deepfake video detection: challenges and opportunities

Deepfake videos are a growing social issue. These videos are manipulated by artificial intelligence (AI) techniques (especially deep learning), an emerging societal issue. Malicious individuals misuse deepfake technologies to spread false information, such as fake images, videos, and audio. The development of convincing fake content threatens politics, security, and privacy. The majority of deepfake video detection methods are data-driven. This survey paper aims to thoroughly analyse deepfake video generation and detection. The paper’s main contribution is the classification of the many challenges encountered while detecting deepfake videos. The paper discusses data challenges such as unbalanced datasets and inadequate labelled training data. Training challenges include the need for many computational resources. It also addresses reliability challenges, including overconfidence in detection methods and emerging manipulation approaches. The research emphasises the dominance of deep learning-based methods in detecting deepfakes despite their computational efficiency and generalisation limitations. However, it also acknowledges the drawbacks of these approaches, such as their limited computing efficiency and generalisation. The research also critically evaluates deepfake datasets, emphasising the necessity for good-quality datasets to improve detection methods. The study also indicates major research gaps, guiding future deepfake detection research. This entails developing robust models for real-time detection.