Deepfake Videos Research Articles

Face Corrupt Identification Using LSTM And ResNext

People will believe in what they see rather than hidden facts. In recent times as technology is improving and it became very easy to change a particular person in an existing image or video with someone else. So, spreading these types of modified videos causes spamming and peculating wrong information over social media and this will cause the great extent of misleading and threatening to the common people. To overcome these tedious situations, we are building a deep learning model that will identify the genuinity of the video or image in terms of the origin of its source. In our proposed system we are using Long Short-Term Memory and ResNext. Key Words: ResNext, Convolution neural network, Recurrent Neural Network (RNN), Long Short- Term Memory (LSTM), Computer vision, Deepfake Video Detection, Pre Processing, Deep Learning, Image Processing, Classification.

How to go from storytelling to fact telling and still sell products, ideas and hope

We exist now in what can sometimes feel like an alternative universe, where fake news headlines go viral, algorithmic bias targets us with content that is popular and engaging but not necessarily trustworthy, chatbots are powered by artificial intelligence and deepfake video technology runs amok. The global rise of misinformation and the decline of consumer trust in both government and media have coincided with technological advances and tarnished demand for brand strategists’ old tricks. One of these is storytelling. Now, if you are a brand that is being even a little disingenuous with your communications today, then to some you may as well be a bald-faced liar. Instead, regardless of the industry, whether B2B or consumer, brands today should be in the business of fact telling. At stake are a business’ reputation, customer loyalty and the bottom line. Furthermore, it is up to us to help ensure that the information environment does not deteriorate further on our watch. The most exciting thing a brand can do today is to be audaciously authentic. But have we forgotten how to be? Moreover, can we tell the truth and still get consumers to add our products to their shopping carts, become brand advocates and loyalists and even learn to trust again? Within, we outline a framework for brands to meaningfully differentiate with truth telling.

Deepfake detection based on cross-domain local characteristic analysis with multi-domain transformer

Deepfake videos present a significant challenge in the current media landscape. While current deepfake detection methods demonstrate satisfactory performance, there is still room for improvement in their ability to generalize and detect unseen scenarios, particularly those involving imperceptible cues. This paper introduces a novel multi-modal deepfake detection model named SpectraVisionFusion Transformer (SVFT), which incorporates spatial and frequency domain statistical artifacts to improve generalization performance. The SVFT framework uses two different backbone encoder models to take advantage of both spatial and frequency domain cues in video sequences, along with a decoder and classifier, for common cross-attention and classification, respectively. The spatial domain branch uses a convolutional transformer-based encoder to analyze facial visual features. In contrast, the frequency domain branch employs a language transformer encoder. Additionally, we introduce a weighted feature embedding fusion mechanism that integrates spectral-based statistical feature embeddings and visual cues to achieve a more comprehensive and balanced spatial-frequency feature representation. By coordinately analyzing these modalities, our model exhibits improved detection and generalization capabilities in unseen scenarios. Our proposed SVFT model achieved 92.57% and 80.63% accuracy in extensive cross-manipulation and dataset evaluation, respectively, while surpassing the performance of traditional and single-domain-based approaches.

Video deepfake detection using Particle Swarm Optimization improved deep neural networks

As complexity and capabilities of Artificial Intelligence technologies increase, so does its potential for misuse. Deepfake videos are an example. They are created with generative models which produce media that replicates the voices and faces of real people. Deepfake videos may be entertaining, but they may also put privacy and security at risk. A criminal may forge a video of a politician or another notable person in order to affect public opinions or deceive others. Approaches for detecting and protecting against these types of forgery must evolve as well as the methods of generation to ensure that proper information is supplied and to mitigate the risks associated with the fast evolution of deepfakes. This research exploits the effectiveness of deepfake detection algorithms with the application of a Particle Swarm Optimization (PSO) variant for hyperparameter selection. Since Convolutional Neural Networks excel in recognizing objects and patterns in visual data while Recurrent Neural Networks are proficient at handling sequential data, in this research, we propose a hybrid EfficientNet-Gated Recurrent Unit (GRU) network as well as EfficientNet-B0-based transfer learning for video forgery classification. A new PSO algorithm is proposed for hyperparameter search, which incorporates composite leaders and reinforcement learning-based search strategy allocation to mitigate premature convergence. To assess whether an image or a video is manipulated, both models are trained on datasets containing deepfake and genuine photographs and videos. The empirical results indicate that the proposed PSO-based EfficientNet-GRU and EfficientNet-B0 networks outperform the counterparts with manual and optimal learning configurations yielded by other search methods for several deepfake datasets.

UFCC: A Unified Forensic Approach to Locating Tampered Areas in Still Images and Detecting Deepfake Videos by Evaluating Content Consistency

Image inpainting and Deepfake techniques have the potential to drastically alter the meaning of visual content, posing a serious threat to the integrity of both images and videos. Addressing this challenge requires the development of effective methods to verify the authenticity of investigated visual data. This research introduces UFCC (Unified Forensic Scheme by Content Consistency), a novel forensic approach based on deep learning. UFCC can identify tampered areas in images and detect Deepfake videos by examining content consistency, assuming that manipulations can create dissimilarity between tampered and intact portions of visual data. The term “Unified” signifies that the same methodology is applicable to both still images and videos. Recognizing the challenge of collecting a diverse dataset for supervised learning due to various tampering methods, we overcome this limitation by incorporating information from original or unaltered content in the training process rather than relying solely on tampered data. A neural network for feature extraction is trained to classify imagery patches, and a Siamese network measures the similarity between pairs of patches. For still images, tampered areas are identified as patches that deviate from the majority of the investigated image. In the case of Deepfake video detection, the proposed scheme involves locating facial regions and determining authenticity by comparing facial region similarity across consecutive frames. Extensive testing is conducted on publicly available image forensic datasets and Deepfake datasets with various manipulation operations. The experimental results highlight the superior accuracy and stability of the UFCC scheme compared to existing methods.

DeepMark: A Scalable and Robust Framework for DeepFake Video Detection

With the rapid growth of DeepFake video techniques, it becomes increasingly challenging to identify them visually, posing a huge threat to our society. Unfortunately, existing detection schemes are limited to exploiting the artifacts left by DeepFake manipulations, so they struggle to keep pace with the ever-improving DeepFake models. In this work, we propose DeepMark, a scalable and robust framework for detecting DeepFakes. It imprints essential visual features of a video into DeepMark Meta (DMM) and uses it to detect DeepFake manipulations by comparing the extracted visual features with the ground truth in DMM. Therefore, DeepMark is future-proof, because a DeepFake video must aim to alter some visual feature, no matter how “natural” it looks. Furthermore, DMM also contains a signature for verifying the integrity of the above features. And an essential link to the features as well as their signature is attached with error correction codes and embedded in the video watermark. To improve the efficiency of DMM creation, we also present a threshold-based feature selection scheme and a deduced face detection scheme. Experimental results demonstrate the effectiveness and efficiency of DeepMark on DeepFake video detection under various datasets and parameter settings.

A novel approach for detecting deep fake videos using graph neural network

Deep fake technology has emerged as a double-edged sword in the digital world. While it holds potential for legitimate uses, it can also be exploited to manipulate video content, causing severe social and security concerns. The research gap lies in the fact that traditional deep fake detection methods, such as visual quality analysis or inconsistency detection, need help to keep up with the rapidly advancing technology used to create deep fakes. That means there's a need for more sophisticated detection techniques. This paper introduces an enhanced approach for detecting deep fake videos using graph neural network (GNN). The proposed method splits the detection process into two phases: a mini-batch graph convolution network stream four-block CNN stream comprising Convolution, Batch Normalization, and Activation function. The final step is a flattening operation, which is essential for connecting the convolutional layers to the dense layer. The fusion of these two phases is performed using three different fusion networks: FuNet-A (additive fusion), FuNet-M (element-wise multiplicative fusion), and FuNet-C (concatenation fusion). The paper further evaluates the proposed model on different datasets, where it achieved an impressive training and validation accuracy of 99.3% after 30 epochs.

A Novel Blockchain-Based Deepfake Detection Method Using Federated and Deep Learning Models

In recent years, the proliferation of deep learning (DL) techniques has given rise to a significant challenge in the form of deepfake videos, posing a grave threat to the authenticity of media content. With the rapid advancement of DL technology, the creation of convincingly realistic deepfake videos has become increasingly prevalent, raising serious concerns about the potential misuse of such content. Deepfakes have the potential to undermine trust in visual media, with implications for fields as diverse as journalism, entertainment, and security. This study presents an innovative solution by harnessing blockchain-based federated learning (FL) to address this issue, focusing on preserving data source anonymity. The approach combines the strengths of SegCaps and convolutional neural network (CNN) methods for improved image feature extraction, followed by capsule network (CN) training to enhance generalization. A novel data normalization technique is introduced to tackle data heterogeneity stemming from diverse global data sources. Moreover, transfer learning (TL) and preprocessing methods are deployed to elevate DL performance. These efforts culminate in collaborative global model training zfacilitated by blockchain and FL while maintaining the utmost confidentiality of data sources. The effectiveness of our methodology is rigorously tested and validated through extensive experiments. These experiments reveal a substantial improvement in accuracy, with an impressive average increase of 6.6% compared to six benchmark models. Furthermore, our approach demonstrates a 5.1% enhancement in the area under the curve (AUC) metric, underscoring its ability to outperform existing detection methods. These results substantiate the effectiveness of our proposed solution in countering the proliferation of deepfake content. In conclusion, our innovative approach represents a promising avenue for advancing deepfake detection. By leveraging existing data resources and the power of FL and blockchain technology, we address a critical need for media authenticity and security. As the threat of deepfake videos continues to grow, our comprehensive solution provides an effective means to protect the integrity and trustworthiness of visual media, with far-reaching implications for both industry and society. This work stands as a significant step toward countering the deepfake menace and preserving the authenticity of visual content in a rapidly evolving digital landscape.

Individual Responsibility Around Deepfakes: It's No Laughing Matter.

The objectives of this research were to examine the contextual factors that impact individual's interpretation of their responsibility in the context of Deepfake videos. Using a test/retest methodology, a total of 1,023 respondents participated in a Deepfake survey instrument which measured perceptions of individual responsibility with respect to Deepfakes, individual concern with Deepfakes, and humorous perception of Deepfakes. The results of the study found that individual responsibility is negatively related to individual concern, indicating the externalization of responsibility for difficult to detect fake online videos designed to be convincing. Further, humorous perception and age impact the participants perception of individual responsibility. Younger participants were more likely to find Deepfakes humorous and this increased their perception of their own responsibility, potentially exposing them to greater harm from malicious Deepfakes.

Deepfake Detection Using Spatiotemporal Transformer

Recent advances in generative models and the availability of large-scale benchmarks have made deepfake video generation and manipulation easier. Nowadays, the number of new hyper-realistic deepfake videos used for negative purposes is dramatically increasing, thus creating the need for effective deepfake detection methods. Although many existing deepfake detection approaches, particularly CNN-based methods, show promising results, they suffer from several drawbacks. In general, poor generalization results have been obtained under unseen/new deepfake generation methods. The crucial reason for the above defect is that CNN-based methods focus on the local spatial artifacts, which are unique for every manipulation method. Therefore, it is hard to learn the general forgery traces of different manipulation methods without considering the dependencies that extend beyond the local receptive field. To address this problem, this paper proposes a framework that combines aper proposes a framework that combines with Vision Transformer (ViT) to improve detection accuracy and enhance generalizability. Our method, named HCiT , exploits the advantages of CNNs to extract meaningful local features, as well as the VIT’s self-attention mechanism to learn discriminative global contextual dependencies in a frame-level image explicitly. In this hybrid architecture, the high-level feature maps extracted from the CNN are fed into the ViT model that determines whether a specific video is fake or real. Experiments were performed on Faceforensics++, DeepFake Detection Challenge preview, Celeb datasets, and the results show that the proposed method significantly outperforms the state-of-the-art methods. In addition, the HCiT method shows a great capacity for generalization on datasets covering various techniques of deepfake generation. The source code is available at: https://github.com/KADDAR-Bachir/HCiT

Contribution of Residual Signals to the Detection of Face Swapping in Deepfake Videos

Contribution of Residual Signals to the Detection of Face Swapping in Deepfake Videos

“He looks very real”: Media, knowledge, and search‐based strategies for deepfake identification

AbstractDeepfakes are a potential source of disinformation and the ability to detect them is imperative. While research focused on algorithmic detection methods, there is little work conducted on how people identify deepfakes. This research attempts to fill this gap. Using semi‐structured interviews, participants were asked to identify real and deepfake videos and explain how their decisions were made. Three categories of deepfake identification strategies emerged: the use of surface video and audio cues, processing of the messages conveyed in the video, and the searching of external sources. Participants often used multiple strategies within each category. However, identification challenges occurred due to participants' preconceived notions of deepfake characteristics and the message embodied in the video. This work contributes to research by shifting the focus from the algorithmic detection of deepfakes to human‐oriented strategies. Practically, the findings provide guidance on how people can identify deepfakes, which can also form the basis for the development of educational materials.

Analyzing temporal coherence for deepfake video detection

<abstract><p>Current facial image manipulation techniques have caused public concerns while achieving impressive quality. However, these techniques are mostly bound to a single frame for synthesized videos and pay little attention to the most discriminatory temporal frequency artifacts between various frames. Detecting deepfake videos using temporal modeling still poses a challenge. To address this issue, we present a novel deepfake video detection framework in this paper that consists of two levels: temporal modeling and coherence analysis. At the first level, to fully capture temporal coherence over the entire video, we devise an efficient temporal facial pattern (TFP) mechanism that explores the color variations of forgery-sensitive facial areas by providing global and local-successive temporal views. The second level presents a temporal coherence analyzing network (TCAN), which consists of novel global temporal self-attention characteristics, high-resolution fine and low-resolution coarse feature extraction, and aggregation mechanisms, with the aims of long-range relationship modeling from a local-successive temporal perspective within a TFP and capturing the vital dynamic incoherence for robust detection. Thorough experiments on large-scale datasets, including FaceForensics++, DeepFakeDetection, DeepFake Detection Challenge, CelebDF-V2, and DeeperForensics, reveal that our paradigm surpasses current approaches and stays effective when detecting unseen sorts of deepfake videos.</p></abstract>

Multi-feature fusion based face forgery detection with local and global characteristics.

The malicious use of deepfake videos seriously affects information security and brings great harm to society. Currently, deepfake videos are mainly generated based on deep learning methods, which are difficult to be recognized by the naked eye, therefore, it is of great significance to study accurate and efficient deepfake video detection techniques. Most of the existing detection methods focus on analyzing the discriminative information in a specific feature domain for classification from a local or global perspective. Such detection methods based on a single type feature have certain limitations in practical applications. In this paper, we propose a deepfake detection method with the ability to comprehensively analyze the forgery face features, which integrates features in the space domain, noise domain, and frequency domain, and uses the Inception Transformer to learn the mix of global and local information dynamically. We evaluate the proposed method on the DFDC, Celeb-DF, and FaceForensic++ benchmark datasets. Extensive experiments verify the effectiveness and good generalization of the proposed method. Compared with the optimal model, the proposed method with a small number of parameters does not use pre-training, distillation, or assembly, but still achieves competitive performance. The ablation experiments evaluate the role of each component.

THREATS TO THE USE OF ARTIFICIAL INTELLIGENCE AND ITS LEGISLATIVE

We are encountering the term artificial intelligence more and more often. In everyday life, we are almost completely unaware that we come into contact with it and use it. It is found in various areas of human life. Artificial intelligence does not have comprehensive legislation to date. There are a number of states active in this area, with their own leaders. These include the United States of America, the United Kingdom and the European Union. There are a number of signed declarations, regulations and procedures in this area, but a complete regulatory framework is still lacking or is only at the beginning and needs to be implemented in society. In this paper, we pay attention to the basic concepts and breakdown of artificial intelligence, which we define in the theoretical part. We then characterize the risks that artificial intelligence may pose. In addition to its relatively large contribution in various fields of development, education, or streamlining administrative affairs, artificial intelligence poses risks that humans can exploit for their own enrichment, information acquisition, or political influence. AI can work efficiently and relatively quickly with large volumes of data. It can analyse it and learn from it. It can therefore be exploited, for example, in the context of censorship, the creation of false content and disinformation, phishing, Ramsomware, cyberattacks on various companies or institutions, deepfake videos and so on. In the conclusion of the present study, we will analyse the activities of the European Union in the field of laying the legislative framework for artificial intelligence from 2020 to the present. These regulatory activities can contribute to positive developments in eliminating the misuse of artificial intelligence for various activities dangerous to society. The speed of implementation and the quality of regulation will be an important factor for the future direction of AI.

The Ethical Risks and Regulatory Logic of Deep Fake Video

The Ethical Risks and Regulatory Logic of Deep Fake Video

Improving Video Vision Transformer for Deepfake Video Detection Using Facial Landmark, Depthwise Separable Convolution and Self Attention

Improving Video Vision Transformer for Deepfake Video Detection Using Facial Landmark, Depthwise Separable Convolution and Self Attention

Exploring Static–Dynamic ID Matching and Temporal Static ID Inconsistency for Generalizable Deepfake Detection

Identity‐based Deepfake detection methods have the potential to improve the generalization, robustness, and interpretability of the model. However, current identity‐based methods either require a reference or can only be used to detect face replacement but not face reenactment. In this paper, we propose a novel Deepfake video detection approach based on identity anomalies. We observe two types of identity anomalies: the inconsistency between clip‐level static ID (facial appearance) and clip‐level dynamic ID (facial behavior) and the temporal inconsistency of image‐level static IDs. Since these two types of anomalies can be detected through self‐consistency and do not depend on the manipulation type, our method is a reference‐free and manipulation‐independent approach. Specifically, our detection network consists of two branches: the static–dynamic ID discrepancy detection branch for the inconsistency between dynamic and static ID and the temporal static ID anomaly detection branch for the temporal anomaly of static ID. We combine the outputs of the two branches by weighted averaging to obtain the final detection result. We also designed two loss functions: the static–dynamic ID matching loss and the dynamic ID constraint loss, to enhance the representation and discriminability of dynamic ID. We conduct experiments on four benchmark datasets and compare our method with the state‐of‐the‐art methods. Results show that our method can detect not only face replacement but also face reenactment, and also has better detection performance over the state‐of‐the‐art methods on unknown datasets. It also has superior robustness against compression. Identity‐based features provide a good explanation of the detection results.

A Deepfake Video Detection Considering Sequential Keyframes and An Estimation of Fake Area by Input Attribution

A Deepfake Video Detection Considering Sequential Keyframes and An Estimation of Fake Area by Input Attribution

Conspiracy thinking and social media use are associated with ability to detect deepfakes

Deepfake videos are highly realistic manipulated videos that are often difficult to distinguish from authentic videos. The technology rapidly evolves, making deepfake videos increasingly realistic. Most research on deepfake videos is focused on the algorithmic detection of deepfakes. Less is known about the human recognition of deepfake videos. The aim of the current study was to investigate the predictors of human performance at recognizing deepfake videos. Our findings show that humans perform better at recognizing deepfake videos of familiar persons compared to deepfakes of unfamiliar persons. Next, our findings show a positive relationship between time spent on social media and deepfake detection performance, as well as conspiracy thinking and deepfake detection performance. No relationships were found between age and gender and deepfake detection performance.