Media Forensics Research Articles

Multimodal Deepfake Detection

Abstract: Deepfake technology has become a significant threat to the integrity of multimedia content, posing challenges to areas such as cybersecurity, media forensics, and information authenticity. To address this, our research introduces a Multimodal Deepfake Detection system capable of identifying manipulated content by combining visual and auditory cues. The model employs convolutional neural networks (CNNs) to analyse video frames and process audio spectrograms, providing a comprehensive approach to detecting deepfake content. Experimental results demonstrate the system's effectiveness in accurately identifying both visual and auditory indicators of deepfake manipulation. This solution shows promise in combating the spread of deepfake content on digital platforms, thereby preserving the integrity and trustworthiness of multimedia content in the digital era

Synthetic Media Analysis Using Deep Learning

In an era characterized by the rapid evolution of digital content creation, synthetic media, particularly deepfake videos, present a formidable challenge to the veracity and integrity of online information. Addressing this challenge requires sophisticated analytical techniques capable of discerning between authentic and manipulated media. This research paper presents a comprehensive study on synthetic media analysis leveraging deep learning methodologies. The suggested method integrates advanced deep learning models, including Convolutional Neural Networks (CNNs) like VGG (Visual Geometry Group), alongside recurrent structures such as LSTM (Long Short-Term Memory). These models are trained and evaluated on a meticulously curated dataset, ensuring diversity and relevance in the synthetic media samples. To facilitate experimentation and reproducibility, the dataset is securely hosted on a reliable platform such as Google Drive. Prior to model training, preprocessing steps including frame extraction are employed to extract essential visual features from the video data. The VGG model serves as a feature extractor, capturing high-level representations of visual content, while the LSTM model learns temporal dependencies and contextual information across frames. Following comprehensive experimentation, the proposed method's ability to detect synthetic media is thoroughly assessed, utilizing metrics such as accuracy. This research contributes to the ongoing discourse on digital media forensics by providing insights into the efficacy of deep learning techniques for synthetic media analysis. The findings underscore the importance of continuous research and development in combating the proliferation of synthetic media, thereby safeguarding the authenticity and trustworthiness of online content. Index Terms—CNN, LSTM, VGG

Artificial Intelligence in Social Media Forensics: A Comprehensive Survey and Analysis

Social media platforms have completely revolutionized human communication and social interactions. Their positive impacts are simply undeniable. What has also become undeniable is the prevalence of harmful antisocial behaviors on these platforms. Cyberbullying, misinformation, hate speech, radicalization, and extremist propaganda have caused significant harms to society and its most vulnerable populations. Thus, the social media forensics field was born to enable investigators and law enforcement agents to better investigate and prosecute these cybercrimes. This paper surveys the latest research works in the field to explore how artificial intelligence (AI) techniques are being utilized in social media forensics investigations. We examine how natural language processing can be used to identify extremist ideologies, detect online bullying, and analyze deceptive profiles. Additionally, we explore the literature on GNNs and how they are applied in social network modeling for forensic purposes. We conclude by discussing the key challenges in the field and suggest future research directions.

Advanced Algorithmic Approaches for Scam Profile Detection on Instagram

Social media platforms like Instagram have become a haven for online scams, employing various deceptive tactics to exploit unsuspecting users. This paper investigates advanced algorithmic approaches to combat this growing threat. We explore various machine learning models for scam profile detection on Instagram. Our methodology involves collecting a comprehensive dataset from a trusted source and meticulously preprocessing the data for analysis. We then evaluate the effectiveness of a suite of machine learning algorithms, including decision trees, logistic regression, SVMs, and other ensemble methods. Each model’s performance is measured using established metrics like accuracy, precision, recall, and F1-scores. Our findings indicate that ensemble methods, particularly random forest, XGBoost, and gradient boosting, outperform other models, achieving accuracy of 90%. The insights garnered from this study contribute significantly to the body of knowledge in social media forensics, offering practical implications for the development of automated tools to combat online deception.

Image Forgery Detection using ResNet50

Abstract: Image forgery detection is crucial in ensuring the integrity of digital media. In this study, we propose a method for detecting image tampering using Error Level Analysis (ELA) and Convolutional Neural Networks (CNNs) with a ResNet50 architecture. Leveraging the CASIA 2.0 Image Tampering Detection Dataset, which consists of authentic (Au) and tampered (Tp) images, along with metadata and annotations provided in the CASIA 2 Ground truth dataset, we develop and evaluate our model. The dataset comprises 7492 authentic images, 5125 tampered images, and 5123 files of ground truth information. ELA transformations highlight compression discrepancies, aiding in the identification of tampered regions. Our ResNet50-based CNN model, augmented with Global Average Pooling, Dense layers, and Dropout regularization, is trained using Adam optimization and binary cross-entropy loss with early stopping. Evaluation metrics, including training and validation loss/accuracy curves and confusion matrices, are used to assess model performance. The trained model is saved for future use and tested on new images to demonstrate its classification capabilities. Our approach achieves a significant level of accuracy in distinguishing between authentic and tampered images, underscoring its potential for practical image forgery detection applications and contributing to advancements in digital media forensics

CatmullRom Splines-Based Regression for Image Forgery Localization

IFL (Image Forgery Location) helps secure digital media forensics. However, many methods suffer from false detections (i.e., FPs) and inaccurate boundaries. In this paper, we proposed the CatmullRom Splines-based Regression Network (CSR-Net), which first rethinks the IFL task from the perspective of regression to deal with this problem. Specifically speaking, we propose an adaptive CutmullRom splines fitting scheme for coarse localization of the tampered regions. Then, for false positive cases, we first develop a novel re-scoring mechanism, which aims to filter out samples that cannot have responses on both the classification branch and the instance branch. Later on, to further restrict the boundaries, we design a learnable texture extraction module, which refines and enhances the contour representation by decoupling the horizontal and vertical forgery features to extract a more robust contour representation, thus suppressing FPs. Compared to segmentation-based methods, our method is simple but effective due to the unnecessity of post-processing. Extensive experiments show the superiority of CSR-Net to existing state-of-the-art methods, not only on standard natural image datasets but also on social media datasets.

CSC-Net: Cross-Color Spatial Co-Occurrence Matrix Network for Detecting Synthesized Fake Images

Recently, the Generative Adversarial Networks (GAN) generated images have been spread over the social networks, which brings the new challenge in the the community of media forensics. Although some reliable forensic tools have advanced the study of detecting GAN generated images, while the detection accuracy cannot be guaranteed when facing the malicious post-processing attacks, especially in the practical social network scenario. Thus, in this context, we propose a novel well-designed deep neural network equipped with handed-crafted features for dealing with this problem. In particular, relying on the Cross-color Spatial Co-occurrence Matrix (CSCM), the discriminative features are extracted after carefully analyzing and selecting the most effective color channels. Next, the fused features are fed into the deep neutral network for training a high-efficient forensic detector. Extensive experimental results empirically verify that in most detection scenarios, our proposed detector performs superiorly to the prior arts, especially in the case of post-processing attacks. Moreover, we also highlight the relevance of the proposed detector over the realistic social network platforms, and its generalization capability in three different scenarios.

Enhancing Fake Image Detection: A Novel Two-Step Approach Combining GANs and CNNs

The proliferation of fake images in today's digital landscape poses a significant threat to various domains, including media integrity, social media, and online security. Recognizing the urgent need to distinguish real images from their deceptive counterparts, this paper underscores the importance of developing a robust detection system. While substantial efforts have been made in the realms of computer vision and deep learning, the advent of Generative Adversarial Networks (GANs) has added a new layer of complexity to this challenge. In response to these evolving threats, we present a novel two-step methodology for detecting fake images, with a specific focus on those generated by GANs. Our approach harnesses the combined strengths of GANs and traditional Convolutional Neural Networks (CNNs), offering a comprehensive solution that significantly enhances accuracy in identifying both fake images and machine-generated fake images. The results of our experiments demonstrate the efficacy of our methodology. Using CNNs alone, we achieved a training accuracy of 87%. However, when employing the collaborative power of GANs and CNNs, our model exhibited a remarkable accuracy rate of 94.4%. This substantial improvement underscores the superiority of the GANs+CNN approach, suggesting its potential as a groundbreaking solution in the realm of fake image detection. This research opens up new horizons in fields such as media forensics, social media monitoring, and online security, where the ability to discern genuine content from manipulated or synthetic media is of paramount importance. The promising outcomes of this study not only provide an immediate and effective solution but also pave the way for further exploration and innovation in this critical area of digital security.

An Enhanced STFT Segmentation Framework for ENF-based Media Forensics

An Enhanced STFT Segmentation Framework for ENF-based Media Forensics

Social Media Forensics: An Adaptive Cyberbullying-Related Hate Speech Detection Approach based on Neural Networks with Uncertainty

Social Media Forensics: An Adaptive Cyberbullying-Related Hate Speech Detection Approach based on Neural Networks with Uncertainty

Improving Detection of DeepFakes through Facial Region Analysis in Images

In the evolving landscape of digital media, the discipline of media forensics, which encompasses the critical examination and authentication of digital images, videos, and audio recordings, has emerged as an area of paramount importance. This heightened significance is predominantly attributed to the burgeoning concerns surrounding the proliferation of DeepFakes, which are highly realistic and manipulated media content, often created using advanced artificial intelligence techniques. Such developments necessitate a profound understanding and advancement in media forensics to ensure the integrity of digital media in various domains. Current research endeavours are primarily directed towards addressing a common challenge observed in DeepFake datasets, which pertains to the issue of overfitting. Many suggested remedies centre around the application of data augmentation methods, with a frequently adopted strategy being the incorporation of random erasure or cutout. This method entails the random removal of sections from an image to introduce diversity and mitigate overfitting. Generating disparities between the altered and unaltered images serves to inhibit the model from excessively adapting itself to individual samples, thus leading to more favourable results. Nonetheless, the stochastic nature of this approach may inadvertently obscure facial regions that harbour vital information necessary for DeepFake detection. Due to the lack of guidelines on specific regions for cutout, most studies use a randomised approach. However, in recent research, face landmarks have been integrated to designate specific facial areas for removal, even though the selection remains somewhat random. Therefore, there is a need to acquire a more comprehensive insight into facial features and identify which regions hold more crucial data for the identification of DeepFakes. In this study, the investigation delves into the data conveyed by various facial components through the excision of distinct facial regions during the training of the model. The goal is to offer valuable insights to enhance forthcoming face removal techniques within DeepFake datasets, fostering a deeper comprehension among researchers and advancing the realm of DeepFake detection. Our study presents a novel method that uses face cutout techniques to improve understanding of key facial features crucial in DeepFake detection. Moreover, the method combats overfitting in DeepFake datasets by generating diverse images with these techniques, thereby enhancing model robustness. The developed methodology is validated against publicly available datasets like FF++ and Celeb-DFv2. Both face cutout groups surpassed the Baseline, indicating cutouts improve DeepFake detection. Face Cutout Group 2 excelled, with 91% accuracy on Celeb-DF and 86% on the compound dataset, suggesting external facial features’ significance in detection. The study found that eyes are most impactful and the nose is least in model performance. Future research could explore the augmentation policy’s effect on video-based DeepFake detection.

ASMF: Ambient social media forensics chain of custody with an intelligent digital investigation process using federated learning

Ambient Intelligence is a concept that relates to a new paradigm of pervasive computing and has the objective of automating responses from the system to humans without any human intervention. In social media forensics, gathering, analyzing, storing, and validating relevant evidence for investigation in a heterogeneous environment is still questionable. There is no hierarchy for automation, even though standardization and secure processes from data collection to validation have not yet been discussed. This poses serious issues for the current investigation procedures and future evidence chain of custody management. This paper contributes threefold. First, it proposes a framework using a blockchain network with a dual chain of data transmission for privacy protection, such as on-chain and off-chain. Second, a protocol is designed to detect and separate local and global cyber threats and undermine multiple federated principles to personalize search space broadly. Third, this study manages personalized updates by means of optimizing backtracking parameters and automating replacements, which directly affects the reduction of negative influence on the social networking environment in terms of imbalanced and distributed data issues. This proposed framework enhances stability in digital investigation. In addition, the simulation uses an extensive social media dataset in different cyberspaces with a variety of cyber threats to investigate. The proposed work outperformed as compared to traditional single-level personalized search and other state-of-the-art schemes.

Enhancing Digital Forensic Investigation: A Focus on Compact Electronic Devices and Social Media Metadata

The rise of portable electronic devices and social media has led to new criminal activities, necessitating advancements in digital forensics. This paper introduces Small-Scale Digital Device Forensics (SSDDF), focusing on the forensic examination of miniature digital devices often used in crimes. SSDDF addresses the challenges posed by these devices, particularly in extracting and analyzing data from them. A key aspect of this research is exploring ontology in social media forensics, particularly within the Android operating system. This involves extracting digital evidence like user accounts, messages, and images from social media platforms. While the paper primarily focuses on social media data, it acknowledges the importance of the devices used in crimes. The integration of SSDDF and the analysis of Android system structures are highlighted as key advancements in digital forensic methodologies. These enhancements are expected to improve the process of collecting and analyzing digital evidence from both compact electronic devices and social media platforms. The study offers significant contributions to the field of digital forensics. It provides new strategies for more efficient and effective forensic investigations, especially in the context of extracting and analyzing digital evidence from small electronic devices and social media, thus paving the way for more robust digital evidence handling in future forensic inquiries.

Digital Forensic Investigation Tools for Cases Related to Social Media and Cybersecurity

The entire world has moved into the digital realm as a result of the quickening pace of technological development. However, this change has also led to an increase in cybercrimes and security breach occurrences, which endanger user security and privacy. As a result, this study sought to examine how digital forensics, a significant advancement in cybersecurity, is used to combat cybercrime. The most recent developments in digital forensics, such as cloud forensics, social media forensics, and IoT forensics, have been examined in this study. With the use of these technologies, cybersecurity experts can protect data while identifying fraudsters by using the digital footprints that data processing and storage leave behind. Technical, operational, and personnel-related issues are among the specific dangers to digital forensics that have been identified by the research. These systems' high level of complexity, volume of data, chain of custody, personnel integrity, and the validity and accuracy of digital forensics are all significant barriers to their widespread adoption. However, the research has also noted the use of artificial intelligence, intrusion detection, and USB forensics as significant prospects for digital forensics that can make the processes simpler, more effective, and safe.

Social Media Metadata Forensic Ontology Model

The escalating use of compact electronic gadgets like smartphones and smartwatches, along with social media platforms, has led to a new dimension of criminal activities. Advancements in digital forensics, which investigates digital evidence, have become noteworthy. Researchers David Christopher Harrill and Richard P. Mislan devised Small-Scale Digital Device Forensics (SSDDF) for examining miniature digital devices used in criminal undertakings. Edlira Kalemi and Sule Yildirim-Yayilgan demonstrated the application of ontology in social media forensics, deciphering Android system frameworks related to social media data extraction. However, their investigations mainly focused on social media, overlooking the role of gadgets used by criminals and victims. Extracting digital data from these devices remains pivotal for securing relevant evidence from social media. Notwithstanding, incorporating SSDDF and understanding Android system structures have significantly contributed to digital forensic methodologies. These enhancements can bolster the investigation process, capturing and analyzing critical digital evidence from compact electronic appliances and social media platforms. The advent of SSDDF and elucidating Android system structures have offered invaluable inputs to digital forensics, enhancing efficacy in amassing significant evidence from small electronic devices and social media platforms.

A Systematic Literature Analysis on the Fake Identification

Abstract- Rapid progress in AI, machine learning, and deep learning over the last few decades has resulted in new methodologies and tools for altering multimedia. Though the technology has generally been utilized for respectable causes such as entertainment and education, rogue people have also used it for illegal or evil purposes. High-quality and realistic fake movies, photos, or audios, for example, have been generated to disseminate disinformation and propaganda, incite political division and hatred, or even harass and blackmail people. Deepfake is a term that refers to modified, high-quality, realistic videos. To address the issues posed by Deepfake, many ways have been discussed in the literature. In this study, we undertake a systematic literature review (SLR) to offer an updated overview of the research efforts in Deepfake detection, summarizing 112 relevant papers from 2018 to 2020 that provided a range of techniques. We classify them into four categories: deep learning-based approaches, traditional machine learning-based methods, statistical techniques, and blockchain-based techniques. We also compare the detection capabilities of the various algorithms across different datasets and find that deep learning-based methods outperform other methods in Deepfake detection. Keyword: Deepfake detection, video or image manipulation, digital media forensics, systematic literature review.

Use of social networking sites and applications in identification of missing and unidentified persons: A robust forensic tool

Social media is an invaluable tool when seeking information or evidence in criminal cases or in finding missing persons. Social media encompasses vast amount of information such as videos, photos, texts posted by criminals, witnesses and/or victims itself. Social media forensics helps recuperation of electronic evidence from social networking sites which often plays a crucial role in the conviction of a suspect. However, there barely exist any applications which can aid in identifying a deceased person.For identification of deceased, forensic odontologists search for antemortem data such as dental casts, X-ray images, portrait pictures showing the anterior teeth etc., for comparison with the post-mortem data. Archetypal features such as tooth rotations, diastemas, missing teeth, supernumerary teeth, crowded teeth, recognizable removable/fixed prostheses and other attributes could be present in the missing person’s profile and can be crosschecked during the autopsy of the unidentified corpse. For this reason, smile photos represent one of the most common sources of dental information, especially when no technical antemortem dental data is available. To date, there is only one forensic application for smartphones which helps in storage of antemortem smile photo of individuals. Storing the antemortem data that is easily accessible from the social media network aids as an important forensic tool especially in natural disasters or terrorist attacks where human identification process can be arduous.

Novel Convolutional Neural Networks based Jaya algorithm Approach for Accurate Deepfake Video Detection

Deepfake videos are becoming an increasing concern due to their potential to spread misinformation and cause harm. In this paper, we propose a novel approach for accurately detecting deepfake videos using the combination of Convolutional Neural Networks (CNNs) with the Jaya algorithm optimization. The approach is evaluated on two publicly available datasets, the DeepFake Detection Challenge (DFDC) dataset and the Celeb-DF dataset, and achieves state-of-the-art performance on both datasets. Our approach achieves an accuracy of 99.3% on the DFDC dataset and 97.6% on the Celeb-DF dataset, with high F1 scores indicating a high precision and recall for detecting deepfake videos. Furthermore, our approach is more robust against adversarial attacks than existing state-of-the-art methods. The combination of CNNs with the Jaya algorithm optimization enables effective capture of the temporal information in the video sequence, while the use of robust evaluation metrics ensures objective measurement and comparison with existing methods. Our proposed approach offers a highly effective solution for detecting deepfake videos, which has the potential to be a valuable tool for media forensics, content moderation, and cyber security.

Human-in-control and quality assurance aspects for a benchmarking framework for DeepFake detection models

Human-in-control is a principle that has long been established in forensics as a strict requirement and is nowadays also receiving more and more attention in many other fields of application where artificial intelligence (AI) is used. This renewed interest is due to the fact that many regulations (among others the the EU Artificial Intelligence Act (AIA)) emphasize it as a necessity for any critical AI application scenario. In this paper, human-in-control and quality assurance aspects for a benchmarking framework to be used in media forensics are discussed and their usage is illustrated in the context of the media forensics sub-discipline of DeepFake detection.

Criminal Behavior Identification Using Social Media Forensics

Human needs consist of five levels, which are: physiological needs, safety needs, love needs, esteem needs and self-actualization. All these needs lead to human behavior. If the environment of a person is positive, healthy behavior is developed. However, if the environment of the person is not healthy, it can be reflected in his/her behavior. Machines are intelligent enough to mimic human intelligence by using machine learning and artificial intelligence techniques. In the modern era, people tend to post their everyday life events on social media in the form of comments, pictures, videos, etc. Therefore, social media is a significant way of knowing certain behaviors of people such as abusive, aggressive, frustrated and offensive behaviors. Behavior detection by crawling the social media profile of a person is a crucial and important idea. The challenge of behavior detection can be sorted out by applying social media forensics on social media profiles, which involves NLP and deep learning techniques. This paper is based on the study of state of the art work on behavior detection, and based on the research, a model is proposed for behavior detection. The proposed model outperformed with an F1 score of 87% in the unigram + bigram class, and in the bigram + trigram class, it gave an F1 score of 88% when compared with models applied on state of the art work. This study is a great benefit to cybercrime and cyber-security agencies in shortlisting the profiles containing certain behaviors to prevent crimes in the future.