Image Forensics Research Articles

Image Forensics in the Encrypted Domain

Image Forensics in the Encrypted Domain

Towards a unified XAI-based framework for digital forensic investigations

Explainable Artificial Intelligence (XAI) aims to alleviate the black-box AI conundrum in the field of Digital Forensics (DF) (and others) by providing layman-interpretable explanations to predictions made by AI models. It also handles the increasing volumes of forensic images that are impossible to investigate via manual methods; or even automated forensic tools. A holistic, generalized, yet exhaustive framework detailing the workflow of XAI for DF is proposed for standardization. A case study examining the implementation of the framework in a network forensics investigative scenario is presented for demonstration. In addition, the XAI-DF project lays the basis for a collaborative effort from the forensics community, aimed at creating an open-source forensic database that may be employed to train AI models for the digital forensics domain. As an onset contribution to the project, we create a memory forensics database of 27 memory dumps (Windows 7, 10, and 11) simulating malware activity and extracting relevant features (specific to processes, injected code, network connections, API hooks, and process privileges) that may be used for training, testing, and validating AI models in keeping with the XAI-DF framework.

Advanced copy-move forgery detection: utilizing AKAZE in conjunction with SIFT algorithm for image forensics

Advanced copy-move forgery detection: utilizing AKAZE in conjunction with SIFT algorithm for image forensics

Image manipulation localization via dynamic cross-modality fusion and progressive integration

Image manipulation localization aims at distinguishing forged regions from the whole test image. Though many outstanding prior arts have been proposed for this task, there are still two issues that need to be further studied: (1) how to fuse diverse types of features with forgery clues; (2) how to progressively integrate multistage features for better localization performance. In this paper, we propose a tripartite progressive integration network (TriPINet) for end-to-end image manipulation localization. First, we extract both visual perception information, e.g., RGB input images, and visual imperceptible features, e.g., frequency and noise traces for forensic feature learning. Second, we develop a guided cross-modality dual-attention (gCMDA) module to fuse different types of forged clues. Third, we design a set of progressive integration squeeze-and-excitation (PI-SE) modules to improve localization performance by appropriately incorporating multiscale features in the decoder. Extensive experiments are conducted to compare our method with state-of-the-art image forensics approaches. The proposed TriPINet obtains competitive results on several benchmark datasets.

Efficient detection of intra/inter-frame video copy-move forgery: A hierarchical coarse-to-fine method

With a simple forgery technique but a realistic result, video copy-move forgery has currently become one of the most popular tampering manners. In the last couple of years, various new techniques deriving from machine intelligence and pattern recognition have been widely proposed for image forensics. However, it still faces a very challenging task in the field of video copy-move forgery for four reasons: i) Low F1 score and high false-alarm; ii) Lack of a synthesis processing framework; iii) Weak detection robustness and accuracy; iv) Low efficiency. A novel Hierarchical Coarse-to-Fine framework for effective video copy-move forgery detection is proposed to overcome these challenges: i) In the coarse forgery frame-pair matching, the coarse copy-move frame-pairs matching algorithm with the newly proposed two-pass filters can locate real forgery frame-pairs (FFP) and also reduce false-alarm. ii) Through further analysis of the actual FFP, the detection of intra-frame and inter-frame copy-move forgeries can be accurately and simultaneously determined. iii) In the fine keypoint-pairs matching, our newly designed two-hierarchical keypoint-pair filtering can accurately localize the forgery region at pixel level under various adverse conditions. iv) The novel Hierarchical Coarse-to-Fine framework (together with the newly designed algorithms above) considers only the real FFP and true keypoint-pairs for computation, resulting in higher efficiency and accuracy. Finally, Delaunay Triangulation-based region filling is employed to indicate the forgery regions. Compared to the latest methods, our algorithm has been tested extensively and found to be the best at detecting forgeries, with a top score of F1=0.77 and no false-alarms, even under different types of attacks, as validated by the well-known GRIP dataset.

Forensic research of satellite images forgery: a comprehensive survey

Satellite imagery is a significant and attractive area in remote sensing applications, widely applied in monitoring, managing, and tracking natural disasters. Due to the proliferation of commercial satellites, there is an increasing availability of high-resolution satellite images. However, the ubiquity of image editing tools and the advancement of image processing technologies have made satellite image forgery relatively easy, allowing for the arbitrary addition, removal, or modification of target objects. In recent years, satellite image forgery has caused significant negative effects and potential threats to the nation, society, and individuals, drawing the attention of many scholars. Although forensics of satellite image tampering is an emerging research field that offers new insights into the field of information security, there has been a scarcity of comprehensive surveys in this area. This paper aims to fill this gap and investigates recent advances in satellite image forensics, focusing on tampering strategies and forensic methodologies. First, we discuss the concept of satellite images, the definition of satellite image forgery from global and local perspectives, and the datasets commonly used for satellite image forensics. We then detail each tampering detection and localization method, including their characteristics, advantages, disadvantages, and performance in detection or localization across various notable datasets. We also compare some representative forensic networks using evaluation metrics and public datasets. Finally, the anticipated future directions for satellite image forgery forensics are discussed.

Cross-attention based two-branch networks for document image forgery localization in the Metaverse

In recent years, the Metaverse has garnered significant attention in social and Metahuman realms, showcasing substantial value and immense developmental potential through its integration of virtual and real worlds. However, this integration has also raised security concerns. For instance, in digital image forensics, the malicious dissemination of false images by wrongdoers could result in serious consequences and the propagation of misinformation. This paper presented a novel two-branch network (abbreviated as CAFTB-Net) to detect and localize the forged regions of document images in the Metaverse. One branch extracts manipulation trace directly from spatial information, e.g., unnatural smears, anomalies between pixels, etc. The other branch employs an SRM filter to transform the input image from the color domain into the noise domain, effectively extracting anomalies, such as global noise inconsistencies from the noise domain. Compared to spatial domain features, the discontinuity of forgery traces within the noise domain aids the network in authenticating the document image. The two branches extract local and global features in the forged document images. Finally, we propose a cross-attention module to fuse local spatial features and global noise features. Extensive experimental results demonstrate that the proposed network achieves F1 scores of 0.819 and 0.948 on the SACP and ICDAR datasets, respectively, with AUC scores of 0.933 and 0.764, outperforming some of the state-of-the-art algorithms.

A Lightweight Double Compression Detector for HEIF Images Based on Encoding Information.

Extensive research has been conducted in image forensics on the analysis of double-compressed images, particularly in the widely adopted JPEG format. However, there is a lack of methods to detect double compression in the HEIF format, which has recently gained popularity since it allows for reduced file size while maintaining image quality. Traditional JPEG-based techniques do not apply to HEIF due to its distinct encoding algorithms. We previously proposed a method to detect double compression in HEIF images based on Farid's work on coding ghosts in JPEG images. However, this method was limited to scenarios where the quality parameter used for the first encoding was larger than for the second encoding. In this study, we propose a lightweight image classifier to extend the existing model, enabling the identification of double-compressed images without heavily depending on the input image's quantization history. This extended model outperforms the previous approach and, despite its lightness, demonstrates excellent detection accuracy.

Scalable Image Clustering to screen for self-produced CSAM

The number of cases involving Child Sexual Abuse Material (CSAM) has increased dramatically in recent years, resulting in significant backlogs. To protect children in the suspect’s sphere of influence, immediate identification of self-produced CSAM among acquired CSAM is paramount. Currently, investigators often rely on an approach based on a simple metadata search. However, this approach faces scalability limitations for large cases and is ineffective against anti-forensic measures. Therefore, to address these problems, we bridge the gap between digital forensics and state-of-the-art data science clustering approaches. Our approach enables clustering of more than 130,000 images, which is eight times larger than previous achievements, using commodity hardware and within an hour with the ability to scale even further. In addition, we evaluate the effectiveness of our approach on seven publicly available forensic image databases, taking into account factors such as anti-forensic measures and social media post-processing. Our results show an excellent median clustering-precision (Homogeinity) of 0.92 on native images and a median clustering-recall (Completeness) of over 0.92 for each test set. Importantly, we provide full reproducibility using only publicly available algorithms, implementations, and image databases.

GHIRO Unveiled: A Comprehensive Approach to Image Forensics and Suspect Detection

GHIRO Unveiled: A Comprehensive Approach to Image Forensics and Suspect Detection

Head poses and grimaces: Challenges for automated face identification algorithms?

In today’s biometric and commercial settings, state-of-the-art image processing relies solely on artificial intelligence and machine learning which provides a high level of accuracy. However, these principles are deeply rooted in abstract, complex “black-box systems”. When applied to forensic image identification, concerns about transparency and accountability emerge. This study explores the impact of two challenging factors in automated facial identification: facial expressions and head poses. The sample comprised 3D faces with nine prototype expressions, collected from 41 participants (13 males, 28 females) of European descent aged 19.96 to 50.89 years. Pre-processing involved converting 3D models to 2D color images (256 × 256 px). Probes included a set of 9 images per individual with head poses varying by 5° in both left-to-right (yaw) and up-and-down (pitch) directions for neutral expressions. A second set of 3,610 images per individual covered viewpoints in 5° increments from −45° to 45° for head movements and different facial expressions, forming the targets. Pair-wise comparisons using ArcFace, a state-of-the-art face identification algorithm yielded 54,615,690 dissimilarity scores. Results indicate that minor head deviations in probes have minimal impact. However, the performance diminished as targets deviated from the frontal position. Right-to-left movements were less influential than up and down, with downward pitch showing less impact than upward movements. The lowest accuracy was for upward pitch at 45°. Dissimilarity scores were consistently higher for males than for females across all studied factors. The performance particularly diverged in upward movements, starting at 15°. Among tested facial expressions, happiness and contempt performed best, while disgust exhibited the lowest AUC values.

Forensic analysis of Scientific Linux image using commercial and opensource forensic tools

Depending on their needs and personal preferences, people choose to use different operating systems (OS) such as Windows, Linux, and Mac. The scientific Linux Operating System (SLOS) is designed to provide a stable, secure, and high-performance computing environment for scientific research and education in a steady, scalable, and extensible manner. When criminal activities are committed by suspects involving computers and the internet, it calls for digital forensics which involves the use of scientific procedures and tools to carry out the forensic investigation and analysis of digital evidence for legal and investigative purposes. Forensic investigators use commercial and opensource tools for analysis and gathering inculpatory and exculpatory pieces of evidence. This paper presents a comparative analysis of EnCase, FTK, Autopsy, bulk-extractor, and Scalpel for analyzing the Scientific Linux image. The test scenarios were designed to find out if the selected forensic tools can be appropriately used for investigating crimes committed using the SLOS. The test scenarios include extraction and analysis of operating system details, user accounts, web browsing history, and the recovery of deleted and shredded files and this paper compares and evaluates the capability of the tools in retrieving the evidence designed in the scenarios. This systematic comparison and evaluation results would assist digital forensics practitioners, researchers, and law enforcement agencies in making informed decisions regarding the selection of tools for Scientific Linux image forensics.

UGEE-Net: Uncertainty-guided and edge-enhanced network for image splicing localization

Image splicing, a prevalent method for image tampering, has significantly undermined image authenticity. Existing methods for Image Splicing Localization (ISL) struggle with challenges like limited accuracy and subpar performance when dealing with imperceptible tampering and multiple tampered regions. We introduce an Uncertainty-Guided and Edge-Enhanced Network (UGEE-Net) for ISL to tackle these issues. UGEE-Net consists of two core tasks: uncertainty guidance and edge enhancement. We employ Bayesian learning to model uncertainty maps of tampered regions, directing the model's focus to challenging pixels. Simultaneously, we employ a frequency domain-auxiliary edge enhancement strategy to imbue localization features with global contour information and fine-grained local details. These mechanisms work in parallel, synergistically boosting performance. Additionally, we introduce a cross-level fusion and propagation mechanism that effectively utilizes contextual information for cross-layer feature integration and leverages channel-level correlations for cross-layer feature propagation, gradually enhancing the localization feature's details. Experiment results affirm UGEE-Net's superiority in terms of detection accuracy, robustness, and generalization capabilities. Furthermore, to meet the growing demand for high-quality datasets in image forensics, we present the HTSI12K dataset, which includes 12,000 spliced images with imperceptible tampering traces and diverse categories, rendering it suitable for real-world auxiliary model training.

Decoding Innocence: Advancing Forensic Facial Discrimination through Comparative Analysis of Conventional CNN and Advanced Architectures

The field of Digital Image Forensics (DIF) faces a critical issue in accurately identifying children in digital images, notably in cases involving the proliferation of child sexual abuse content. Existing techniques face hurdles due to model architecture limitations, dataset suitability concerns, and classification imbalance, impeding their ability to recognize children to deter pornographic images. Addressing this challenge, this study introduces Implicit Feature Extraction (IFE), a specialized approach for distinguishing child and adult images in object detection. Leveraging Convolutional Neural Networks (CNNs), the IFE method automates the extraction of discriminative facial features, surpassing the constraints of Explicit Feature Extraction (EFE) methods, which achieve an accuracy of around 70%. The research focuses on three core objectives introducing IFE for detailed face feature detection in DIF's child and adult image identification, implementing IFE with CNNs to enhance image classification, and conducting a thorough evaluation of the proposed technique's performance using key metrics like accuracy and balanced classification results and comparing the result with a basic CNN model’s performance. This research's significance lies in its notable contributions to digital image forensics, particularly in combatting child exploitation. The fusion of IFE with CNNs showcases 92% accuracy in distinguishing child and adult images, promising advancements with practical implications in child protection and forensic investigations. The comprehensive evaluation using the UTKFace dataset underscores the proposed technique's efficacy, marking a substantial improvement in child image identification within digital image forensics.

Semi-supervised image manipulation localization with residual enhancement

Images have become a significant medium for information transmission, while image forensics has garnered widespread attention from researchers. Due to the scarcity of finely annotated images in the field of image manipulation detection and localization, existing methods for locating manipulated images can only utilize a limited amount of data for model training. However, deep learning models typically require a large number of finely annotated images to fully leverage their powerful fitting capabilities. In this paper, we propose a semi-supervised image manipulation localization framework, employing semi-supervised learning to use unannotated images for deep learning model training. To achieve this goal, we first design a residual enhancement module that contains an encoding-decoding structure. The regression target of this branch is the pristine regions in the images to generate the reconstructed images. Next, we perform a residual operation between the reconstructed and original images to roughly locate the anomalous regions and refine the features extracted by the encoder. Secondly, we employ weakly-supervised learning by adding an image-level classification head to the final layer of the encoder. The classification head generates a class active map based on image-level classification results, which further guides the model’s feature augmentation. Finally, we formulate a specialized semi-supervised framework tailored for image manipulation detection, enabling the utilization of a large volume of unannotated data for model training. Extensive experimental results highlight the notable efficacy of the proposed approach, and outperforms state-of-the-art approaches.

Forgered Image Perception System using CNN Algorithms

The progression and diversification of methodologies in digital image forensics and manipulation detection are evident in existing work, presenting innovative techniques tailored to specific aspects of forgery detection. These methodologies encompass various challenges such as copy-move forgery, manipulation detection, demosaicing artifacts, image splicing, edge detection, and compression analysis. Together, these studies exemplify ongoing efforts to enhance the accuracy, efficiency, and reliability of forgery detection methods within the constantly evolving landscape of digital image manipulation. The proposed image forgery detection system represents a pioneering advancement, surpassing the limitations of conventional methodologies. It integrates a comprehensive array of detection techniques, including copy-move analysis, color filter array scrutiny, noise variance inconsistency detection, and double JPEG artifact identification. These techniques are bolstered by Convolutional Neural Networks (CNNs). This innovative fusion not only addresses the limitations inherent in existing systems but also signifies a significant leap forward in functionality and efficacy. By synergistically leveraging the strengths of each detection method within a CNN framework, our approach promises heightened accuracy, robustness, and adaptability in discerning even the most sophisticated forms of image tampering. Keywords— Image Tampering, Double JPEG Compression, Demosaicing Artifacts, Recursive Edge Detection, Copy-Move Forgery Detection.

A robust self-supervised image hashing method for content identification with forensic detection of content-preserving manipulations

Image content identification systems have many applications in industry and academia. In particular, a hash-based content identification system uses a robust image hashing function that computes a short binary identifier summarizing the perceptual content in a picture and is invariant against a set of expected manipulations while being capable of differentiating between different pictures. A common approach to designing these algorithms is crafting a processing pipeline by hand. Unfortunately, once the context changes, the researcher may need to define a new function to adapt. A deep hashing approach exploits the feature learning capabilities in deep networks to generate a feature vector that summarizes the perceptual content in the image, achieving outstanding performance for the image retrieval task, which requires measuring semantic and perceptual similarity between items. However, its application to robust content identification systems is an open area of opportunity. Also, image hashing functions are valuable tools for image authentication. However, to our knowledge, its application to content-preserving manipulation detection for image forensics tasks is still an open research area. In this work, we propose a deep hashing method exploiting the metric learning capabilities in contrastive self-supervised learning with a new modular loss function for robust image hashing. Moreover, we propose a novel approach for content-preserving manipulation detection for image forensics through a sensitivity component in our loss function. We validate our method through extensive experimentation in different data sets and configurations, validating the generalization properties in our work.

A Deep Analysis of Printed Image Forensics with Auto Machine Learning

This research undertakes an exhaustive exploration of the Auto Machine Learning-based approach (AutoML) to ascertain the source printer for microscopic printed image forensics. The primary focus is on enhancing precision and efficacy in a scenario with limited sample data, achieved by integrating traditional image processing methods with diverse AutoML representatives. The investigation involves a meticulous evaluation and comparative analysis of classical Machine Learning (ML) models and AutoML models in microscopic printed image forensics. Through extensive experiments, we highlight the inherent limitations of traditional models and underscore the compelling advantages offered by AutoML. Notably, we recognize AutoML’s unique capability to discern varying degrees of uncertainty within printed patterns. The outcomes emphasize the prowess of Auto Machine Learning in improving the accuracy of printed image forensics, making it particularly promising for future research, especially given the tabular nature of the data and the imperative for a lightweight model suitable for embedded systems.

Content bias in deep learning image age approximation: A new approach towards better explainability

In the context of temporal image forensics, it is not evident that a neural network, trained on images from different time-slots (classes), exploits solely image age related features. Usually, images taken in close temporal proximity (e.g., belonging to the same age class) share some common content properties. Such content bias can be exploited by a neural network. In this work, a novel approach is proposed that evaluates the influence of image content. This approach is verified using synthetic images (where content bias can be ruled out) with an age signal embedded. Based on the proposed approach, it is shown that a deep learning approach proposed in the context of age classification is most likely highly dependent on the image content. As a possible countermeasure, two different models from the field of image steganalysis, along with three different preprocessing techniques to increase the signal-to-noise ratio (age signal to image content), are evaluated using the proposed method.

A convolutional neural network based on noise residual for seam carving detection

Seam carving is a method of resizing images based on content awareness. It can realize image retargeting while retaining the main content of the image. However, it may also be maliciously used to tamper with images, such as changing image semantic content by object removal. Therefore, seam carving detection has become important in image forensics. In this paper, a noise residual-based deep learning method is proposed to detect seam carving images. We try to learn the local noise in-consistency of images to recognize them. Firstly, a noise residual extraction segment is used to learn local noise features in the image, and a noise augmentation module is designed to enrich the features, which leverages the noise features extracted from a steganalysis rich model filter to discover the noise in-consistency between authentic and tampered regions. Then, through the feature dimensionality reduction section, the features are further learned and the size of feature maps are reduced. Finally, the output is obtained through global average pooling and a fully-connected layer. A careful testing strategy is further proposed, which greatly improves the detection performance, especially for seam carving with small scaling ratios. The experimental results demonstrate that our method achieves state-of-the-art performance at various scales, and has good robustness and generalization compared with other methods.