Image Tamper Localization Research Articles

A two-branch encoder-decoder network for image tampering localization

Tampered images with false information can mislead viewers and pose security issues. Tampering traces in images are difficult to detect. To locate tampering traces effectively, a dual-domain deep-learning-based image tampering localization method based on RGB and frequency stream branches is proposed in this work. The former branch learns and extracts tampered features on the image and content features of the tampered region. The latter branch extracts tampered features from the frequency domain to complement the RGB stream branch. In addition, an attention mechanism is used to integrate the features from both branches at the fusion stage. In the experiments, the F1 score of the proposed method outperformed those of the baselines on the NIST16 dataset (with a 15.3 % improvement), and the AUC score outperformed those of the baselines on the NIST16 and COVERAGE datasets (improvements of 3.9 % and 4.7 %, respectively). This study provides a beneficial alternative to image tampering localization techniques.

Transferable adversarial attack on image tampering localization

It is significant to evaluate the security of existing digital image tampering localization algorithms in real-world applications. In this paper, we propose an adversarial attack scheme to reveal the reliability of such deep learning-based tampering localizers, which would be fooled and fail to predict altered regions correctly. Specifically, two practical adversarial example methods are presented in a unified attack framework. In the optimization-based adversarial attack, the victim image forgery is treated as the parameter to be optimized via Adam optimizer. In the gradient-based adversarial attack, the invisible perturbation yielded by Fast Gradient Sign Method (FGSM) is added to the tampered image along gradient ascent direction. The black-box attack is achieved by relying on the transferability of such adversarial examples to different localizers. Extensive experiments verify that our attacks can sharply reduce the tampering localization accuracy while preserving high visual quality for attacked images. Source code is available at https://github.com/multimediaFor/AttackITL.

A Contribution-Aware Noise Feature representation model for image manipulation localization

Noise feature modules play a prevalent role in image manipulation localization. However, existing approach involves utilizing a specific noise feature module tailored to address distinct categories of tampering procedures, limiting its adaptability across diverse scenarios. This limitation becomes particularly pronounced in image manipulation localization where a singular noise module struggles to comprehensively represent all potential tampering types, given the inherent uncertainty surrounding the nature of tampering. A novel Contribution-Aware Noise Feature (CANF) representation model, specifically designed to tackle the intricacies of image tamper noise, is introduced in this paper. The proposed model integrates existing noise feature modules, autonomously enhancing their capabilities. This enhancement not only contributes significantly to image manipulation localization during the inference phase but also enables a quantitative evaluation of the distinct contributions of various noise modules to the inference results. To optimize the utilization of image information, the Red–Green–Blue (RGB) and CANF data are fused, leading to the development of a standardized image manipulation localization architecture. This architecture exhibits heightened efficiency in harnessing diverse types of information, thereby enhancing the overall performance of image manipulation localization. Experimental results demonstrate that our approach, alongside established methods IF-OSN, MSFF, PCL, and NCL, surpasses the state-of-the-art NCL by 8.5% and 5.4% concerning average F1 and AUC metrics across four standard image manipulation localization datasets: NIST16, CASIAv1, Columbia, and Coverage. In addition, we comprehensively explore the impact of various noise characterization modules on different types of tampering and the fact that Late Fusion is more conducive to image tampering localization.

MGQFormer: Mask-Guided Query-Based Transformer for Image Manipulation Localization

Deep learning-based models have made great progress in image tampering localization, which aims to distinguish between manipulated and authentic regions. However, these models suffer from inefficient training. This is because they use ground-truth mask labels mainly through the cross-entropy loss, which prioritizes per-pixel precision but disregards the spatial location and shape details of manipulated regions. To address this problem, we propose a Mask-Guided Query-based Transformer Framework (MGQFormer), which uses ground-truth masks to guide the learnable query token (LQT) in identifying the forged regions. Specifically, we extract feature embeddings of ground-truth masks as the guiding query token (GQT) and feed GQT and LQT into MGQFormer to estimate fake regions, respectively. Then we make MGQFormer learn the position and shape information in ground-truth mask labels by proposing a mask-guided loss to reduce the feature distance between GQT and LQT. We also observe that such mask-guided training strategy has a significant impact on the convergence speed of MGQFormer training. Extensive experiments on multiple benchmarks show that our method significantly improves over state-of-the-art methods.

EC-Net: General image tampering localization network based on edge distribution guidance and contrastive learning

General Image Tampering Localization (GITL) is a challenging task that aims to locate tampered regions in manipulated images accurately. Despite notable achievements in recent years, existing methods still struggle with generalization and pixel-level segmentation, particularly in the face of diverse tampering forms and invisible traces. To address these issues, we propose EC-Net, a GITL network based on edge distribution guidance and contrastive learning. EC-Net decomposes the task into two phases: the coarse locating phase and the fine locating phase. In the coarse locating phase, a contrastive learning strategy is incorporated to improve the model's generalization capabilities. Additionally, an edge distribution prediction method is introduced to anticipate the global contour information of tampered regions. In the fine locating phase, we introduced a cross-scale fusion mechanism to synthesize feature information across different scales, enhancing the model's ability to discern subtle changes in tampered regions. Additionally, an edge reconstruction mechanism is introduced to restore details, thereby improving the accuracy of the final localization results. Evaluation on seven datasets shows that EC-Net significantly outperforms seven state-of-the-art methods, improving localization performance by 0.095 (F1), 0.088 (IoU), and 0.043 (AUC) compared to the second-best method on average. Additionally, we present a tampered image dataset (RLS26K) for model auxiliary training, featuring diverse real-life scenes with various tampering types, diverse styles, and rich content.

Research on Improved MobileViT Image Tamper Localization Model

Research on Improved MobileViT Image Tamper Localization Model

Effective image tampering localization with multi-scale ConvNeXt feature fusion

With the widespread use of powerful image editing tools, image tampering becomes easy and realistic. Existing image forensic methods still face challenges of low generalization performance and robustness. In this letter, we propose an effective image tampering localization scheme based on ConvNeXt encoder and multi-scale Feature Fusion (ConvNeXtFF). Stacked ConvNeXt blocks are utilized as an encoder to capture hierarchical multi-scale features, which are then fused in decoder for locating tampered pixels accurately. Combined loss function and effective data augmentation strategies are adopted to further improve the model performance. Extensive experimental results show that both localization accuracy and robustness of the ConvNeXtFF scheme outperform other state-of-the-art ones. The source code is available at https://github.com/multimediaFor/ConvNeXtFF.

UP-Net: Uncertainty-Supervised Parallel Network for Image Manipulation Localization

Image manipulation localization remains a hot topic due to its inherent semantic-independent nature and realistic needs. Virtually all localization studies are devoted to solving arbitrary tampering using multi-branch networks based on deep features or skip-connection structures based on full features, which may induce the loss of manipulation details or noisy interference from image semantics. This poses a challenge for existing localization methods to fully capture invisible manipulations, especially in post-processing settings and across dataset scenarios. To address the above issues, we propose an uncertainty-supervised parallel network (UP-Net) for image tampering localization that preserves more manipulation details while avoiding semantic noise. UP-Net cascades the frequency and RGB domains of the manipulated image as dual-domain embedding, instead of dual-domain parallel learning as in previous work. To learn semantic-independent manipulation features, two structurally identical parallel branches are designed to learn tampering inconsistencies from intermediate and deep coding features for gradually obtaining the initial and final localization predictions. Where attention-guided partial decoder (AGPD) integrates more precise manipulation edges and manipulation semantics without introducing additional noise by focusing on channel correlation and spatial dependence, making a significant contribution to performance. Moreover, the new concept of uncertainty-constrained loss supervision is introduced to guide UP-Net to continuously improve confidence in locating difficult pixels, which are easily misclassified due to post-processing operations. Experiments on three public manipulation datasets and two real challenge datasets show that our end-to-end UP-Net achieves significant performance in manipulation localization, generalization across datasets, and robustness compared to state-of-the-art methods.

Semi-Blind watermarking using convolutional attention-based turtle shell matrix for tamper detection and recovery of medical images

“Telemedicine or e-medicine” refers to an electronic communication system that shares medical information from one location to another through the transmission channel. The increase in multimedia data sharing/transmission across the communication channel has raised to an uncontrollable stage. Thus, the transmission of medical reports across the network has increased higher, which has a chance of tampering either intentional or unintentional attacks. In the case of deliberate attacks, tamper detection and localization are the highly required criteria to verify the originality. In such cases, medical image authentication, security, and integrity verification are also essential criteria in telemedicine applications to verify whether the report belongs to the right person or not. To achieve these criteria and to satisfy these requirements Digital watermarking techniques are highly suggested. Much research has been carried out to address this issue, but some drawbacks and limitations still need to be addressed. The objective is to accomplish a Tamper detection, localization, and recovery (TDLR) system with minimal source information at the receiver end with a better payload or embedding capacity. This paper presents a novel Conventional Attention network (CoAtNet) based invariant watermark generation. A two-level embedding is proposed to enhance watermark security, where image tamper localization and recovery information is generated. The primary level of embedding is carried out by the Turtle Shell Data Hiding Algorithm (TSDH), and the secondary level of embedding is carried out by the DWT-SVD transform. The result shows that the proposed model achieves high imperceptibility and tamper detection accuracy of 60.24 and 99.51% respectively. Furthermore, the suggested system performed well in evaluations of tamper detection and image restoration for various malicious attacks against state-of-the-art systems.

A distortion model-based pre-screening method for document image tampering localization under recapturing attack

Document images are vulnerable to tampering by image editing tools. The forgery trace can be concealed by a simple but effective counter-forensic measure, i.e., recapturing the altered document image. It is of practical need to study the tampering localization method under recapturing attack. In this work, we first study spatial and spectral distortion models in the printing and scanning process. The distortion models are then employed in extracting spectral features in both tampered and untampered regions. The proposed forensic scheme can then be established by comparing the spectral features in both the questioned document image and the reference halftone patterns (obtained by exploiting the prior knowledge of the printing device). To evaluate the performance of our approach, we gather a high-quality image database of 528 captured or recaptured documents (about 185K patches) as well as 72 tampered-and-recaptured documents (about 27K patches). The experimental results show that the proposed method can accurately classify recaptured document images with AUC as high as 0.9999 even though the training and testing samples are collected by different devices. In the tampering localization experiment, the proposed method can be combined with some generic CNN models to yield a two-stage scheme with high efficiency and accuracy, i.e., F1-score as high as 0.9. Finally, we also show that the proposed method is a practical solution even without the prior knowledge of the printer model is unavailable. To benefit the academic society, the resource of our work is online available at http://shorturl.at/jxELP.

Differentiating digital image forensics and tampering localization by a novel hybrid approach

Differentiating digital image forensics and tampering localization by a novel hybrid approach

Deep Learning Based Image Forgery Detection Methods

Increasingly advanced image processing technology has made digital image editing easier and easier. With image processing software at one’s fingertips, one can easily alter the content of an image, and the altered image is so realistic that it is illegible to the naked eye. These tampered images have posed a serious threat to personal privacy, social order, and national security. Therefore, detecting and locating tampered areas in images has important practical significance, and has become an important research topic in the field of multimedia information security. In recent years, deep learning technology has been widely used in image tampering localization, and the achieved performance has significantly surpassed traditional tampering forensics methods. This paper mainly sorts out the relevant knowledge and latest methods in the field of image tampering detection based on deep learning. According to the two types of tampering detection based on deep learning, the detection tasks of the method are detailed separately, and the problems and future prospects in this field are discussed. It is quite different from the existing work: 1) This paper mainly focuses on the problem of image tampering detection, so it does not elaborate on various forensic methods. 2) This paper focuses on the detection method of image tampering based on deep learning. 3) This paper is driven by the needs of tampering targets, so it pays more attention to sorting out methods for different tampering detection tasks.

Sudoku-Based Watermarking Scheme for Image Tampering Localization and Self-Recovery

Sudoku-Based Watermarking Scheme for Image Tampering Localization and Self-Recovery

Efficient image tampering localization using semi-fragile watermarking and error control codes

In this paper, we propose an image tampering localization algorithm using semi-fragile watermarking and Error-Locating codes in the DWT domain. By introducing different families of codes, we show the benefit in terms of image tampering localization and complexity of using control code error localization as an authentication function. Indeed, we first experimentally show that error localization block codes is as precise as using classical error correcting codes (Reed-Solomon and BCH codes) to locate image tampering. However, their corresponding decoding algorithms complexity is at least quadratic which make them impractical for some real time applications. To solve this problem, we introduce error-control codes called Error-Locating codes where error localization is reduced to a single syndrome computation performed with low number of binary operations (detailed later in the paper). We provide comparisons of image quality and tampering localization performances using error-detection, error-localization and error-correction approaches with different error control codes.

Image Tampering Localization Using Demosaicing Patterns and Singular Value Based Prediction Residue

Almost all image sensors measure only one color per pixel through the color filter array. Missing pixels are estimated using a demosaicing process. For this reason, a demosaiced image leaves a particular trace. When an image is manipulated or tampered, the demosaicing trace can be changed. This change can serve as a basic clue for detecting or localizing image tampering. Demosaicing pattern-based tampering localization algorithms require a re-interpolation process, and the prediction residue between the given image and the re-interpolated image is commonly used to localize tampered regions. However, the prediction residue is not always valid because the demosaicing interpolation kernel cannot be known, which deteriorates the localization performance. This paper presents an effective re-interpolation process using singular value decomposition for an unknown demosaicing method. First, the green channel of the given image is decomposed into four sub-images according to the Bayer pattern. For a small block of each sub-image, the singular value decomposition is performed. The prediction residue is obtained by reconstructing the image block after removing the largest singular value. The feature to localize the forged regions is extracted by the logarithm ratio of the prediction residue variance. The proposed method does not require any statistical model for the extracted feature, because the prediction residue is more accurate than that of conventional methods. We perform intensive experiments for three test datasets and compare the proposed method with state-of-the-art tampering localization methods, the results of which indicate that the proposed scheme outperforms existing approaches.

Image Tampering Localization Using a Dense Fully Convolutional Network

The emergence of powerful image editing software has substantially facilitated digital image tampering, leading to many security issues. Hence, it is urgent to identify tampered images and localize tampered regions. Although much attention has been devoted to image tampering localization in recent years, it is still challenging to perform tampering localization in practical forensic applications. The reasons include the difficulty of learning discriminative representations of tampering traces and the lack of realistic tampered images for training. Since Photoshop is widely used for image tampering in practice, this paper attempts to address the issue of tampering localization by focusing on the detection of commonly used editing tools and operations in Photoshop. In order to well capture tampering traces, a fully convolutional encoder-decoder architecture is designed, where dense connections and dilated convolutions are adopted for achieving better localization performance. In order to effectively train a model in the case of insufficient tampered images, we design a training data generation strategy by resorting to Photoshop scripting, which can imitate human manipulations and generate large-scale training samples. Extensive experimental results show that the proposed approach outperforms state-of-the-art competitors when the model is trained with only generated images or fine-tuned with a small amount of realistic tampered images. The proposed method also has good robustness against some common post-processing operations.

Source Camera Identification With Dual-Tree Complex Wavelet Transform

Sensor pattern noise (SPN) extraction is a critical stage of the sensor based source camera identification (SCI). However, the quality of the extracted SPN with the traditional discrete wavelet transform (DWT) based method is poor around strong edges and along with the image border. To fill this gap, we propose a dual tree complex wavelet transform (DTCWT) based method to extract the SPN from a given image, which achieves better performance in the area around strong edges. Furthermore, symmetric boundary extension instead of the periodized boundary extension is used for enhancing the quality of SPN along with the image border. Extensive experimental results on both synthetic noisy images and real-world photographs clearly demonstrate the superior SCI performance of the proposed method over state-of-the-arts. Moreover, the proposed method also shows potential in the application of image tampering localization.

Lossless image hierarchical recovery based on POB number system

This paper proposes an effective method for image tampering localization and recovery. In the proposed scheme, the host image is firstly encrypted by a chaotic system, and then bit-plane separation is conducted on the encrypted image. Afterwards, the certificate watermark and the recovery watermark are calculated respectively on the MSB-plane and LSB-plane. Right after, watermarks are embedded into the two bit-planes with the help of chaotic mapping and permutation ordered binary number system (POB number system), thus, the two shares are generated. The recovery of images contains a two-level comparison, neighbor-based refinement and watermark-based refinement. For the tampered share, the comparison can localize all the tampers, two refinements can eliminate the error judgments generated by two-level comparison and recover the image losslessly. Experimental results demonstrate that the proposed method can recover the tampered image losslessly with one of two shares tampered; some analysis results and comparisons with some existing schemes are presented to show that the proposed scheme achieves the better performance in some criteria.

Multi-scale segmentation strategies in PRNU-based image tampering localization

With the rapid development of advanced media technology, especially the popularization of digital cameras and image editing software, digital images can be easily forged without leaving visible clues. Therefore, image forensics technology for identifying the accuracy, integrity, and originality of digital images has become increasingly important. Photo-response non-uniformity (PRNU) noise, a unique fingerprint of imaging sensors, is a valuable forgery detection tool because of its consistently good detection performance. All kinds of forgeries, including copy-move and splicing, can be dealt with in a uniform manner. This paper addresses the problem of forgery localization based on PRNU estimation and aims to improve the resolution of PRNU-based algorithms. Different from traditional overlapping and sliding window-based methods, in which PRNU correlations are estimated on overlapped patches, the proposed scheme is analyzed based on nonoverlapping and irregular patches. First, the test image is segmented into nonoverlapped patches with multiple scales. Second, correlations of PRNU are estimated on nonoverlapped patches to obtain the real-valued candidate tampering probability map for each individual scale. Then, all of the candidate maps are fused into a single and more reliable probability map through an adaptive window strategy. In the final step, the final decision map is obtained by adopting a conditional random field (CRF) to model neighborhood interactions. The contributions of this work include the following: a novel PRNU-based forgery localization scheme using multi-scale nonoverlapping segmentation is proposed for the first time. Furthermore, the adaptive fusion strategy involves selecting the best candidate tampering probability individually for each location in the image. Additionally, the experimental results prove that the proposed scheme can achieve much better detection results and robustness compared with the existing state-of-the-art PRNU-based methods.

Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries.

With advanced image journaling tools, one can easily alter the semantic meaning of an image by exploiting certain manipulation techniques such as copy clone, object splicing, and removal, which mislead the viewers. In contrast, the identification of these manipulations becomes a very challenging task as manipulated regions are not visually apparent. This paper proposes a high-confidence manipulation localization architecture that utilizes resampling features, long short-term memory (LSTM) cells, and an encoder-decoder network to segment out manipulated regions from non-manipulated ones. Resampling features are used to capture artifacts, such as JPEG quality loss, upsampling, downsampling, rotation, and shearing. The proposed network exploits larger receptive fields (spatial maps) and frequency-domain correlation to analyze the discriminative characteristics between the manipulated and non-manipulated regions by incorporating the encoder and LSTM network. Finally, the decoder network learns the mapping from low-resolution feature maps to pixel-wise predictions for image tamper localization. With the predicted mask provided by the final layer (softmax) of the proposed architecture, end-to-end training is performed to learn the network parameters through back-propagation using the ground-truth masks. Furthermore, a large image splicing dataset is introduced to guide the training process. The proposed method is capable of localizing image manipulations at the pixel level with high precision, which is demonstrated through rigorous experimentation on three diverse datasets.