Consistent Noise Research Articles

Learning Compressed Artifact for JPEG Manipulation Localization Using Wide-Receptive-Field Network

JPEG image manipulation localization aims to accurately classify and locate tampered regions in JPEG images. Existing image manipulation localization schemes usually consider diverse data streams of spatial domain, e.g. noise inconsistency and local content inconsistency. They, however, easily ignore an objective scenario: data stream features of spatial domain are hard to directly apply to compressed image format, e.g., JPEG, because tampered JPEG images may contain severe re-compression inconsistency and re-compression artifacts, when they are re-compressed to JPEG format. As a result, the traditional localization schemes relying on general data streams of spatial domain may result in a large number of false detection of tampered region in JPEG images. To address the above problem, we a new JPEG image manipulation localization scheme, in which a wide-receptive-field attention network is designed to effectively learn JPEG compressed artifacts. We firstly introduce the wide-receptive-field attention mechanism to re-construct U-Net network, which can effectively capture contextual information of JPEG images and analyze tampering traces from different image regions. Furthermore, a flexible JPEG compressed artifact learning module is designed to capture the image noise caused by JPEG compression, in which the weights can be adjusted flexibly based on image quality, without the need for decompression operations on JPEG images. Our proposed method can significantly strength the differentiation capability of detection model for tampered and non-tampered regions. A series of experiments are performed over different image sets, and the results demonstrate that the proposed scheme can achieve an overall localization performance for multi-scale JPEG manipulation regions and outperform most of state-of-the-art schemes in terms of detection accuracy, generalization and robustness.

Cross-attention based two-branch networks for document image forgery localization in the Metaverse

In recent years, the Metaverse has garnered significant attention in social and Metahuman realms, showcasing substantial value and immense developmental potential through its integration of virtual and real worlds. However, this integration has also raised security concerns. For instance, in digital image forensics, the malicious dissemination of false images by wrongdoers could result in serious consequences and the propagation of misinformation. This paper presented a novel two-branch network (abbreviated as CAFTB-Net) to detect and localize the forged regions of document images in the Metaverse. One branch extracts manipulation trace directly from spatial information, e.g., unnatural smears, anomalies between pixels, etc. The other branch employs an SRM filter to transform the input image from the color domain into the noise domain, effectively extracting anomalies, such as global noise inconsistencies from the noise domain. Compared to spatial domain features, the discontinuity of forgery traces within the noise domain aids the network in authenticating the document image. The two branches extract local and global features in the forged document images. Finally, we propose a cross-attention module to fuse local spatial features and global noise features. Extensive experimental results demonstrate that the proposed network achieves F1 scores of 0.819 and 0.948 on the SACP and ICDAR datasets, respectively, with AUC scores of 0.933 and 0.764, outperforming some of the state-of-the-art algorithms.

A convolutional neural network based on noise residual for seam carving detection

Seam carving is a method of resizing images based on content awareness. It can realize image retargeting while retaining the main content of the image. However, it may also be maliciously used to tamper with images, such as changing image semantic content by object removal. Therefore, seam carving detection has become important in image forensics. In this paper, a noise residual-based deep learning method is proposed to detect seam carving images. We try to learn the local noise in-consistency of images to recognize them. Firstly, a noise residual extraction segment is used to learn local noise features in the image, and a noise augmentation module is designed to enrich the features, which leverages the noise features extracted from a steganalysis rich model filter to discover the noise in-consistency between authentic and tampered regions. Then, through the feature dimensionality reduction section, the features are further learned and the size of feature maps are reduced. Finally, the output is obtained through global average pooling and a fully-connected layer. A careful testing strategy is further proposed, which greatly improves the detection performance, especially for seam carving with small scaling ratios. The experimental results demonstrate that our method achieves state-of-the-art performance at various scales, and has good robustness and generalization compared with other methods.

Learning Discriminative Noise Guidance for Image Forgery Detection and Localization

This study introduces a new method for detecting and localizing image forgery by focusing on manipulation traces within the noise domain. We posit that nearly invisible noise in RGB images carries tampering traces, useful for distinguishing and locating forgeries. However, the advancement of tampering technology complicates the direct application of noise for forgery detection, as the noise inconsistency between forged and authentic regions is not fully exploited. To tackle this, we develop a two-step discriminative noise-guided approach to explicitly enhance the representation and use of noise inconsistencies, thereby fully exploiting noise information to improve the accuracy and robustness of forgery detection. Specifically, we first enhance the noise discriminability of forged regions compared to authentic ones using a de-noising network and a statistics-based constraint. Then, we merge a model-driven guided filtering mechanism with a data-driven attention mechanism to create a learnable and differentiable noise-guided filter. This sophisticated filter allows us to maintain the edges of forged regions learned from the noise. Comprehensive experiments on multiple datasets demonstrate that our method can reliably detect and localize forgeries, surpassing existing state-of-the-art methods.

Wavelet Based Inpainting Detection

With the advancement in image editing tools, manipulating digital images has become alarmingly easy. Inpainting, which is used to remove objects or fill in parts of an image, serves as a powerful tool for both image restoration and forgery. This paper introduces a novel approach for detecting image inpainting forgeries by combining DT-CWT with Hierarchical Feature segmentation and with noise inconsistency analysis. The DT-CWT offers several advantages for this task, including inherent shift-invariance, which makes it robust to minor manipulations during the inpainting process, and directional selectivity, which helps capture subtle artifacts introduced by inpainting in specific frequency bands and orientations. By first applying colour image segmentation and then analysing for each segment, noise inconsistency obtained via DT-CW we can identify patterns indicative of inpainting forgeries. The proposed method is evaluated on a benchmark dataset created for this purpose and is compared with existing forgery detection techniques. Our approach demonstrates superior results compared with SOTA in detecting inpainted images. The proposed methodology source code is uploaded here: https://github.com/jmaba/Wavelet-based-inpainting-detection

Dual-branch multi-scale densely connected network for image splicing detection and localization

In the last two decades, numerous methods have been proposed for image tampering forensics, especially when deep learning is leveraged as a powerful feature extraction technique. However, existing deep learning-based methods are not efficient to jointly explore robust representations of tampering traces from both low-level forensics patterns and high-level forensics semantics. Besides, they become unreliable for spliced regions with unknown scales in the evaluation stage. To overcome the above-mentioned limitations, a novel Dual-branch Multi-scale Densely Connected Network (DMDC-Net) is proposed in this work for image splicing detection. DMDC-Net is constructed based on an encoder-decoder framework, which equips a multi-scale dense (MD) module for feature reuse and pixel-wise feature fusion (PFF) module to leverage forensics clues from different data components adaptively. In DMDC-Net, the given input image will be fed into two individual subnets to obtain deep representation maps from RGB and high-frequency components, which contains both visual and noise inconsistencies for further analysis. Besides, the MD module is applied in Encoder of each subnet to make the model jointly learn low-level forensics patterns and high-level semantic representations, which incorporates local and global features by a multi-scale manner of dense connections. The outputs of each subnet are fused in the PFF module to take advantage of their complementarity by trainable weights and get the final detection score map. Experimental results demonstrate the effectiveness of our method and the superior performance than other state-of-the-art methods in CASIA v2.0 and Columbia dataset for different scenarios, such as post-processing operations.

Image Manipulation Localization Using Multi-Scale Feature Fusion and Adaptive Edge Supervision

Image manipulation localization is a technique that can efficiently segment the tampered regions from a suspicious image. Existing work usually trains a detection model by fusing the features from diverse data streams, e.g., noise inconsistency, recompression inconsistency, and local inconsistency. They, however, ignore a fact that not all tampered images contain these data streams. As a result, high feature redundancy may cause a large number of false detection for tampered region. To address this problem, this paper designs an end-to-end high-confidence localization network architecture. First, deep convolutional neural networks are utilized to extract multi-scale feature sets from the RGB streams. We then design a semantic refined bi-directional feature integration module to fully fuse multi-scale adjacent features and significantly enhance feature representation. Subsequently, morphological operations are introduced to extract multi-scale edge information, which can efficiently reduce feature redundancy by generating wider high-resolution edges during image reconstructing. Finally, a deep semantic residual decoder is sequentially re-constructed by spreading deep semantic information into each decoding stage. The proposed method can not only improve the manipulation localization accuracy, but also guarantee the model robustness. Extensive experiments demonstrate that our method can obtain an effective performance in locating forged regions over different large-scale image sets, and outperforms most of state-of-the-art methods with higher localization accuracy and stronger robustness.

Learning Traces by Yourself: Blind Image Forgery Localization via Anomaly Detection With ViT-VAE

Most of existing deep learning models for image forgery localization rely on a large number of high-quality labeled samples for training. The training procedures are performed off-line and without adaptation to the image under scrutiny. In this paper, we propose to perform run-time learning of the forgery traces from the suspicious image itself. To this aim, a Variational Auto-Encoder (VAE) model is learned to reconstruct small cliques of the suspicious image, and those cliques with anomalous larger reconstruction errors are therein identified as forged. To further enhance performance, Vision Transform (ViT) is employed as the VAE encoder, and multi-modal input information is explored by considering noise inconsistency, high-pass residual inconsistency, and edge discontinuity. Evaluation on widely used benchmark datasets shows our method outperforms existing blind methods by a large margin, and is competitive against approaches that use ground-truth for supervised training.

DS-UNet: A dual streams UNet for refined image forgery localization

Nowadays, image manipulation has become very easy with the help of image editing applications. The abuse of tampered images would mislead people’s judgment, or even make the society suffer from instability if without any interference. To address this issue, a dual-stream UNet named DS-UNet is presented in this paper to detect image tampering and localize forgery areas. The DS-UNet employs an RGB stream to extract both high-level and low-level manipulated traces for a coarse localization, and adopts a Noise stream to expose the local noise inconsistency for a refined localization. The lightweight hierarchical fusion way provides the DS-UNet with the ability to perceive tampered objects at different scales for the reason that tampered objects always vary in shapes and sizes. After that, the DS-UNet applies a single decoder to receive the abundant low-level manipulated traces and spatial localization information in the skip connection path. Through the decoder, the object details and spatial dimensions are gradually recovered, and the high-resolution prediction maps are generated. In the comparative analysis, more evaluation indices are introduced than the existing works in order to obtain a more comprehensive assessment. Extensive experiments with satisfying results on five datasets demonstrate its superior performance over state-of-the-art methods.

Detection and localization of image tampering in digital images with fused features

SummaryIn digital forensics, image tamper detection and localization have attracted increased attention in recent days, where the standard methods have limited description ability and high computational costs. As a result, this research introduces a novel picture tamper detection and localization model. Feature extraction, tamper detection, as well as tamper localization are the three major phases of the proposed model. From the input digital images, a group of features like “Scale‐based Adaptive Speeded Up Robust Features (SA‐SURF), Discrete Wavelet Transform (DWT) based Patched Local Vector Pattern (LVP) features, HoG feature with harmonic mean based PCA and MBFDF” are extracted. Then, with this extracted feature strain the “optimized Convolutional Neural Network (CNN)” will be trained in the tamper detection phase. Since it is the key decision‐maker about the presence/absence of tamper, its weighting parameters are fine‐tuned via a novel improved Sea‐lion Customized Firefly algorithm (ISCFF) model. This ensures the enhancement of detection accuracy. Once an image is recognized to have tampers, then it is essential to identify the tamper localization. In the tamper localization phase, the copy‐move tampers are localized using the SIFT features, splicing tampers are localized using the DBN and the noise inconsistency is localized with a newly introduced threshold‐based tamper localization technique. The simulation outcomes illustrate that the adopted model attains better tamper detection as well as localization performance over the existing methods.

Self-Adversarial Training Incorporating Forgery Attention for Image Forgery Localization

Image editing techniques enable people to modify the content of an image without leaving visual traces and thus may cause serious security risks. Hence the detection and localization of these forgeries become quite necessary and challenging. Furthermore, unlike other tasks with extensive data, there is usually a lack of annotated forged images for training due to annotation difficulties. In this paper, we propose a self-adversarial training strategy and a reliable coarse-to-fine network that utilizes a self-attention mechanism to localize forged regions in forgery images. The self-attention module is based on a Channel-Wise High Pass Filter block (CW-HPF). CW-HPF leverages inter-channel relationships of features and extracts noise features by high pass filters. Based on the CW-HPF, a self-attention mechanism, called forgery attention, is proposed to capture rich contextual dependencies of intrinsic inconsistency extracted from tampered regions. Specifically, we append two types of attention modules on top of CW-HPF respectively to model internal interdependencies in spatial dimension and external dependencies among channels. We exploit a coarse-to-fine network to enhance the noise inconsistency between original and tampered regions. More importantly, to address the issue of insufficient training data, we design a self-adversarial training strategy that expands training data dynamically to achieve more robust performance. Specifically, in each training iteration, we perform adversarial attacks against our network to generate adversarial examples and train our model on them. Extensive experimental results demonstrate that our proposed algorithm steadily outperforms state-of-the-art methods by a clear margin in different benchmark datasets.

Two-stream encoder–decoder network for localizing image forgeries

This paper proposes a novel two-stream encoder–decoder network that utilizes both the high-level and the low-level image features for precisely localizing forged regions in a manipulated image. This is motivated by the fact that the forgery creation process generally introduces both the high-level artefacts (e.g., unnatural contrast) and the low-level artefacts (e.g., noise inconsistency) to the forged images. In the proposed two-stream network, one stream learns the low-level manipulation-related features in the encoder side by extracting noise residuals through a set of high-pass filters in the first layer. In the second stream, the encoder learns the high-level image manipulation features from the input image RGB values. The coarse feature maps each encoder are upsampled by the corresponding decoder network to produce the dense feature maps. The dense feature maps of the two streams are concatenated and fed to a final convolutional layer with sigmoidal activation to produce the pixel-wise prediction. We have carried out experimental analyses on multiple standard forensics datasets to evaluate the performance of the proposed method. The experimental results show the efficacy of the proposed method with respect to the state-of-the-art.

Detecting adversarial examples by additional evidence from noise domain

Deep neural networks are widely adopted powerful tools for perceptual tasks. However, recent research indicated that they are easily fooled by adversarial examples, which are produced by adding imperceptible adversarial perturbations to clean examples. Here the steganalysis rich model (SRM) is utilized to generate noise feature maps, and they are combined with RGB images to discover the difference between adversarial examples and clean examples. In particular, a two-stream pseudo-siamese network that fuses the subtle difference in RGB images with the noise inconsistency in noise features is proposed. The proposed method has strong detection capability and transferability, and can be combined with any model without modifying its architecture or training procedure. The extensive empirical experiments show that, compared with the state-of-the-art detection methods, the proposed approach achieves excellent performance in distinguishing adversarial samples generated by popular attack methods on different real datasets. Moreover, this method has good generalization, it trained by a specific adversary can defend against other adversaries effectively.

Forgery Detection in Digital Images by Multi-Scale Noise Estimation.

A complex processing chain is applied from the moment a raw image is acquired until the final image is obtained. This process transforms the originally Poisson-distributed noise into a complex noise model. Noise inconsistency analysis is a rich source for forgery detection, as forged regions have likely undergone a different processing pipeline or out-camera processing. We propose a multi-scale approach, which is shown to be suitable for analyzing the highly correlated noise present in JPEG-compressed images. We estimate a noise curve for each image block, in each color channel and at each scale. We then compare each noise curve to its corresponding noise curve obtained from the whole image by counting the percentage of bins of the local noise curve that are below the global one. This procedure yields crucial detection cues since many forgeries create a local noise deficit. Our method is shown to be competitive with the state of the art. It outperforms all other methods when evaluated using the MCC score, or on forged regions large enough and for colorization attacks, regardless of the evaluation metric.

Fighting Fake News: Two Stream Network for Deepfake Detection via Learnable SRM

Benefitting from the development of deep generative networks, modern fake news generation methods called Deepfake rapidly go viral over the Internet, calling for efficient detection methods. Existing Deepfake detection methods basically use binary classification networks trained on frame-level inputs and lack leveraging temporal information in videos. Besides, the accuracy of these methods will rapidly decrease when processing low-quality data. In this work, we propose a two-stream network to detect Deepfake in video level with the capability of handling low-quality data. The proposed architecture firstly divides the input video into segments and then feeds selected frames of each segment into two streams: The first stream takes RGB information as input and tries to learn the semantic inconsistency. The second stream parallelly leverages noise features extracted by spatial rich model (SRM) filters. Additionally, our experiments found that traditional SRM filters with fixed weights contribute insignificant improvement, we thus design novel learnable SRM filters, which can better fit the noise inconsistency in tampered regions. Segmental fusion and stream fusion are conducted at last to combine the information from segments and streams. We evaluate our algorithm on the existing largest Deepfake dataset FaceForensics++ and the experimental results show that we obtain state-of-the-art performance.

Exposing Speech Transsplicing Forgery with Noise Level Inconsistency

Splicing is one of the most common tampering techniques for speech forgery in many forensic scenarios. Some successful approaches have been presented for detecting speech splicing when the splicing segments have different signal-to-noise ratios (SNRs). However, when the SNRs between the spliced segments are close or even same, no effective detection methods have been reported yet. In this study, noise inconsistency between the original speech and the inserted segment from other speech is utilized to detect the splicing trace. First, noise signal of the suspected speech is extracted by a parameter-optimized noise estimation algorithm. Second, the statistical Mel frequency features are extracted from the estimated noise signal. Finally, the spliced region is located by utilizing a change point detection algorithm on the estimated noise signal. The effectiveness of the proposed method is evaluated on a well-designed speech splicing dataset. The comparative experimental results show that the proposed algorithm can achieve better detection performance than other algorithms.

Forensic image analysis using inconsistent noise pattern

With the advancement of image acquisition devices and social networking services, a huge volume of image data is generated. Using different image and video processing applications, these image data are manipulated, and thus, original images get tampered. These tampered images are the prime source of spreading fake news, defaming the personalities and in some cases (when used as evidence) misleading the law bodies. Hence before relying totally on the image data, the authenticity of the image must be verified. Works of the literature are reported for the verification of the authenticity of an image based on noise inconsistency. However, these works suffer from limitations of confusion between edges and noise, post-processing operation for localization and need of prior knowledge about an image. To handle these limitations, a noise inconsistency-based technique has been presented here to detect and localize a false region in an image. This work consists of three major steps of pre-processing, noise estimation and post-processing. For the experimental purpose two, publicly available datasets are used. The result is discussed in terms of precision, recall, accuracy and f1-score on the pixel level. The result of the presented work is also compared with the recent state-of-the-art techniques. The average accuracy of the proposed work on datasets is 91.70%, which is highest among state-of-the-art techniques.

Time-efficient spliced image analysis using higher-order statistics

Image forgery is gaining huge momentum as changing the content is no longer arduous. One of the leading techniques of this category is image splicing. This technique generates a composite image formed by combining regions of images. Once the image is forged, it becomes nearly impossible for the human expert to substantiate. Hence, for detecting and localizing the spliced region in the forged image, a tool is to be developed which has become the need of the hour. Articles have been reported that one of the key ingredients for such a tool is noise inconsistency, among others. The spliced region contains the non-homogeneous distribution of noise which acts as a feature to localize it. State-of-the-art techniques based on inconsistent noise are suffering from challenges like the requirement of prior knowledge about the image, localization of spliced region and estimation of inconsistent non-gaussian noise. In this paper, a blind local noise estimation technique has been introduced using a fourth-order central moment to localize the spliced region. This paper tries to overcome the challenges of state-of-the-art techniques. Experimental analysis has been done on images of three publicly available datasets. The results are evaluated on pixel level using confusion matrix and some other performance measures. The result of the given approach is compared with previously reported techniques and found better than them.

Splicing Localization Based on Noise Level Inconsistencies in Residuals of Color Channel Differences

With the rapid usage of networking sites there is an enormous increase in image sharing over the internet. At the same time altering or tempering images has become much easier with the availability of photo editing software. Splicing is one of the tempering method, where an object from one image is copied and pasted into another image, is often used with the aim of either getting attention for fun or misleading the general masses. Thus, authenticity of images shared on internet is debatable. Active research is going on in the field of image forensics in order to examine the trustworthiness of the images. Amongst several techniques available for dealing with image splicing, the statistical based methods are gaining attention in research community as it uses image’s local statistics. We propose a simple and effective method based on noise inconsistencies in residuals of Color channel difference for forensic analysis to localize the splicing image forgery. First the image is decomposed in to super pixels and extracted in regular shapes. From each super pixel, three color channel differences are extracted and noise level is estimated on the residual. Finally, the super pixels are clustered into two groups using Farthest Distributed Centroids Clustering (FDCC) method for classifying superpixel as tampered or original. The experimental results show the simplicity and effectiveness of the proposed method over the state of the art.

A fusion framework based on fuzzy integrals for passive-blind image tamper detection

The innate complexity and uncertainty in the domain of image forensics has made the application of fusion technology an obligatory requirement. Fuzzy integrals, that provide a meaningful formalism for combining different information sources has gained limited consideration in image forensics. The current paper presents a fuzzy integral based fusion framework for image tamper detection that can exploit the interaction between multiple forensic tools for collaborative decision making. Four tools that expose traces of semantic manipulation based on specific statistical cues are designed that work cohesively to allow detection of forgeries in single image (copy-move), composite image (splicing) and two generic artefacts (double JPEG compression and noise inconsistency). The measurement level fusion of tool outcome is carried out with Sugneo and Choquet integrals as the underlying aggregation operators. The classification competency of each tool is evaluated on a specialized dataset of forged images with the respective forgery trace. The empirical evaluation of the fusion framework for blind tamper detection is carried out on a combined master dataset comprising the various forgery traces under study. The Choquet integral based aggregation exhibits an enhanced classification competency on comparison with other fusion approaches like Feature level, Fuzzy Logic based on if-else rules, Behaviour Knowledge Space, Dempster-Shafer combination and classifier ensemble architecture based on Wolpkart’s stacked generalization.