A Contribution-Aware Noise Feature representation model for image manipulation localization

Abstract

Noise feature modules play a prevalent role in image manipulation localization. However, existing approach involves utilizing a specific noise feature module tailored to address distinct categories of tampering procedures, limiting its adaptability across diverse scenarios. This limitation becomes particularly pronounced in image manipulation localization where a singular noise module struggles to comprehensively represent all potential tampering types, given the inherent uncertainty surrounding the nature of tampering. A novel Contribution-Aware Noise Feature (CANF) representation model, specifically designed to tackle the intricacies of image tamper noise, is introduced in this paper. The proposed model integrates existing noise feature modules, autonomously enhancing their capabilities. This enhancement not only contributes significantly to image manipulation localization during the inference phase but also enables a quantitative evaluation of the distinct contributions of various noise modules to the inference results. To optimize the utilization of image information, the Red–Green–Blue (RGB) and CANF data are fused, leading to the development of a standardized image manipulation localization architecture. This architecture exhibits heightened efficiency in harnessing diverse types of information, thereby enhancing the overall performance of image manipulation localization. Experimental results demonstrate that our approach, alongside established methods IF-OSN, MSFF, PCL, and NCL, surpasses the state-of-the-art NCL by 8.5% and 5.4% concerning average F1 and AUC metrics across four standard image manipulation localization datasets: NIST16, CASIAv1, Columbia, and Coverage. In addition, we comprehensively explore the impact of various noise characterization modules on different types of tampering and the fact that Late Fusion is more conducive to image tampering localization.