Multi-semantic CRF-based attention model for image forgery detection and localization

Abstract

In this paper, a novel image forgery detection and localization scheme is proposed based on deep convolutional neural network (CNN) which incorporates multi-semantic CRF-based attention model. The proposed method is based on the key observation that the boundary transition artifacts arising from the blending operations are ubiquitous in various image forgery manipulations, which is well characterized in our method with CRF (conditional random field) based attention model by generating attention maps to represent the probability of being forged for each pixel in the image. The resulting attention maps are then used to re-weight the convolutional feature maps for noise suppression and highlighting the informative regions surrounding the forged boundaries, guiding the network to capture more intrinsic features for image forgery rather than manipulation-specific artifacts. Multi-scale attention maps with various semantics are adopted to take full advantages of both the local and global information for improvement of generalization capability, which is then incorporated with a CNN model for effective image forgery detection and localization. Extensive experimental results on several public datasets show that the proposed scheme outperforms or rivals to other state-of-the-art methods in image forgery detection and localization.