Using XAI for Deep Learning-Based Image Manipulation Detection with Shapley Additive Explanation

Abstract

In the arena of image forensics, detecting manipulations in an image is extremely significant because of the use of images in different fields. Various detection techniques have been suggested in the literature that are based on digging out the features from images to unveil the traces left by manipulation operations. In this paper, a deep learning-based approach is proposed in which a residual network is used to learn deep, complex features from preprocessed images for classification into authentic and forged images. There is statistical symmetry in similar types of images and asymmetry in different types of images. The proposed scheme can highlight the statistical asymmetry between authentic and forged images. In the proposed scheme, firstly, an RGB image is analyzed for different JPEG compression levels. The obtained difference between the error levels is used to extract enhanced LBP code. Then, the scale- and direction-invariant LBP (SD-LBP) code is transformed into SD-LBP feature maps to feed to a deep residual network. Next, the concept of explainable artificial intelligence (XAI) is used to help provide explanations and interpret the output, thereby raising the credibility of the proposed approach. The unique feature selection approach employed is the kernel SHAP method, which is focused on the Shapley values. This technique is used to pinpoint the specific characteristics that are responsible for the aberrant behavior of the forged images dataset. Later, the deep learning-based model is trained and validated using these feature sets. A pre-activation version of ResNet-50 architecture is used that achieved an accuracy of 99.31%, 99.52%, 98.05%, and 99.10% on CASIA v1, CASIA v2, IMD 2020, and DVMM datasets, respectively. The capability of the pretrained residual network and rich textural features, which are scale- and direction-invariant, helps to expand the detection accuracy of the proposed approach. The results confirmed that the method either produced competitive results or outperformed existing methods.