Distinguishing natural and computer generated images using Multi-Colorspace fused EfficientNet

Abstract

The problem of distinguishing natural images from photo-realistic computer generated ones either addresses natural images versus computer graphics or natural images versus GAN images at a time. But in a real-world image forensic scenario, it is highly essential to consider all categories of image generation since in most cases image generation is unknown. We for the first time to our best knowledge, approach the problem of distinguishing natural images from photo-realistic computer generated images as a three-class classification task classifying natural, computer graphics and GAN images. For the task, we propose a Multi-Colorspace fused EfficientNet model by parallelly fusing three EfficientNet networks that follow transfer learning methodology where each of the three networks operates in a different colorspace, one in RGB, the other in LCH and the last in HSV that are chosen after analyzing the efficacy of various colorspace transformations in this image forensics problem. Our model outperforms the baselines in terms of accuracy, robustness towards post-processing and generalizability towards other datasets. We conduct psychophysics experiments to understand how accurately humans can distinguish natural, computer graphics and GAN images where we could observe that humans find difficulty in classifying these images, particularly the computer generated images, indicating the necessity of computational algorithms for the task. We also analyze the behavior of our model through visual explanations to understand salient regions that contribute to model’s decision making and compare with manual explanations provided by human participants in the form of region markings where we could observe similarities in both the explanations indicating powerful nature of our model to take the decisions meaningfully.