Manipulation Classification for JPEG Images Using Multi-Domain Features

In-Jae Yu,Wonhyuk Ahn,Myung-Joon Kwon,Heung-Kyu Lee,Seung-Hun Nam

doi:10.1109/access.2020.3037735

Abstract

Image forensics comprises the analyses and classifications of manipulations that have been applied to images. The ability to classify various manipulations that have been employed in the process of forgery is essential. Techniques to identify multiple manipulations applied to uncompressed images have been reported thus far, but the forensic approach for JPEG images compressed with various qualities has not been proposed. In this paper, we propose the manipulation classification network (MCNet) to exploit multi-domain features of the spatial, frequency, and compression domains. The proposed MCNet learns several forensic features for each domain through a multi-stream structure and distinguishes manipulations by comprehensively analyzing the fused features. Our work jointly considers visual artifacts caused by image manipulations and compression artifacts due to JPEG compression; therefore, rich forensic features can be explored and learned in the training phase. To enable forgery analysis in the real-world environment, data were generated based on twenty types of manipulation algorithms and various compression parameters. To demonstrate the effectiveness of the proposed MCNet, extensive experiments were conducted using state-of-the-art baselines. Compared to these baselines, our proposed method outperforms in terms of multi-class manipulation classification. In addition, we experimentally proved that the fine-tuned model based on the multi-class manipulation task was effective for different forensic tasks such as DeepFake detection or integrity authentication of JPEG images.

Highlights

Editing images has become easier than ever before because of the development and distribution of smartphones with high-end cameras and various editing applications
We propose a manipulation classification network (MCNet) that classifies the various types of image manipulations on the premise that JPEG compression has been applied
NETWORK ARCHITECTURE To classify the manipulation traces remaining after JPEG compression, we propose the convolutional neural network (CNN)-based forensic system, referred to as MCNet, for learning and exploring the multidomain features

Summary

INTRODUCTION

Editing images has become easier than ever before because of the development and distribution of smartphones with high-end cameras and various editing applications. As the importance of manipulation classification emerges, high-performance CNN-based approaches [2], [3], [5] for identifying manipulated images by learning the generated data through various manipulations and parameters have been proposed These approaches are suitable for capturing the manipulations applied to uncompressed images, but have limitations in that they only consider image compression (i.e., JPEG compression) as one of the manipulation types. Both visual artifacts for image manipulation and compression artifacts due to JPEG compression jointly; rich forensic features of real-world manipulations can be explored and learned. Compared to the available baselines [3], [5], [25], [28], the proposed MCNet, which effectively learns and explores visual and compression artifacts based on multi-domain features, achieves state-of-the-art performance.

BACKGROUND

EXPERIMENTAL RESULTS

LOCALIZATION OF MANIPULATION

CONCLUSION