Learning to predict the quality of distorted-then-compressed images via a deep neural network

Abstract

Being captured by amateur photographers, reciprocally propagated through multimedia pipelines, and compressed with different levels, real-world images usually suffer from a wide variety of hybrid distortions. Faced with this scenario, full-reference (FR) image quality assessment (IQA) algorithms can not deliver promising predictions due to the inferior references. Meanwhile, existing no-reference (NR) IQA algorithms remain limited in their efficacy to deal with different distortion types. To address this obstacle, we explore a NR-IQA metric by predicting the perceptual quality of distorted-then-compressed images using a deep neural network (DNN). First, we propose a novel two-stream DNN to handle both authentic distortions and synthetic compressions and adopt effective strategies to pre-train the two branches of the network. Specifically, we transfer the knowledge learned from in-the-wild images to account for authentic distortions by utilizing a pre-trained deep convolutional neural network (CNN) to provide meaningful initializations. Meanwhile, we build a CNN for synthetic compressions and pre-train it on a dataset including synthetic compressed images. Subsequently, we bilinearly pool these two sets of features as the image representation. The overall network is fine-tuned on an elaborately-designed auxiliary dataset, which is annotated by a reliable objective quality metric. Furthermore, we integrate the output of the authentic-distortion-aware branch with that of the overall network following a two-step prediction manner to boost the prediction performance, which can be applied in the distorted-then-compressed scenario when the reference image is available. Extensive experimental results on several databases especially on the LIVE Wild Compressed Picture Quality Database show that the proposed method achieves state-of-the-art performance with good generalizability and moderate computational complexity.