Multi-task Deep Learning for No-Reference Screen Content Image Quality Assessment

Abstract

The past decades have witnessed growing development of image quality assessment (IQA) for natural images (NIs). However, since screen content images (SCIs) exhibit different visual characteristics from the NIs, few of NIs-oriented IQA methods can be directly applied on SCIs. In this paper, we present a quality prediction approach specially designed for SCIs, which is based on multi-task deep learning. First, we split a SCI into \(32\times 32\) patches and design a novel convolutional neural network (CNN) to predict the quality score of each SCI patch. Then, we propose an effective adaptive weighting algorithm for patch-level quality score aggregation. The proposed CNN is built on an end-to-end multi-task learning framework, which integrates the histogram of oriented gradient (HOG) features prediction task to the SCI quality prediction task for learning a better mapping between input SCI patch and its quality score. The proposed adaptive weighting algorithm for patch-level quality score aggregation further improves the representation ability of each SCI patch. Experimental results on two-largest SCI-oriented databases demonstrate that our proposed method is superior to the state-of-the-art no-reference IQA methods and most of the full-reference IQA methods.