Deep blind image quality assessment based on multiple instance regression

Abstract

In recent years, the research of image quality assessment (IQA) based on deep learning, especially convolutional neural network (CNN), has made rapid development. The most widely used way to build a CNN-based IQA model is to divide image into patches and conduct a patch-based training. However, this method has a critical defect that the local ground-truth for each patch is not available. This defect leads to a sub-optimal result. To address this issue, we propose a novel deep blind IQA algorithm under the multiple instance regression (MIR) framework. Specifically, we assume each instance (patch) has a certain probability to be the prime instance of the bag (image), which is responsible for the bag label. Then the global quality score of the bag can be computed by the weighted summation of the local quality scores of the instances, where the weights are the probabilities of the instances to be the prime one. To simplify the training procedure, we propose an EM-like algorithm, called conditional EM algorithm, to train the deep MIR IQA model. Experimental results show that the proposed deep MIR IQA algorithm performs better than the traditional deep blind IQA algorithm. Moreover, the proposed algorithm can be used as a unified framework to improve the performance of any patch-based deep IQA models.