No-reference stereoscopic image quality assessment using a multi-task CNN and registered distortion representation

Abstract

Scene discrepancy between the left and right views presents more challenges for image quality assessment (IQA) of stereoscopic images as opposed to monocular ones. Existing no-reference stereoscopic IQA (NR-SIQA) metrics cannot achieve a good performance on asymmetrically distorted stereoscopic images. In this paper, we propose an NR-SIQA index that first addresses scene discrepancy by means of image registration. It then uses a registered distortion representation based on the left and registered right views to represent the distortion in the stereoscopic image. Because different distortion types influence image quality differently, a multi-task convolutional neural network (CNN) is employed to learn image quality prediction and distortion-type identification simultaneously. We first design a one-column multi-task CNN model, that learns from the registered distortion representation. Then, we extend the one-column model to a three-column model, which also learns from the left and right views. Our experimental results validate the effectiveness of the proposed registered distortion representation and multi-task CNN architecture. The proposed one- and three-column models outperform the state-of-the-art NR-SIQA metrics, especially for asymmetrically distorted stereoscopic images.