Blind light field image quality measurement via four-stream convolutional neural network

Abstract

In the era of immersive multimedia, light field image quality assessment (LF-IQA) has attracted significant research interest. However, traditional evaluation methods primarily rely on manually extracted features, while deep learning-based evaluation methods face challenges such as overfitting due to small databases and difficulty in considering intrinsic factors affecting light field image (LFI) quality. To address these issues, this paper proposes an innovative four-stream convolutional neural network (CNN) for disentangling LF-IQA. Firstly, a specific patch selection strategy is employed to augment the LFI database to obtain a considerable quantity of relatively high-quality training samples. Subsequently, a specially designed decoupling mechanism is used to decouple high-dimensional light field (LF) data into four different low-dimensional 2D representations. Leveraging a specifically designed four-stream CNN architecture, spatial information, angle information, and spatial-angle correlated information are extracted from LFIs and further used to predict patch scores. Finally, the quality scores of selected images are integrated to obtain the quality score of the LFI. Experimental results on popular LF-IQA databases demonstrate superior performance compared to many typical LF-IQA metrics.