DeLFIQE—A Low-Complexity Deep Learning-Based Light Field Image Quality Evaluator

Abstract

Light field image quality assessment (LF-IQA) has attracted increasing research interests due to the fast-growing demands for immersive media experience. The existing LF-IQA metrics, however, heavily rely on high-complexity statistics-based feature extraction, which is computationally intensive and hardly sustainable in real-time systems. In this article, we propose a low-complexity deep learning-based light field image quality evaluator (DeLFIQE) that achieves both improved accuracy and significantly reduced complexity in measuring LF image quality. On one hand, we propose the novel discriminative epipolar plane image (D-EPI) patches, which reduces the high-complexity 4-D light field IQA processing to lower dimension and lower complexity 2-D IQA processing. On the other hand, we design a distortion-aware multitask learning scheme in DeLFIQE to improve the quality measurement accuracy and generalization. Specifically, alongside the main task that predicts the light field image (LFI) quality, we design an auxiliary classification task that classifies the D-EPI patches based on their distortion types and severity levels, which we believe both are key factors affecting the LFI quality. To the best of our knowledge, this is the first work in LF-IQA with a dedicatedly designed deep multitask convolutional neural network (CNN) model. Our extensive experimental results show that DeLFIQE outperforms the state-of-the-art metrics on representative benchmark datasets with significantly reduced complexity.