Deep Neural Network for Blind Visual Quality Assessment of 4K Content

Abstract

The 4K content can deliver a more immersive visual experience to consumers due to the huge improvement in spatial resolution. However, the high spatial resolution brings a great challenge for video transmission and storage. Therefore, it is necessary to compress or downscale the 4K content before transmitting it to end-users. Existing blind image quality assessment (BIQA) methods are not suitable for 4K contents due to the high spatial resolution and specific distortions caused by upscaling methods. In this paper, we propose a deep learning-based BIQA model for 4K content, aiming to recognize true and pseudo 4K content and meanwhile evaluate their visual quality. Considering the characteristic that high spatial resolution can represent more abundant high-frequency information, we first propose a Grey-level Co-occurrence Matrix (GLCM) based texture complexity measure to select three representative image patches from a 4K image, which can reduce the computational complexity and is proven to be very effective for the overall quality prediction. Then, we extract various visual features from the intermediate layers of the convolutional neural network (CNN) and integrate them into the quality-aware feature representation. Finally, two multilayer perception (MLP) networks are utilized to map the quality-aware features into the class probability and the quality score of each patch respectively. The overall quality index is obtained through averaging the results of all patches. The proposed model is trained via the multi-task learning manner and the uncertainty principle is introduced to balance the losses of the classification and regression tasks. The experimental results show that the proposed model outperforms all compared BIQA metrics on four 4K content quality assessment databases.