Perceptual Quality Enhancement with Multi-scale Deep Learning for Video Transmission: A QoE Perspective

Abstract

With the development of mobile Internet technologies, wireless communication is facing huge challenges under the explosive growth of multimedia data, e.g. video conferences, online education. This makes it difficult to guarantee the communication quality where communication resources (bandwidth, channel, etc.) are limited. In this paper, we propose an image enhancement method to transform blurred images into images with high perceptual quality. The proposed method serves as a post-processing part for communication systems and is incorporated into the receiver. Specifically, we learn the prior of high quality images using a collected dataset. We train a neural network to accomplish this task and adopt a multi-scale perceptual loss as the objective, which is more consistent with the quality of experience (QoE). To validate the proposed method, we train our model on a large dataset with both blurred images and high quality images. Experimental results show that, using a pre-collected dataset with high quality images, the proposed approach can effectively restore the blurred images.