Generative Adversarial Network-Based Intra Prediction for Video Coding

Abstract

In this paper, a novel intra prediction method is proposed to improve the video coding performance, in which the generative adversarial network (GAN) is adopted to intelligently remove the spatial redundancy with the inference process. The proposed GAN-based method improves the prediction by exploiting more information and generating more flexible prediction patterns. In particular, the intra prediction is modeled as an inpainting task, which is accomplished with the GAN model to fill in the missing part by conditioning on the available reconstructed pixels. As such, the learned GAN model is incorporated into both video encoder and decoder, and the rate-distortion optimization is performed for the competition between GAN-based intra prediction and traditional angular-based intra prediction to achieve better coding performance. The proposed scheme is implemented into the high-efficiency video coding test model (HM 16.17) and the versatile video coding test model (VTM 1.1). The experimental results show that the proposed algorithm can achieve 6.6%, 7.5%, and 7.5% under HM 16.17 and 6.75%, 7.63%, and 7.65% under VTM 1.1 bit rate savings on average for luma and chroma components in the intra coding scenario.