HEVC Intra Frame Coding Based on Convolutional Neural Network

Abstract

The HEVC standard offers high performance with a lower bitrate for intra frame coding, but still requires many bits. An alternative intra frame coding framework based on convolutional neural network (CNN) is proposed in this paper. Two CNN models, simplified CNN (S-CNN) and complicated CNN (C-CNN), were designed and trained to improve the coding performance of HEVC. In intra frame coding, the trained CNN predicts the residual for reconstructed blocks to enhance visual quality. Due to the high computational complexity of CNN in HEVC encoding, we further explore the tradeoff between computational complexity and coding performance. For S-CNN, an early termination mechanism is proposed to further reduce the HEVC encoding complexity. With regard to C-CNN, a GPU-based heterogeneous architecture is proposed to accelerate CNN processing. Experimental results show that the proposed method with S-CNN achieves bitrate savings of 3.1% with a 37% increase in time cost, and 2.8% in bitrate savings with only a 23% increase in time cost when applying the early termination algorithm. In the case of C-CNN, the execution time of CNN can reach increases in speed of 8.8–17.8 $\times$ and a bitrate savings of up to 5.1% on average.