Fully Connected Network for HEVC CU Split Decision equipped with Laplacian Transparent Composite Model

Abstract

High Efficiency Video Coding (HEVC) improves rate distortion (RD) performance significantly, but at the same time is computationally expensive due to the adoption of a large variety of coding unit (CU) sizes in its RD optimization. In this paper, we investigate the application of fully connected neural networks (NNs) to this time-sensitive application to improve its time complexity, while controlling the resulting bitrate loss. Specifically, four NNs are introduced with one NN for each depth of the coding tree unit. These NNs either split the current CU or terminate the CU search algorithm. Because training of NNs is time-consuming and requires large training data, we further propose a novel training strategy in which offline training and online adaptation work together to overcome this limitation. Our features are extracted from original frames based on the Laplacian Transparent Composite Model (LPTCM). Experiments carried out on all-intra configuration for HEVC reveal that our method is among the best NN methods, with an average time saving of 38% and an average controlled bitrate loss of 1.6%, compared to original HEVC.