Low-complexity QTMT partition based on deep neural network for Versatile Video Coding

Abstract

Versatile Video Coding (VVC), the newest standard for future video coding, is currently under development. This proposal aimed to improve the encoder performance over the latest standard namely High Efficiency Video Coding, carried with a high increase in coding complexity. The VVC partition structure is mainly based on the quadtree with nested multi-type tree (QTMT) block scheme. Such an improvement leads to a more flexible block partition and promotes a high encoding efficiency, but generates a huge coding complexity. In order to deal with this issue, a fast QTMT intra partition algorithm, based on a deep neural network named Early Terminated Hierarchical Convolution Neural Network, is applied to predict the $$64\times $$ 64 block QT partition structure. The proposed algorithm determines the QTMT partition structure based on the decision of whether to split or skip the corresponding CU, in order to get $$128\times $$ 128 Coding Tree Unit partition architecture. In this paper, the proposed intra partition work achieves a significant speedup in encoding gain that reaches 32.96% in best cases for Ultra High Definition video sequences compared to the reference VVC software VTM-3.0. For all video sequences, 24.49% time saving is reached on average. This improvement comes with an increase of 4.18% and a decrease of 0.18 dB in terms of BDBR and BDPSNR, respectively.