Convolutional neural network-based optimal R-λ intra rate control in Versatile Video Coding

Abstract

Since intra frame rate control has no prior information to rely on, it is difficult to obtain suitable initial rate control parameters for achieving optimal intra coding. The optimal bit allocation and λ decision are proposed based on a convolutional neural network (CNN) in R-λ intra frame rate control. First, according to the rate distortion (R-D) and rate lambda (R-λ) relationships of the coding tree unit (CTU), a CNN with four outputs is designed to predict the key parameters of R-D and R-λ curves. Second, with the R-D optimization of a frame, the coding optimization equation of the frame λ and the target bit can be built to derive the optimal CTU bit allocation. Last, a suitable CTU λ can be obtained based on the allocated bit and predicted R-λ curve for a CTU to minimize the total distortion of the current frame. The experimental results show that the proposed algorithm can improve the intra frame rate control performance by 0.31 dB with a good rate control accuracy of 4.76%.