Low-Complexity Rate-Distortion Optimization for HEVC Encoders

Abstract

Typical high efficiency video coding (HEVC) encoders use rate-distortion optimization (RDO) to select the best coding parameters among numerous candidates to achieve high compression efficiency. However, calculating the rate-distortion cost for each coding parameter entails high computational complexity, which limits the real-time application of these encoders. In this study, time-consuming processes, such as reconstruction and entropy coding, are removed during the calculation of the full RD cost to reduce the complexity of the encoder. Instead, RD costs are estimated for transformed coefficients by using the RD cost model, which is made up of a distortion model based on the distribution features of the transformed coefficients, a residual rate model based on the characteristics of the entropy coding module, and a simple and convenient header rate model. In addition to the full RD cost, typical encoders also adopt fast RD cost, which is composed of the sum of absolute transform differences (SATD) and header bit. To further reduce encoding complexity, three simplified SATD (SSATD) methods based on DCT and DST features are proposed to serve as substitutes for SATD in fast RD cost, thus reducing the computational complexity of SATD calculation. Experimental results show that 35.53%, 14.57%, and 14.37% of coding time can be reduced respectively under AI, RA and LDP configurations with negligible RD performance loss when applying the proposed RD cost model and the SSATD methods to HM16.7. Besides, the proposed methods also can reduce 4%-9% of encoding time and exhibits the same RD performance as state-of-the-art methods.