A Fast Zero-Quantized Percentage Model for Video Coding with RDO Quantization

Abstract

In video coding, the percentage of zero-quantized coefficients, denoted by ρ, is directly determined by the quantization algorithm adopted. ρ-domain rate distortion (RD) modeling is widely employed to optimize the implementation algorithm for customizable modules such as rate control and mode decision etc. How to calculate or estimate ρ according to quantization algorithm is the first step task for ρ-domain RD modeling. There are two typical quantization algorithm, soft-decision quantization such as dead-zone, and soft-decision quantization such as rate distortion optimized quantization (RDOQ). RDOQ is more frequently employed in the latest video encoders compared with deadzone quantization due to its inspiring coding performance. In HDQ based video encoder, ρ can be easily obtained by simply rounding. However, it is computation-intensive to calculate ρ in video encoder with RDOQ, in which complicated trellis search is employed. This paper focus on developing estimation model for quickly estimating ρ for RDOQ based video coding. The contribution of this article is as follows: First, this paper develops the ρ model adaptively according to an adaptive deadzone offset model, which is modeled by imitating the behavior of RDOQ. Second, an accurate ρ model is adaptively built offline as function of weighted SATD (sum of absolute transformed distortion) denoted by WSATD, quantization step size q, and average WSATD/q estimated from ensemble. The weight in WSATD is adaptively determined according to the adaptive offset to simulate the behavior pattern of RDOQ as much as possible. Experimental results verify that the proposed model can quickly and accurately predict the ρ results of RDOQ with moderate implementation complexity. The proposed ρ model can be employed to estimate the percentage of zero quantized coefficients which can be used for fast all-zero detection and ρ domain rate distortion modeling in RDOQ based HEVC video encoder.