Content-aware rate control scheme for HEVC based on static and dynamic saliency detection

Abstract

High efficiency video coding (HEVC) greatly outperforms previous standards H.264/AVC in terms of coding bit rate and video quality. However, it does not take into account the human visual system (HVS), that people pay more attention to specific areas and moving objects. In this paper, we present a content-aware rate control scheme for HEVC based on static and dynamic saliency detection. The proposed strategy mainly consists of three techniques, static saliency detection, dynamic saliency detection, and adaptive bit rate allocation. Firstly, we train a deep convolution network (DCN) model to extract the static saliency map by highlighting semantically salient regions. Compared to traditional texture-based or color-based region of interest (ROI) extraction techniques, our models are more in line with the HVS. Secondly, we develop a moving object segmentation technique to automatically extract the dynamic salient regions for each frame. Furthermore, according to the fusion saliency map, a coding tree unit (CTU) level bit control technique is exploited to realize flexible and adaptive bit rate allocation. As a result, the quality of salient regions is improved by allocating more bits, while allocating fewer bits to the non-salient regions. We verified the proposed method on both the JCT-VC recommended data set and eye-tracking data set. Experiment results show that the PSNR of salient regions can improve by an average of 1.85 dB without adding bit rate burden, which significantly improves the visual experience.