Complexity Reduction on HEVC Intra Mode Decision with modified LeNet-5

Abstract

The HEVC (H.265) standard was finalized in April 2013, currently being as the prevalent video coding standard. One key contributor to the performance gain over H.264 is the intra prediction that extended a large number of prediction directions on various sizes of prediction units (PUs), thus at a cost of very high computational complexity. When HEVC has been emerged, several fast Intra prediction and Coding Unit (CU) size decision algorithms are being developed for practical applications. Actually, these two components would cost around 60% to 70% encoding time in the all-intra HEVC encoding. In this paper, a novel CNN-based solution is proposed and evaluated. The main idea is to elect a smallest set of adequate intra directions using our modified LeNet-5 CNN model, thus reduce the computational complexity of (further) rate distortion optimization to a tolerable limit. Besides, two filters are employed: the edge strength extractor in [4] and the early terminated CU partition in [7] to skip most of the unlikely directions and to decrease the number of CUs, respectively. The experimental results demonstrate that the proposed method provides a decrease of up to 66.59% computation with a slightly increase in the bit-rate (1.1% on average) and a little reduction of picture quality (0.109% on average in PSNR) at most.