Fast intra mode decision algorithm for HEVC based on dominant edge assent distribution

Abstract

As the latest video coding standard, high efficiency video coding (HEVC) is a successor to H.264/AVC. To improve the coding efficiency of intra coding, HEVC employs a flexible quad-tree coding block partitioning structure and 35 intra prediction modes. The optimal prediction mode is selected through rough mode decision (RMD) and rate distortion optimisation (RDO) process. Due to the huge search space of all of the possible depth levels (CU sizes) and intra prediction modes, intra coding of HEVC is a very time-consuming and complicated process, which limits the application of HEVC. In order to reduce the intra coding complexity, we propose a fast mode decision algorithm for HEVC intra prediction which is based on dominant edge assent (DEA) and its distribution. The four DEAs in the directions of degree 0, 45, 90 and 135 are computed first; then, the dominant edge is decided according to the minimum DEA. Next, a subset of prediction modes in accordance with the dominant edge is chosen for the RMD process. The rule is as follows: When the standard deviation of DEA is distinctly small, we skip the RMD process and take the direct current (DC) mode and planar modes as the candidate modes for the RDO process; when the minimum DEA is distinctly small, we select seven modes as the candidate modes for the RMD process; otherwise, we select 11 modes for the RMD process. Lastly, the prediction unit (PU) size-based number of RDO candidate modes (3 for PU size 4?×?4 and 8?×?8 and 1 for the other PU sizes) is modified according to experimental analysis. Compared with HM 9.1, Shen's proposal and da Silva's proposal, which are two state-of-the-art fast intra mode decision algorithms, the experimental results reveal that the proposed algorithm can save 36.26, 13.85 and 20.81 % coding time on average with a negligible loss of coding efficiency, respectively.