A fast intra mode decision algorithm combining neighboring information for H.264/AVC high profile

Abstract

Intra coding in H.264/AVC can significantly improve the coding efficiency but at the cost of high computational complexity due to the use of rich prediction modes and rate-distortion optimization technique. To reduce the complexity, this paper proposes a fast mode decision algorithm for intra prediction in H.264/AVC high profile. The best prediction mode not only relies on the content of current block but also depends on the neighboring blocks because of the spatial continuity. Based on the reconstructed neighboring pixels, different prediction mode leads to different residual block which is the difference between the original block and predicted block. Since small residual block is advantageous to get good coding including the small distortion and few bit numbers, the feature from the residual block will be used to help do mode decision. Besides the neighboring pixels, the best modes of encoded neighboring blocks are also used to select the candidate modes. The proposed algorithm uses the sum of absolute transformed difference (SATD) to measure the residual block and uses the most probable mode to indicate the influence of the prediction modes of the neighboring blocks. Based on the statistic on mode decision accuracy, the most probable mode and the modes with the smallest residual block are considered as the candidate modes. Meanwhile, the early termination rules which can directly select only one mode are given in the proposed algorithm. Experimental results show that the proposed algorithm effectively reduces the complexity of intra prediction with slight coding performance degradation compared with the full search algorithm.