Early Termination of Intra Mode Decision Based on Most Probable Mode and SATD for H.264/AVC encoding

Abstract

Intra coding in H.264/AVC can significantly improve the coding efficiency but at the cost of high computational complexity due to the use of rich prediction modes and rate-distortion optimization technique. In this paper, an early termination algorithm of intra mode decision is presented to address the complexity issue in 4×4 intra prediction. The proposed algorithm is motivated by two facts: the most probable mode defined based on the spatial similarity has a high possibility to be the best mode, and a good prediction usually has a small residual block which is measured by the sum of absolute transformed difference (SATD). The most probable mode and two modes with the smallest SATD values are investigated to select the candidate modes by using the proposed early termination rules. Experimental results show that the proposed algorithm effectively reduces the complexity of 4×4 intra prediction while maintaining almost the same coding performance on peak signal-to-noise ratio and bit rate compared with the full search algorithm.