Fast adaptive inter-prediction mode decision method for H.264 based on spatial correlation

Abstract

H.264 defines 7 different coding modes for macroblocks (MBs) in P slices. In order to achieve a coding performance as high as possible, the H.264 encoder calculates rate distortion costs of all possible modes to determine the best mode of a MB. The computation complexity is so large that make it difficult to be used in practical applications especially in real time environment. In this paper, a fast adaptive intermodes decision method is proposed to reduce the complexity of H.264 encoder. Firstly the candidate inter modes used in rate distortion optimization can be limited in a small mode group (MG) by using the characteristics of the motion compensated residual image. Then the two most probable modes of the chosen MG are obtained on the basis of the modes of the up MB and the left MB. By calculating and comparing the rate distortion cost of the two modes, the optimum mode of the MB is determined. The overlapped mode groups and dynamic adjusted thresholds adopted in the proposed method can make the best mode lie within the chosen MG with great possibility which leads to extensive computation reduction with acceptable loss in quality. The experimental results show that the proposed method can save the encoding time up to 65% on average with -0.24dB performance degradation.