Motion-Compensated Frame Rate Up-Conversion—Part I: Fast Multi-Frame Motion Estimation

Abstract

Motion-compensated frame rate up-conversion is used to convert video/film materials of low frame rates to a higher frame rate so that the materials can be displayed with smooth motion and high-perceived quality. It consists of two key elements: motion estimation and motion-compensated frame interpolation. It requires accurate motion trajectories to ensure quality results and low computational cost to ensure practical applications. This paper presents a novel motion estimation algorithm that combines the accuracy of maximum a posteriori probability (MAP) estimation with the speed of hierarchical block-matching algorithm (BMA). This MAP estimation uses three consecutive pictures, instead of the conventional two, and one previously estimated motion field to exploit the temporal correlation between motion fields and to determine motion in occluded areas. The optimization of the MAP estimation is performed using full-search and implemented by means of look-up tables. The full search ensures that the optimization converges to the global minimum, while the look-up tables dramatically reduce the computational cost. Experimental results show that the proposed algorithm provides motion trajectories that are much more accurate than those obtained using either the full-search BMA or hierarchical BMA alone. Also, it is much faster than the full-search BMA.