Hierarchical prediction-based motion vector refinement for video frame-rate up-conversion

Abstract

Motion-compensated frame-rate up-conversion (MC-FRUC) often exploits either bilateral motion estimation (ME) or unidirectional ME with a fixed block size, which constrains the perceptual quality of up-converted video. In this paper, an advanced MC-FRUC approach is proposed by exploiting hierarchical prediction-based motion vector refinement. To reduce block mismatching in texture regions and color areas, an adaptive multi-layered block matching criterion is designed to extract color and edge information, which is integrated with motion information as constraint term. A hierarchical prediction-based motion vector refinement approach is proposed to obtain more accurate and dense motion vector fields (MVFs). To eliminate the outliers of MVFs, a robust dual-weighted motion vector smoothing scheme is adopted by using both spatial correlation and reliability of neighboring blocks. Experimental results show that the proposed approach has low computational complexity and outperforms state-of-the-art works in both objective and subjective qualities of interpolated frames.