Occlusion-Aware Motion Layer Extraction Under Large Interframe Motions

Abstract

Extracting motion layers from videos is an important task for video representation, analysis, and compression. For videos with large interframe motions, motion layer extraction is challenging in two respects: the estimation of large disparity motions and the awareness of large occluded regions. In this paper, we propose an effective method for motion layer extraction under large disparity motions. To robustly estimate large displacement motions, we have developed an efficient voting-based method that estimates planar homographies from sparse feature matches. To handle occlusions, we first integrate color and motion consistency into a Markov random field framework to achieve per-pixel assignment with occlusion detection. Then, we perform motion-color segmentation and an earth mover's distance-based comparison to determine motion labels for occluded pixels. Experimental results show that our proposed method achieves good performance in automatically extracting multiple moving objects under large disparity motions while maintaining a low computational cost.