GPU based layer extraction using motion, color and spatial information

Abstract

Layer extraction from video is very important in applications such as recognition, video compression, coding, and multi-object tracking. Similar to many previous approaches, the motion information is the main cue for our method. The main contribution of this study is that it can well deal with the inherent problems of conventional optical flow vectors within textureless areas by taking advantage of image segmentation. After a processing step for those textureless regions, the motion of each segment is described by an affine motion model. Then initial motion segments are clustered to form robust initial layers according to motion similarities. Finally, the assignment of segments to layers is improved by combining with motion and color information. Given the temporal consistency constraints of video, we adopt the maximum a posterior probability framework that uses multiple cues, such as spatial location, motion and color to deal with the rest images of video, which is suitable for an efficient parallel implementation by the graphic processor unit (GPU).