An effective segmentation cue for moving object segmentation from a moving camera

Abstract

To overcome the difficulty in segmenting moving objects from a video sequence taken by a freely moving camera, we propose a new segmentation cue based on a joint spatial-color representation for the foreground and background appearances. Given the segmentation result in the previous frame, two sample sets can be extracted for foreground and background, respectively. We transform the spatial locations of the samples in the two sets towards the current frame, then use these transformed samples to evaluate the foreground and background likelihood maps, and combine these maps to form a likelihood ratio map which is further exploited as a segmentation cue and integrated into a conditional random field energy function. The total conditional random field energy is minimized by the graph cut, leading to a binary mask of moving objects for each video frame. We validate the proposed segmentation cue using several video sequences taken by hand-held cameras in outdoor urban scenes and the results show the effiency of the segmentation cue.