Video Segmentation with Motion Smoothness

Abstract

In this paper, we present an interactive and intuitive graph-cut-based video segmentation system while taking both color and motion information into consideration with a stroke-based user interface. Recently, graph-cut-based methods become prevalent for image and video segmentation. However, most of them deal with color information only and usually failed under circumstances where there are some regions in both foreground and background with similar colors. Unfortunately, it is usually hard to avoid, especially when the objects are filmed under a natural environment. To make such methods more practical to use, we propose a graph-cut-based video segmentation method based on both color and motion information, since the foreground objects and the background usually have different motion patterns. Moreover, to make the refinement mechanism easy to use, the strokes drawn by the user are propagated to the temporal-spatial video volume according to the motion information for visualization, so that the user can draw some additional strokes to refine the segmentation result in the video volume. The experiment results show that by combining both color and motion information, our system can resolve the wrong labeling due to the color similarity, even the foreground moving object is behind an occlusion object.