Interactive stereo image segmentation via adaptive prior selection

Abstract

Interactive stereo image segmentation (i.e., cutting out objects from stereo pairs with limited user assistance) is an important research topic in computer vision. Given a pair of images, users mark a few foreground/background pixels, based on which prior models are formulated for labeling unknown pixels. Note that color priors might not help if the marked foreground and background have similar colors. However, integrating multiple types of priors, e.g., color and disparity in segmenting stereo pairs, is not trivial. This is because differing pairs of images and even differing pixels in the same image might require different proportions of the priors. Besides, disparities of natural images are too noisy to be directly used. This paper presents a method that can adaptively determine the proportion of the priors (color or disparity) for each pixel. Specifically speaking, the segmentation problem is defined in the framework of MRF (Markov Random Field). We formulate an MRF energy function which is composed of clues from the two types of priors, as well as neighborhood smoothness and stereo correspondence constraints. The weights of the color and disparity priors at each pixel are treated as variables which are optimized together with the label (foreground or background) of the pixel. In order to overcome the noise problem, the weight of the disparity prior is controlled by a confidence value learned from data. The energy function is optimized by using multi-label graph cut. Experimental results show that our method performs well.