Combining pixel-level and patch-level information for segmentation

Abstract

The image patch has been widely used in many applications, such as image denoising and segmentation, in which the patch-level information is utilized by comparing the similarities between pairwise patches. Nevertheless, the similarities between patches are not accurate enough for irregular shaped objects, which may lead to over-smoothness for the details around image boundary regions. Most graph cut based methods introduce the patch-level information into the boundary term of the energy function, which, however, would dramatically increase the time complexity due to the similarity computations of all pairwise patches. To overcome above issues and effectively utilize the patch information in graph cut model, in this paper, we first utilize the Gaussian mixture model (GMM) to construct the structural features of patches. Therefore, the patch-level information can be effectively obtained based on the learning of GMM. Then we combine the pixel-level information and patch-level information together to further improve the segmentation accuracy for the details around boundary regions. Finally, the combination information is introduced into the region energy term of the graph cut framework. Experimental results on various images from the Berkeley and MSRC data sets demonstrate the effectiveness of the proposed method.