Large-scale semantic co-labeling of image sets

Abstract

As evidenced by video segmentation and cosegmentation approaches, exploiting multiple images is key to the success of visual scene understanding. With the availability of increasingly large sets of images, there is a clear need for methods that can efficiently analyze the similarities and structure across huge numbers of image pixels. Furthermore, to make effective use of this data, these similarities should not just be considered locally between neighboring pixels, but between all pairs of pixels across all images. In this paper, we tackle this challenging scenario by introducing a semantic co-labeling approach that performs efficient inference in a fully-connected CRF defined over the pixels, or superpixels, of an image set. Our experimental evaluation demonstrates that our approach yields improved accuracy while coming at no additional computation cost compared to performing segmentation sequentially on individual images. Furthermore, our formulation lets us perform inference over ten thousand images in a matter of seconds.