Scene Parsing with Deep Features and Spatial Structure Learning

Abstract

Conditional Random Field (CRF) is a powerful tool for labeling tasks, and has always played a key role in object recognition and semantic segmentation. However, the quality of CRF labeling depends on selected features, which becomes the bottleneck of the accuracy improvement. In this paper, our semantic segmentation problem is calculated in the same way within the framework of Conditional Random Field. Different from other CRF-based strategies, which use appearance features of image, revealing only little information, we combined our framework together with deep learning strategy, such as Convolutional Neural Networks (CNNs), for feature extraction, which have shown strong ability and remarkable performance. This combination strategy is called deep-feature CRF (dCRF). Through dCRF, the deep informantion of image is illustrated and gets ultilized, and the segmentation accuracy is also increased. The proposed deep CRF strategy is adopted on SIFT-Flow and VOC2007 datasets. The segmentation results reveals that if we use features learned from deep networks into our CRF framework, the performance of our semantic segmentation strategy would increase significantly.