Improving Semantic Image Segmentation With a Probabilistic Superpixel-Based Dense Conditional Random Field

Abstract

Deep convolutional neural networks (DCNNs) have been driving significant advances in semantic image segmentation due to their powerful feature representation for recognition. However, their performance in preserving object boundaries is still not satisfactory. Visual mechanism theory indicates that image segmentation tasks require not only recognition, like DCNNs, but also local visual attention capability. Considering that superpixel is good at grasping detailed local structure, we propose a probabilistic superpixel-based dense conditional random field model (PSP-CRF) to refine label assignments as a post-processing optimization method. First, the well-known fully convolutional networks (FCN) and Deeplab-ResNet are employed to produce coarse prediction probabilistic maps at each pixel. Second, we construct a fully connected CRF model based on the PSP generated by the simple linear iterative clustering algorithm. In our approach, an effective refining algorithm with entropy is developed to convert the pixel-level appearance and position features to the normalized PSP, which works well for CRF. Third, our method optimizes the PSP-CRF to obtain the final label assignment results by employing a highly efficient mean field inference algorithm and some quadratic programming relaxation related algorithms. The experiments on the PASCAL VOC segmentation dataset demonstrate the effectiveness of our methods which can improve the segmentation performance of DCNNs to 82% in mIoU while increasing the computational efficiency by 47%.