Saliency detection via integrating deep learning architecture and low-level features

Abstract

Deep learning methods, with their good performance in semantic representation of different images, have been widely used for saliency detection. Recent saliency detection methods have applied deep learning to obtain high-level features and combined them with hand-crafted low-level features to estimate saliency in the images. However, it is difficult to find the relationship between high-level and low-level features, resulting in incomplete integration framework for saliency detection. In this paper, we novely propose a saliency detection model by integrating high-level and low-level features with joint probability estimation. Firstly, the high-level features from FCN-8S network are used to estimate the probability of each superpixel as foreground or background region. Secondly, low-level features are extracted from each superpixels and clustered via affinity propagation (AP) clustering. The distributions of vectors from different clusters are consequently utilized to calculate the conditional probability of each superpixel as salient object under different assumptions. Thirdly, the joint probability of each superpixel as salient object in foreground or background is computed to compose the saliency map of the whole image. To further improve the uniformity of saliency in the same object region, the structured random forest (SRF) method is used to detect the contour of the image and the saliency of superpixels in homogeneous regions are uniformly merged. The advantage of high-level features in representing semantic regions and that of low-level features in differentiating local details in the image are unified and restrained by the joint probability estimation in the proposed model. Experimental results demonstrate that the proposed method provide better saliency detection performance than the state-of-the-art methods on 5 public databases.