Self-supervised image co-saliency detection

Abstract

Self-supervised learning has been shown to be highly effective to avoid the expensive cost of collecting and annotating large-scale datasets. In this paper, we explore its use in image co-saliency detection task which detects and segments the co-concurrent and conspicuous patterns in one image group, and a self-supervised framework is presented that contains three novel module designs. Specifically, we first design an unsupervised graph clustering algorithm to generate an initial detection result from extracted semantical features of convolutional networks. Then, the post-processed output is taken as the pseudo labels of the second module that learns a neural network for more accurate sample affinity estimation, followed by a manifold ranking based label smoothing. Finally, with the smoothed labels as supervision, we design a saliency network that aggregates a hierarchy of multi-scale features to predict final optimized co-saliency maps. In addition, the proposed framework can be readily extended to image saliency object detection task when giving a single image as input. Extensive evaluations on three co-saliency and five saliency object detection benchmark datasets demonstrate that the proposed framework achieves favorable performance against a variety of supervised and unsupervised state-of-the-arts.