Learning graph affinities for spectral graph-based salient object detection

Abstract

In this paper, we propose a novel method for learning graph affinities for salient object detection. First, we assume that a graph representation of an image is given with a predetermined connectivity rule and representative features for each of its nodes. Then, we learn to predict affinities related to this graph, that ensures a decent salient object detection performance, when used with a spectral graph based foreground detection method. To accomplish this task, we modify convolutional kernel networks (CKNs) for graph affinity calculation, which were originally proposed to predict similarities between images. Subsequently, we employ a spectral graph based salient object detection method – Extended Quantum Cuts (EQCut) – using these graph affinities. We show that the salient object detection error of such a system is differentiable with respect to the parameters of the CKN. Therefore, the proposed system can be trained end-to-end by applying error backpropagation and CKN parameters can be learned for salient object detection task. The comparative evaluations over a large set of benchmark datasets indicate that the proposed method has an insignificant computational burden on, but significantly outperforms the baseline EQCut – which uses color affinities – and achieves a comparable performance level with the state-of-the-art in some performance measures.