Video Salient Object Detection via Robust Seeds Extraction and Multi-graphs Manifold Propagation

Abstract

Video salient object detection aims at distinguishing the salient objects from the complex background and highlighting them uniformly in the spatiotemporal domain, which still suffers from the interference of the complicated dynamic background in unconstrained videos. To address this problem, we propose a novel coarse-to-fine spatiotemporal salient object detection method. Specifically, we first model a novel motion energy to exclude the motion noise by exploiting the motion magnitude and motion orientation. Then, a supervoxel-level inter-frame graph model is constructed for each pair of adjacent frames independently, and a robust graph clustering-based saliency seed generation method is proposed to produce a coarse saliency map. Furthermore, the supervoxel-level inter-frame graph model is reconstructed by considering the regional spatiotemporal consistency constraint based on the coarse saliency map. The prior information obtained from pixel clustering is also taken into account to optimize the weight of the inter-frame graph model. Finally, a multi-graphs saliency propagation method is exploited under the manifold regularization framework by fusing the motion energy and appearance feature to refine the coarse saliency map. The extensive experiments on two widely used datasets validate the effectiveness and superiority of the proposed method against 13 state-of-the-art methods in terms of PR-curves, scores of S-measure, $F_{\beta }$ , and MAE.