Video Saliency Detection via Graph Clustering With Motion Energy and Spatiotemporal Objectness

Abstract

We present a novel, robust estimation method to distinguish salient objects from complicated, dynamic backgrounds in videos. In this method, we propose a novel approach to model motion energy based on motion magnitude, motion orientation, gradient flow field, and spatial gradient of the video frame. Furthermore, an effective spatiotemporal objectness map is also proposed to estimate a compact object-like region in the current video frame leveraging both the objectness proposals and the saliency map of the previous frame. Then the current video frame is oversegmented into the granularity of superpixels using the simple linear iterative clustering algorithm. Each superpixel is designated as a node of a graph. The similarity between adjacent superpixels will be assigned as the weight of an edge that connects these two nodes. The feature values of motion energy and spatiotemporal objectness within each superpixel will be averaged respectively, and used to graphically cluster similar superpixels to form the detected salient object. Extensive experiments comparing this proposed new method against twelve existing salient object detection (SOD) methods have been performed using the benchmark datasets unconstrained video saliency detection and densely annotated video segmentation. Superior performance of this proposed SOD method has been observed through three well-known performance metrics: precision-recall curves, F-measure curves, and the mean absolute error.