Graph learning model for saliency detection in thermal pedestrian videos

Abstract

Thermal imaging is becoming popular recently due to its all-weather and all-time capability. However, video saliency detection, an active topic in computer vision, has not been well studied for thermal videos. This paper proposes an effective saliency detection method for thermal pedestrian videos. Unlike graph-based saliency detection methods with fixed graph structures, we introduce a saliency-guided graph learning model (SGL), which integrates a multi-view graph and saliency to achieve joint graph learning and saliency diffusion. Then, we apply SGL to thermal video saliency detection by constructing the objectness descriptor based motion saliency and a multi-view spatiotemporal graph. Furthermore, a challenging thermal pedestrian video dataset IPV with 67 sequences and 1462 frames in total is built to compensate for the insufficient scale and complexity of existing datasets. Extensive experiments on a public dataset and our newly built dataset demonstrate the superior performance of our method. Compared to 11 state-of-the-art (SOTA) video saliency detection methods, the proposed approach achieves the best performance of 74.0 %, 75.1 %, and 1.8 % in F-measure, S-measure, and MAE respectively, on the IPV dataset. It also achieves better performance than SOTA on the public dataset GTFD with 69.9 %, 75.6 %, and 1.1 %, respectively.