TSFNet: Two-Stage Fusion Network for RGB-T Salient Object Detection

Abstract

Salient object detection (SOD) based on convolutional neural networks has achieved remarkable success. However, further improving the detection performance on challenging scenes (e.g., low-light scenes) requires additional investigation. Thermal infrared imaging captures thermal radiation from the surface of objects. Thus, it is insensitive to lighting conditions and can provide uniform imaging of objects. Accordingly, we propose a two-stage fusion network (TSFNet) integrating RGB and thermal information for RGB-T SOD. For the first fusion stage, we propose a feature-wise fusion module that captures and aggregates united information and intersecting information in each local region of the RGB and thermal images, and then independent decoding is applied to the RGB and thermal features. For the second fusion stage, we propose a bilateral auxiliary fusion module that extracts auxiliary spatial features from the foreground and background of the thermal and RGB modalities. Finally, we use multiple supervision to further improve the SOD performance. Comprehensive experiments demonstrate that TSFNet outperforms 11 state-of-the-art models under various indicators on three RGB-T SOD datasets.