Frequency-aware feature aggregation network with dual-task consistency for RGB-T salient object detection

Abstract

RGB-Thermal salient object detection (SOD) aims to merge two spectral images to segment visually appealing objects. Current methods primarily extract salient object information in the pixel perspective. However, biological and psychological research indicates notable frequency sensitivity of the human visual system (HVS). The high-frequency (HF) and low-frequency (LF) information in images are processed by different neural channels, which has been overlooked in SOD. In this study, we argue that the objective of RGB-T SOD is not only to enhance feature representation in the pixel-aware but also to emulate human visual mechanisms. To our best knowledge, we explore RGB-T SOD from the frequency perspective for the first time. Specifically, we first present a frequency-aware multi-spectral feature aggregation module (FMFA) to exploit the separability and complementarity of frequency-aware features, generating and making full use of LF and HF cues. FMFA improves the feature representation of RGB-T from the frequency perspective and provides stronger frequency cues for boundary auxiliary tasks. Then, we develop an HF-guided signed distance map prediction module (HF-SDM) with dual-task consistency to effectively alleviate the coarse mask caused by blur boundary. HF-SDM employs the geometric relationship of objects which boosts the interaction between salient regions and boundaries. As a result, the model can gain sharper boundaries for salient objects. Finally, we propose a frequency-aware feature aggregation network (FFANet) incorporated with dual-task learning. Extensive experiments on RGB-T SOD datasets demonstrate that our proposed method outperforms other state-of-the-art methods. Ablation studies and visualizations further verify the effectiveness and interpretability of our method.