Bridging feature complementarity gap between encoder and decoder for salient object detection

Abstract

Deep convolutional neural networks have pushed the resulting performances of salient object detection to the new state-of-the-art. However, most of the existing methods mainly focus on the calibration of the encoder and decoder but ignore the complementarity between them, which may lose some vital saliency areas or produce noises. Moreover, the research on the saliency structure-level loss function remains scarce. To address the above issues, we propose a network that BRIdgEs the Feature complementarity gap (BriefNet), which consists of a common encoder-decoder structure, a collaborative complementarity module (CCM), an encoder clustering module (ECM), and a feature aggregation module (FAM) optimized by a novel hybrid loss. Specifically, CCM and ECM are embedded after the encoder-decoder structure to recalibrate the features of the encoder and decoder, and FCM aggregates them. Firstly, CCM includes a feature supplement operation (FSO) and a feature generality extraction (FGE). FSO makes up the lost features of the decoder. FGE aims at exploring the generality features between encoder and decoder, aiming to constrain the saliency. Secondly, by clustering the local and global features of encoders, ECM enhances the connectivity and aggregates the relevancy semantics information. Thirdly, CCM and ECM are aggregated by FAM to predict the final results. Finally, we propose a structure recalibration loss (SRL) to further recalibrate the fuzzy structure features. The hybrid loss that consists of the pixel-level binary cross entropy loss (BCE), the structure-level loss SRL, and the map-level Dice loss is used to optimize the network. Experimental results on five prevailing datasets show that the proposed BriefNet achieves consistently superior performances under various evaluation metrics. In addition, we put forward a simplified BriefNet, which also achieves competitive results by only increasing a few parameters (10 M) compared with the baseline.