DMNet: Dynamic Memory Network for RGB-D Salient Object Detection

Abstract

RGB-D salient object detection aims to identify the most salient object using the color and depth information of images. Currently, most of the existing salient object detection models use the classical U-Net architecture, which uses up-sampling and short link decoding to exploit saliency cues after generating multi-level features by successive convolution and pooling operations. However, the multi-level encoding and progressive up-sampling decoding used by these models may lose some of the high-level semantic information during feature extraction and information fusion and some of the detailed information during image recovery, which in turn affects the quality of generated saliency maps. To solve these problems, we propose a dynamic memory network (DMNet) consisting of an interactive enhancement encoder and a dynamic memory decoder, which achieves high-precision detection of salient objects with a more complete and superior encoding and decoding strategy. Specifically, in the interactive enhancement encoder, we propose a feature interactive fusion module (FIFM), which can enhance intermediate-scale RGB features by fusing RGB features of three adjacent scales. Moreover, we design a depth-guided dense fusion module (DDFM) to fuse the features from RGB and depth modalities. In the dynamic memory decoder, we design a full-dimensional dynamic convolutional expansion module (FDCEM) to enhance the information representation capability of features at different scales. Furthermore, we also design a gated decoding module (GDM) to decode features of different scales and eliminate non-salient information. Extensive experimental results on STERE, SIP, NLPR, NJU2K, SSD, and DES datasets demonstrate that our model outperforms most of the state-of-the-art RGB-D SOD methods. In addition, the experimental results on three RGB-T datasets, VT821, VT1000, and VT5000 also show that our model can be effectively used for RGB-T SOD.