Context-aware network for RGB-D salient object detection

Abstract

Convolutional neural networks (CNNs) have shown unprecedented success in object representation and detection. Nevertheless, CNNs lack the capability to model context dependencies among objects, which are crucial for salient object detection. As the long short-term memory (LSTM) is advantageous in propagating information, in this paper, we propose two variant LSTM units for the exploration of contextual dependencies. By incorporating these units, we present a context-aware network (CAN) to detect salient objects in RGB-D images. The proposed model consists of three components: feature extraction, context fusion of multiple modalities and context-dependent deconvolution. The first component is responsible for extracting hierarchical features in color and depth images using CNNs, respectively. The second component fuses high-level features by a variant LSTM to model multi-modal spatial dependencies in contexts. The third component, embedded with another variant LSTM, models local hierarchical context dependencies of the fused features at multi-scales. Experimental results on two public benchmark datasets show that the proposed CAN can achieve state-of-the-art performance for RGB-D stereoscopic salient object detection.