Salient Object Detection Performance Research Articles

Salient object detection via multi-grained refinement polygon topology positive feedback

Despite significant progress in current salient object detection (SOD) tasks, they are limited by overlapping regions, broken connections, voids and neglecting explicit exploration of background regions. In this paper, we propose a multi-grained refinement polygonal topology network (MPT) that explores two important components of background mining information and the impact of topological segmentation errors. (1) Unlike existing models that focus on inter-image relationships, we introduce a multi-grained refinement module to discriminate between saliency objects and background information, and introduce inter-image relationships into the pixel-by-pixel segmentation features to enhance the discrimination of segmentation features, and this Multi-grained refinement is the basis for mining saliency objects. (2) Based on the multi-grained refinement features, we introduce a polygonal topology-aware module and add topological constraints as penalty terms to regularize the focus loss to effectively resolve topological errors to enhance the quality of the performance of salient object detection. Meanwhile, we introduce a measurements-based on positive feedback model to integrate the decision fusion of multi-model perception results to obtain higher accuracy of saliency prediction while ensuring that the prediction results will not deteriorate. Extensive experiments on six benchmark and extended datasets demonstrate that our proposed MPTNet exhibits excellent performance both qualitatively and quantitatively and outperforms state-of-the-art saliency detectors.

Salient Object Detection With Edge-Guided Learning and Specific Aggregation

Recently, the performance of salient object detection (SOD) has been significantly improved by utilizing edge information for auxiliary training. However, the extraction and utilization of edge cues and multi-level feature fusion are still two issues in existing edge-aware models. In this paper, we devise a novel SOD network with edge-guided learning and specific aggregation, named ELSA-Net, to cooperatively address these two issues. First, we propose the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">edge-guided learning strategy</i> , which utilizes edge cues as low-level guidance to improve saliency prediction. Specifically, we design a two-stream model that uses a saliency branch and an edge branch to detect the interior and the boundary of salient objects, respectively. Then, an edge-guided interaction module (EGI) is further designed to achieve feature enhancement by embedding edge information into the saliency branch as the spatial weights. In addition, two <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">specific aggregation modules</i> are proposed for the progressive fusion of multi-level features in the above two streams, thus making full use of semantic and detailed information. The high-level interactive fusion module (HIF) leverages the correlation between two deeper features to obtain more powerful global contexts. And the low-level weighted fusion module (LWF) focuses on the complement of fine information by selectively integrating input features. Extensive experiments show that the proposed approach outperforms 19 state-of-the-art methods on five datasets, which validates its effectiveness both quantitatively and qualitatively.

Weakly Supervised Salient Object Detection by Hierarchically Enhanced Scribbles

The performance of salient object detection (SOD) has been significantly advanced by using deep convolutional networks. However, it largely depends on the high cost of pixel-level annotations. To reduce human effort while improving the prediction accuracy, we propose a novel two-phase learning framework. The weakly supervised information in terms of scribbles is provided as initial labels. Then, as the first phase, high-quality pseudo-labels are generated by mapping scribbles onto object/object-part contours. These contour maps are predicted by the hierarchical contour detection algorithm, providing superior accuracy and smoothness. In the second phase, a deep neural network is alternately trained and predicted. The pseudo-labels are refined in an iterated process, where a conditional random field (CRF) model and a filter module are designed to promote the performance. Extensive experiments on five benchmarks show that our framework can achieve comparable results with the state-of-the-art fully and weakly supervised methods.

PSNet: Parallel symmetric network for RGB-T salient object detection

RGB-Thermal (RGB-T) salient object detection (SOD) aims to utilize RGB and thermal infrared data to discover the most salient object(s) in an image. Although SOD has achieved great progress, few efforts are devoted to RGB-T SOD. For this task, on the one hand, it is worth paying attention to how to aggregate critical saliency cues from both modalities to enhance salient feature representation and thus produce accurate salient object detection results. On the other hand, effective multi-level feature fusion remains a big challenge. To overcome the above mentioned issues, this paper proposes a Parallel Symmetric Network (PSNet) for RGB-T salient object detection. Specifically, we first develop a cascaded aggregation module (CAM), which fully accumulates and excavates the valuable saliency semantics from two different modalities to strengthen feature representation by cascading the designed residual-based enhancement unit. Then, we design a parallel-symmetric fusion (PSF) module to integrate crucial saliency cues from adjacent layers for saliency prediction in a parallel and symmetric manner. Besides, to make full use of multi-level features, we introduce a guidance strategy that enhances the details of the saliency map with the low-level features to boost the performance of salient object detection. Extensive experiments show that our proposed model significantly outperforms the existing fifteen state-of-the-art models on three challenging benchmark datasets. Moreover, the superior performance on RGB-D SOD also demonstrates the generalization and robustness of the proposed method. The source code will be made publicly available soon.

Progressive Dual-Attention Residual Network for Salient Object Detection

Due to the rapid development of deep learning, the performance of salient object detection has been constantly refreshed. Nevertheless, it is still challenging for existing methods to distinguish the location of salient objects and retain fine structural details. In this paper, a novel progressive dual-attention residual network (PDRNet) is proposed to exploit two complementary attention maps to guide residual learning, thus progressively refining prediction in a coarse-to-fine manner. We design a dual-attention residual module (DRM) to achieve residual refinement with the help of the dual attention (DA) scheme. Specifically, an attention map and its corresponding reverse attention map are used to make the network be aware of learning residual details from the perspective of the salient and non-salient regions, thus utilizing their complementarity to correct the mistakes of object parts and boundary details. Besides, a hierarchical feature screening module (HFSM) is designed to capture more powerful global contextual knowledge for locating salient objects. It establishes cross-scale skip connections among multi-scale features and utilizes the intra-channel dependency of these scales to enhance information interaction and feature representation. Extensive experiments have proved that our proposed PDRNet performs favorably against 18 state-of-the-art competitors on five benchmark datasets, demonstrating the effectiveness and superiority of our method.

Saliency detection network with two-stream encoder and interactive decoder

In recent years, there has been an increasing amount of works on improving the salient object detection performance based on the edge information. However, previous studies applied a parameter-sharing encoder to extract edge features and region features, leading to incomplete regions and blurry edges. In this paper, we focus on the independence and complementarity between salient edges and regions, and propose a novel salient object detection network with two-stream encoder and interactive decoder(TSEID). Specifically, instead of a parameter-sharing encoder, we construct a two-stream encoder consisting of the region extraction branch and the edge extraction branch to balance the feature domain differences between the regions and edges. Meanwhile, we develop a super-pixel segmentation-based method to extract the salient edges, so that the generated distinctive edges will ensure the integrality of edge features. The region features are obtained via a feature pyramid network and hierarchically enhanced with Channel Adaptation (CA) and Spatial Adaptation (SA). Further, the extracted region features and edge features are fed into an interactive decoder to explore their complementarity. Accordingly, a location guidance module (LGM) and an edge compensation module (ECM) are designed to refine object positions, enrich object boundaries, and generate saliency features. Extensive experiments on five challenging benchmarks well demonstrate that our approach outperforms existing deep learning-based methods, including the latest edge-guided ones.

APNet: Adversarial Learning Assistance and Perceived Importance Fusion Network for All-Day RGB-T Salient Object Detection

To improve the performance of salient object detection (SOD) in scenes with low-light conditions (e.g., nighttime) and cluttered backgrounds, infrared thermal images are used to supplement RGB images to achieve good all-day imaging as infrared images are insensitive to light source changes. Therefore, we built an adversarial learning assistance and perceived importance fusion network (APNet) for all-day RGB-thermal (RGB-T) SOD. First, an iterative adversarial learning approach was used to establish a generator and three discriminators. The generator provides salient maps that are eventually accepted by the discriminators and are used to determine their similarities with the labels. Second, a progressively guided optimization structure with high-level features is used to refine low-level features across multiple scales gradually. To further improve the detection results, a perceived importance fusion module (PIFM) is used to weigh and fuse different modalities in cases where the presence of noise may degrade sensor fusion. Extensive experiments on existing RGB-T datasets demonstrate that the proposed APNet notably outperforms state-of-the-art models.

Unsupervised Domain Adaptive Salient Object Detection through Uncertainty-Aware Pseudo-Label Learning

Recent advances in deep learning significantly boost the performance of salient object detection (SOD) at the expense of labeling larger-scale per-pixel annotations. To relieve the burden of labor-intensive labeling, deep unsupervised SOD methods have been proposed to exploit noisy labels generated by handcrafted saliency methods. However, it is still difficult to learn accurate saliency details from rough noisy labels. In this paper, we propose to learn saliency from synthetic but clean labels, which naturally has higher pixel-labeling quality without the effort of manual annotations. Specifically, we first construct a novel synthetic SOD dataset by a simple copy-paste strategy. Considering the large appearance differences between the synthetic and real-world scenarios, directly training with synthetic data will lead to performance degradation on real-world scenarios. To mitigate this problem, we propose a novel unsupervised domain adaptive SOD method to adapt between these two domains by uncertainty-aware self-training. Experimental results show that our proposed method outperforms the existing state-of-the-art deep unsupervised SOD methods on several benchmark datasets, and is even comparable to fully-supervised ones.

MEANet: Multi-modal edge-aware network for light field salient object detection

Abundant light field cues have been demonstrated helpful to boost performance of salient object detection (SOD) in challenging scenarios. However, the majority of existing deep light field SOD models focus on exploring spatial interrelations across focal slices and seldom consider boundary accuracy of salient objects, which inevitability limits the detection performance. Meanwhile, in addition to focal stacks, several other data forms/modalities can be derived simultaneously from the light field. Therefore existing UNet-like strategies widely adopted for a single modality may not be very suitable for this task. To address these issues, we propose a novel multi-modal edge-aware network for light field SOD, named MEANet. MEANet has two innovative components elaborately designed for this task, i.e., the cross-modal compensation (CMC) module and the multi-modal edge supervision (MES) module. CMC utilizes the attention mechanism to explore cross-modal complementarities and overcome the information loss of focal stack cues, whereas MES generates explicit object edges and edge features in order to progressively refine regional features to achieve edge-aware detection. Comprehensive evaluations on four benchmark datasets show that MEANet outperforms state-of-the-art light field and RGB-D/RGB SOD models, generating saliency maps with fine-grained accurate object boundaries effectively from multiple data inputs. The code and models are publicly available at https://github.com/jiangyao-scu/MEANet.

Salient object detection based on Two-stream edge attention guidance

The existing U-Net structure convolutional neural network is widely used in salient object detection, but the problem of target boundary blur is caused by the convolution pooling operation. In order to keep the edge structure of salient objects clear, a two-stream edge attention guided convolutional neural network (SEANet) method is proposed to strengthen the guided learning of edge features, mainly by guiding the edge features to compare the original image features and the depth features are enhanced to improve the accuracy of salient object detection. The loss is minimized by using consistent cross entropy loss and IOU loss to maximize the coincidence rate of the true value map and the predicted map, as well as the actual boundary and the predicted boundary. At the same time, the obtained prediction map is compared with the current mainstream nine models, and good results have been achieved. Four sets of experiments are carried out in the ablation experiment, and the experimental results also confirmed that the model has a great improvement effect on the performance of salient object detection.

A pooling-based feature pyramid network for salient object detection

How to effectively utilize and fuse deep features has become a critical point for salient object detection. Most existing methods usually adopt the convolutional features based on U-shape structures and fuse multi-scale convolutional features without fully considering the different characteristics between high-level features and low-level features. Furthermore, existing salient object detection methods rarely consider the role of pooling in convolutional neural networks. Moreover, there is still much room to improve the detection performance for objects in complex scenes. To address the problems mentioned above, we propose a pooling-based feature pyramid (PFP) network to boost salient object detection performance in this paper. First, we design two U-shaped feature pyramid modules to capture rich semantic information from high-level features and to obtain clear saliency boundaries from low-level features respectively. Second, a pyramid pooling refinement module is designed to utilize the pooling to capture more semantic information. Third, a universal channel-wise attention (UCA) module is designed to select effective high-level features of multi-scale and multi-receptive-field for rich semantic information, even in complex scenes. Finally, we fuse the selected high-level features and low-level features together, followed by an edge preservation loss to obtain accurate boundary location. Extensive experiments are conducted on five datasets and the experimental results indicate that our proposed method has the ability to get better salient object detection performance compared to the state-of-the-art methods.

Hierarchical and Interactive Refinement Network for Edge-Preserving Salient Object Detection.

Salient object detection has undergone a very rapid development with the blooming of Deep Neural Network (DNN), which is usually taken as an important preprocessing procedure in various computer vision tasks. However, the down-sampling operations, such as pooling and striding, always make the final predictions blurred at edges, which has seriously degenerated the performance of salient object detection. In this paper, we propose a simple yet effective approach, i.e., Hierarchical and Interactive Refinement Network (HIRN), to preserve the edge structures in detecting salient objects. In particular, a novel multi-stage and dual-path network structure is designed to estimate the salient edges and regions from the low-level and high-level feature maps, respectively. As a result, the predicted regions will become more accurate by enhancing the weak responses at edges, while the predicted edges will become more semantic by suppressing the false positives in background. Once the salient maps of edges and regions are obtained at the output layers, a novel edge-guided inference algorithm is introduced to further filter the resulting regions along the predicted edges. Extensive experiments on several benchmark datasets have been conducted, in which the results show that our method significantly outperforms a variety of state-of-the-art approaches.

Weakly Supervised Salient Object Detection With Spatiotemporal Cascade Neural Networks

Recently, deep learning techniques have substantially boosted the performance of salient object detection in still images. However, the salient object detection in videos by using traditional handcrafted features or deep learning features is not fully investigated, probably due to the lack of sufficient manually labeled video data for saliency modeling, especially for the data-driven deep learning. This paper proposes a novel weakly supervised approach to the salient object detection in a video, which can learn a robust saliency prediction model by using very limited manually labeled data and a large amount of weakly labeled data that could be easily generated in a supervised approach. Furthermore, we propose a spatiotemporal cascade neural network architecture for saliency modeling, in which two fully convolutional networks are cascaded to evaluate the visual saliency from both spatial and temporal cues to lead the optimal video saliency prediction. The proposed approach is extensively evaluated on the widely used challenging data sets, and the experiments demonstrate that our proposed approach substantially outperforms the state-of-the-art salient object detection models.

Iterative Feedback Control-Based Salient Object Segmentation

In this paper, we establish a mathematical model that relates the control states and the saliency values in salient object detection. We show that a linear feedback control system (LFCS) is amenable to saliency detection tasks owing to its functional properties. This inspired us to employ an LFCS to detect salient objects in static images. Based on the novel iteration method, the system gradually converges to an optimized stable state, which is associated with an accurate saliency map. In addition, to initialize the system, we propose a so-called boundary homogeneity based on a priori knowledge of the boundary in order to estimate the background likelihood and indirectly obtain a foreground (saliency) map. The experimental results indicate that such a feedback control model can offer significant improvement in salient object detection performance.

Learning graph affinities for spectral graph-based salient object detection

In this paper, we propose a novel method for learning graph affinities for salient object detection. First, we assume that a graph representation of an image is given with a predetermined connectivity rule and representative features for each of its nodes. Then, we learn to predict affinities related to this graph, that ensures a decent salient object detection performance, when used with a spectral graph based foreground detection method. To accomplish this task, we modify convolutional kernel networks (CKNs) for graph affinity calculation, which were originally proposed to predict similarities between images. Subsequently, we employ a spectral graph based salient object detection method – Extended Quantum Cuts (EQCut) – using these graph affinities. We show that the salient object detection error of such a system is differentiable with respect to the parameters of the CKN. Therefore, the proposed system can be trained end-to-end by applying error backpropagation and CKN parameters can be learned for salient object detection task. The comparative evaluations over a large set of benchmark datasets indicate that the proposed method has an insignificant computational burden on, but significantly outperforms the baseline EQCut – which uses color affinities – and achieves a comparable performance level with the state-of-the-art in some performance measures.

Cauchy graph embedding based diffusion model for salient object detection.

Salient object detection has been a rather hot research topic recently, due to its potential applications in image compression, scene classification, image registration, and so forth. The overwhelming majority of existing computational models are designed based on computer vision techniques by using lots of image cues and priors. Actually, salient object detection is derived from the biological perceptual mechanism, and biological evidence shows that the spread of the spatial attention generates the object attention. Inspired by this, we attempt to utilize the emerging spread mechanism of object attention to construct a new computational model. A novel Cauchy graph embedding based diffusion (CGED) model is proposed to fulfill the spread process. Combining the diffusion model and attention prediction model, a salient object detection approach is presented through perceptually grouping the multiscale diffused attention maps. The effectiveness of the proposed approach is validated on the salient object dataset. The experimental results show that the CGED process can obviously improve the performance of salient object detection compared with the input spatial attention map, and the proposed approach can achieve performance comparable to that of state-of-the-art approaches.

Detection of salient objects with focused attention based on spatial and temporal coherence

The understanding and analysis of video content are fundamentally important for numerous applications, including video summarization, retrieval, navigation, and editing. An important part of this process is to detect salient (which usually means important and interesting) objects in video segments. Unlike existing approaches, we propose a method that combines the saliency measurement with spatial and temporal coherence. The integration of spatial and temporal coherence is inspired by the focused attention in human vision. In the proposed method, the spatial coherence of low-level visual grouping cues (e.g. appearance and motion) helps per-frame object-background separation, while the temporal coherence of the object properties (e.g. shape and appearance) ensures consistent object localization over time, and thus the method is robust to unexpected environment changes and camera vibrations. Having developed an efficient optimization strategy based on coarse-to-fine multi-scale dynamic programming, we evaluate our method using a challenging dataset that is freely available together with this paper. We show the effectiveness and complementariness of the two types of coherence, and demonstrate that they can significantly improve the performance of salient object detection in videos.