Salient Object Detection Task Research Articles

LDWS-net: A learnable deep wavelet scattering network for RGB salient object detection

In the salient object detection task, convolutional neural network (CNN) based models have been extensively used. However, preserving a variety of boundary features of objects is equally important while detecting the salient objects. Detecting salient objects with poor boundaries have a significant detrimental effect on the salient object detection (SOD) models' robustness and accuracy. The proposed method leverages a unique and novel edge-directed salient object detection network, which combines wavelet scattering network features with CNN-based features. This integration enables the network to capture both textural and high-level semantic information, leading to improved SOD performance. Additionally, a learnable wavelet scattering network allows for the efficient collection and preservation of textural aspects of objects. This network is seamlessly embedded into the encoder section of the proposed architecture, enhancing the discriminative power of the model. Furthermore, a weighted feature integration module (WFIM) is proposed in the decoder section to adaptively integrate linked nearby features by evaluating their relevance, resulting in improved representation and discrimination capabilities. Extensive testing on well-known benchmark SOD datasets demonstrates that the proposed LDWS-Net outperforms state-of-the-art techniques, exhibiting accurate identification of salient objects and efficient detection of their edges.

Absolute and Relative Depth-Induced Network for RGB-D Salient Object Detection

Detecting salient objects in complicated scenarios is a challenging problem. Except for semantic features from the RGB image, spatial information from the depth image also provides sufficient cues about the object. Therefore, it is crucial to rationally integrate RGB and depth features for the RGB-D salient object detection task. Most existing RGB-D saliency detectors modulate RGB semantic features with absolution depth values. However, they ignore the appearance contrast and structure knowledge indicated by relative depth values between pixels. In this work, we propose a depth-induced network (DIN) for RGB-D salient object detection, to take full advantage of both absolute and relative depth information, and further, enforce the in-depth fusion of the RGB-D cross-modalities. Specifically, an absolute depth-induced module (ADIM) is proposed, to hierarchically integrate absolute depth values and RGB features, to allow the interaction between the appearance and structural information in the encoding stage. A relative depth-induced module (RDIM) is designed, to capture detailed saliency cues, by exploring contrastive and structural information from relative depth values in the decoding stage. By combining the ADIM and RDIM, we can accurately locate salient objects with clear boundaries, even from complex scenes. The proposed DIN is a lightweight network, and the model size is much smaller than that of state-of-the-art algorithms. Extensive experiments on six challenging benchmarks, show that our method outperforms most existing RGB-D salient object detection models.

Self-progress aggregate learning for weakly supervised salient object detection

The task of salient object detection aims to find the most salient object from the samples. In the field of weakly supervised learning, the existing weakly supervised salient object detection (WSSOD) methods often fail to utilize the limited label information (such as self-affinity or edge features, and scale transform) for learning. Therefore, this paper proposes a self-progress aggregate learning method named SPAL. First, the feature optimization scheme of the edge information module is put forward based on analysis of the problems existing in the current convolutional neural network for detection of the edge information of the object. Obviously, the salient object has a low requirement for high-level information, In particular, in order to improve the utilization rate of the network structure without increasing its complexity, the affinity global context is design in view of the particularity of the structure of a salient object. The structure of a salient object not only depends on the deep-level semantic feature information to a certain extent, but also has a certain guiding effect on the object position and edge information. Second, high-level affinity information is used to complement the slight-level edge information globally, and the scale attention module is adopted to guide the network to adapt the multi-scale reinforcement feature learning ability for the salient object regions. Our method SPAL achieved better experimental results than the other competitive models for comparison on five benchmark data sets (i.e. for DUTS-TEST, compared with CCFNet our method achieved an improvement of 0.6% for mean absolute error (MAE), 5.1% for Fb, and 1.1% for Eℰ ), which demonstrates the effectiveness of our proposed method.

PoolNet+: Exploring the Potential of Pooling for Salient Object Detection.

We explore the potential of pooling techniques on the task of salient object detection by expanding its role in convolutional neural networks. In general, two pooling-based modules are proposed. A global guidance module (GGM) is first built based on the bottom-up pathway of the U-shape architecture, which aims to guide the location information of the potential salient objects into layers at different feature levels. A feature aggregation module (FAM) is further designed to seamlessly fuse the coarse-level semantic information with the fine-level features in the top-down pathway. We can progressively refine the high-level semantic features with these two modules and obtain detail enriched saliency maps. Experimental results show that our proposed approach can locate the salient objects more accurately with sharpened details and substantially improve the performance compared with the existing state-of-the-art methods. Besides, our approach is fast and can run at a speed of 53 FPS when processing a 300 ×400 image. To make our approach better applied to mobile applications, we take MobileNetV2 as our backbone and re-tailor the structure of our pooling-based modules. Our mobile version model achieves a running speed of 66 FPS yet still performs better than most existing state-of-the-art methods. To verify the generalization ability of the proposed method, we apply it to the edge detection, RGB-D salient object detection, and camouflaged object detection tasks, and our method achieves better results than the corresponding state-of-the-art methods of these three tasks. Code can be found at http://mmcheng.net/poolnet/.

Weakly supervised salient object detection via image category annotation.

The rapid development of deep learning has made a great progress in salient object detection task. Fully supervised methods need a large number of pixel-level annotations. To avoid laborious and consuming annotation, weakly supervised methods consider low-cost annotations such as category, bounding-box, scribble, etc. Due to simple annotation and existing large-scale classification datasets, the category annotation based methods have received more attention while still suffering from inaccurate detection. In this work, we proposed one weakly supervised method with category annotation. First, we proposed one coarse object location network (COLN) to roughly locate the object of an image with category annotation. Second, we refined the coarse object location to generate pixel-level pseudo-labels and proposed one quality check strategy to select high quality pseudo labels. To this end, we studied COLN twice followed by refinement to obtain a pseudo-labels pair and calculated the consistency of pseudo-label pairs to select high quality labels. Third, we proposed one multi-decoder neural network (MDN) for saliency detection supervised by pseudo-label pairs. The loss of each decoder and between decoders are both considered. Last but not least, we proposed one pseudo-labels update strategy to iteratively optimize pseudo-labels and saliency detection models. Performance evaluation on four public datasets shows that our method outperforms other image category annotation based work.

Distilling Knowledge From Super-Resolution for Efficient Remote Sensing Salient Object Detection

Current state-of-the-art remote sensing salient object detectors always require high-resolution spatial context to ensure excellent performance, which incurs enormous computation costs and hinders real-time efficiency. In this work, we propose a universal super-resolution assisted learning (SRAL) framework to boost performance and accelerate the inference efficiency of existing approaches. To this end, we propose to reduce the spatial resolution of the input remote sensing images (RSIs), which is model-agnostic, and can be applied to existing algorithms without extra computation cost. Specifically, a transposed saliency detection decoder (TSDD) is designed to upsample interim features progressively. On top of it, an auxiliary super-resolution decoder (ASRD) is proposed to build a multitask learning (MTL) framework to investigate an efficient complementary paradigm of saliency detection and super-resolution. Furthermore, a novel task-fusion guidance module (TFGM) is proposed to effectively distill domain knowledge from the super-resolution auxiliary task to the salient object detection task in optical RSIs. The presented ASRD and TFGM can be omitted in the inference phase without any extra computational budget. Extensive experiments on three datasets show that the presented SRAL with 224×224 input is superior to more than 20 algorithms. Moreover, it can be successfully generalized to existing typical networks with significant accuracy improvements in a parameter-free manner. Codes and models are available at https://github.com/lyf0801/SRAL.

White blood cell detection using saliency detection and CenterNet: A two-stage approach.

White blood cell (WBC) detection plays a vital role in peripheral blood smear analysis. However, cell detection remains a challenging task due to multi-cell adhesion, different staining and imaging conditions. Owing to the powerful feature extraction capability of deep learning, object detection methods based on convolutional neural networks (CNNs) have been widely applied in medical image analysis. Nevertheless, the CNN training is time-consuming and inaccuracy, especially for large-scale blood smear images, where most of the images are background. To address the problem, we propose a two-stage approach that treats WBC detection as a small salient object detection task. In the first saliency detection stage, we use the Itti's visual attention model to locate the regions of interest (ROIs), based on the proposed adaptive center-surround difference (ACSD) operator. In the second WBC detection stage, the modified CenterNet model is performed on ROI sub-images to obtain a more accurate localization and classification result of each WBC. Experimental results showed that our method exceeds the performance of several existing methods on two different data sets, and achieves a state-of-the-art mAP of over 98.8%.

GSCINet: Gradual Shrinkage and Cyclic Interaction Network for Salient Object Detection

Feature Pyramid Network (FPN) has been widely applied in the task of salient object detection (SOD), which has achieved great performance. However, most existing FPN-based SOD methods still have some limitations, such as insufficient guidance due to gradual dilution of semantic information, excessive computation leading to slow inference speed, and low efficiency of training models. In this paper, we design a novel Gradual Shrinkage and Cyclic Interaction Network (GSCINet) for efficient and accurate SOD, consisting of a Multi-Scale Contextual Attention Module (MSCAM) and an Adjacent Feature Shrinkage and Interaction Module (AFSIM). Specifically, the MSCAM aims at efficiently capturing multi-scale and multi-receptive-field contextual attention information through a series of well-designed convolutions and attention weight matrices of different scales to enhance the performance of initial input features. Subsequently, in AFSIM, we propose a gradual shrinkage structure and introduce a circular interaction mechanism to optimize the compressed features with less calculation cost, thereby enabling fast and accurate inference of salient objects. Extensive experimental results demonstrate the high efficiency and superiority of GSCINet against 17 state-of-the-art (SOTA) saliency detection methods under multiple evaluation metrics.

R2Net: Residual refinement network for salient object detection

The multi-scale features and the fusion strategy of contextual features are the keys to salient object detection task. Previous multi-scale-based works often overlooked the completeness of features when acquiring multi-scale features. Moreover, the decoders were hard to accurately capture the salient object and refine the object's boundaries simultaneously when in a complex environment, which leads to unsatisfactory saliency maps. To address the above problems, we present a Residual Refinement Network (R2Net), which is composed of the Residual Pyramid Module (RPM), the Residual Fusion Module (RFM) and the Feature Optimize Module (FOM), for salient object detection. RPM integrates different feature information through different receptive fields, can not only obtain multi-scale information but also retain the local details information of features. RFM can better locate salient objects and refine the boundaries through the interweaving and fusion of multi-layer features. And FOM is designed to further refine the fused features. Furthermore, we propose a Structural Polishing (SP) loss, which better guides the network through pixel-level supervision, global supervision and boundary supervision to generate high-quality saliency maps with fine boundaries. Experimental results on 6 benchmark datasets demonstrate that the proposed method has superior performance compared with 18 state-of-the-art methods. The code and results of our method are available at https://github.com/zhangjin12138/R2Net

Aggregate interactive learning for RGB-D salient object detection

The task of salient object detection is to find the most noticeable areas in the image. On the one hand, most of the existing RGB-D saliency object detection requires additional networks to process depth features, and the sub-networks that process depth features rely too much on the RGB network, resulting in higher computational costs. On the other hand, when dealing with multi-scale features, most models tend to produce information loss, the semantic information representation ability is weak, and good detection results cannot be achieved, which limits its practical application. Firstly, this paper proposes a strategy of aggregation and interaction to extract edge features, depth features and salient features, while maintaining local details, fully extracting global information. Secondly, in the learning process of high-level features, depth features and salient features are extracted at the same time, which reduces the complexity of the network and does not require the additional sub-networks. On the other hand, the deformable convolution network is used to solve the multi-scale problem to ensure that more detailed feature information is extracted. Finally, taking advantage of the complementarity between features, using the one-to-one feature fusion module, the problem of information redundancy in the feature fusion process is solved, and the fused features can accurately locate the salient results with clear details. The experimental results on six datasets show that compared with other state-of-the-art algorithms, the algorithm in this paper has excellent performance.

Global and local information aggregation network for edge-aware salient object detection

Aggregation of local and global contextual information by exploiting multi-level features in a fully convolutional network is a challenge for the pixel-wise salient object detection task. Most existing methods still suffer from inaccurate salient regions and blurry boundaries. In this paper, we propose a novel edge-aware global and local information aggregation network (GLNet) to fully exploit the integration of side-output local features and global contextual information and utilization of contour information of salient objects. The global guidance module (GGM) is proposed to learn discriminative multi-level information with the direct guidance of global semantic knowledge for more accurate saliency prediction. Specifically, the GGM consists of two key components, where the global feature discrimination module exploits the inter-channel relationship of global semantic features to boost representation power, and the local feature discrimination module enables different side-output local features to selectively learn informative locations by fusing with global attentive features. Besides, we propose an edge-aware aggregation module (EAM) to employ the correlation between salient edge information and salient object information for generating estimated saliency maps with explicit boundaries. We evaluate our proposed GLNet on six widely-used saliency detection benchmark datasets by comparing with 17 state-of-the-art methods. Experimental results show the effectiveness and superiority of our proposed method on all the six benchmark datasets.

Multi-scale iterative refinement network for RGB-D salient object detection

The extensive research leveraging RGB-D information has been exploited in salient object detection. However, salient visual cues appear in various scales and resolutions of RGB images due to semantic gaps at different feature levels. Meanwhile, similar salient patterns are available in cross-modal depth images as well as multi-scale versions. Cross-modal fusion and multi-scale refinement are still an open problem in RGB-D salient object detection task. In this paper, we begin by introducing top-down and bottom-up iterative refinement architecture to leverage multi-scale features, and then devise attention based fusion module (ABF) to address on cross-modal correlation. We conduct extensive experiments on seven public datasets. The experimental results show the effectiveness of our devised method.

ERBANet: Enhancing region and boundary awareness for salient object detection

Existing methods based on convolutional neural networks (CNNs) have progressed significantly in the salient object detection task. However, in some cases, the accurate detection of intact objects and maintenance of their original detailed structures, such as boundaries, was not possible. In this paper, we propose an Enhancing Region and Boundary Awareness Network (ERBANet), equipped with attentional feature enhancement (AFE) modules to improve the detection performance. The AFE modules act on high-level and low-level features to generate corresponding attentional features. High-level attentional features are used to highlight entire salient objects, while low-level attentional features help retain their boundaries. The proposed ERBANet aims to effectively aggregate high-level and low-level attentional features, fully utilizing their respective advantages. Furthermore, we propose a novel boundary maintenance loss (BML) for learning to preserve the original boundaries of salient objects. Meanwhile, dice loss is combined for learning to enhance the integrity of salient regions. The experimental results on five benchmark datasets demonstrate that our proposed method outperforms recent state-of-the-art methods and achieves better performance.

Decomposition and Completion Network for Salient Object Detection.

Recently, fully convolutional networks (FCNs) have made great progress in the task of salient object detection and existing state-of-the-arts methods mainly focus on how to integrate edge information in deep aggregation models. In this paper, we propose a novel Decomposition and Completion Network (DCN), which integrates edge and skeleton as complementary information and models the integrity of salient objects in two stages. In the decomposition network, we propose a cross multi-branch decoder, which iteratively takes advantage of cross-task aggregation and cross-layer aggregation to integrate multi-level multi-task features and predict saliency, edge, and skeleton maps simultaneously. In the completion network, edge and skeleton maps are further utilized to fill flaws and suppress noises in saliency maps via hierarchical structure-aware feature learning and multi-scale feature completion. Through jointly learning with edge and skeleton information for localizing boundaries and interiors of salient objects respectively, the proposed network generates precise saliency maps with uniformly and completely segmented salient objects. Experiments conducted on five benchmark datasets demonstrate that the proposed model outperforms existing networks. Furthermore, we extend the proposed model to the task of RGB-D salient object detection, and it also achieves state-of-the-art performance. The code is available at https://github.com/wuzhe71/DCN.

STA-Net: spatial-temporal attention network for video salient object detection

This paper conducts a systematic study on the role of spatial and temporal attention mechanism in the video salient object detection (VSOD) task. We present a two-stage spatial-temporal attention network, named STA-Net, which makes two major contributions. In the first stage, we devise a Multi-Scale-Spatial-Attention (MSSA) module to reduce calculation cost on non-salient regions while exploiting multi-scale saliency information. Such a sliced attention method offers an individual way to efficiently exploit the high-level features of the network with an enlarged receptive field. The second stage is to propose a Pyramid-Saliency-Shift-Aware (PSSA) module, which puts emphasis on the importance of dynamic object information since it offers a valid shift cue to confirm salient object and capture temporal information. Such a temporal detection module is able to encourage precise salient region detection. Exhaustive experiments show that the proposed STA-Net is effective for video salient object detection task, and achieves compelling performance in comparison with state-of-the-art.

Salient Object Detection by Fusing Local and Global Contexts

Benefiting from the powerful discriminative feature learning capability of convolutional neural networks (CNNs), deep learning techniques have achieved remarkable performance improvement for the task of salient object detection (SOD) in recent years. However, most existing deep SOD models do not fully exploit informative contextual features, which often leads to suboptimal detection performance in the presence of a cluttered background. This paper presents a context-aware attention module that detects salient objects by simultaneously constructing connections between each image pixel and its local and global contextual pixels. Specifically, each pixel and its neighbors bidirectionally exchange semantic information by computing their correlation coefficients, and this process aggregates contextual attention features both locally and globally. In addition, an attention-guided hierarchical network architecture is designed to capture fine-grained spatial details by transmitting contextual information from deeper to shallower network layers in a top-down manner. Extensive experiments on six public SOD datasets show that our proposed model demonstrates superior SOD performance against most of the current state-of-the-art models under different evaluation metrics.

Salient Object Detection in the Distributed Cloud-Edge Intelligent Network

Intelligent network is crucial in the building of telecom networks because it utilizes artificial intelligent technologies to improve the performance. Salient object detection has increasingly attracted interest from intelligent network research since estimating human attention to objects is a crucial step in various surveillance applications. However, the computational-consuming and memory-consuming detection model is still less effective when it is deployed only either on the cloud or on the edge. In this article, we propose a specially-designed cloud-edge distributed framework for salient object detection based on the intelligent network. This framework can overcome the difficulty to transmit massive data in the cloud-only deployment scheme, as well as the difficulty to analyze massive data in the edge-only deployment scheme. However, the traditional cloud-edge distributed schemes are unsuitable to salient object detection task because of two challenges: 1) balance between the within-semantic knowledge and cross-semantic knowledge for the model training in different servers; 2) contradiction between extracting the semantic knowledge with global contextual information and local detailed information. To tackle the first challenge, our framework enables a hierarchical information allocation strategy in the cloud. It can prompt the salient object detection model in the edge to learn more from the similar scenes or semantics with where the edge server is located, while preserving the generalization ability of the model in the different scenes. To address the second challenge, our framework proposes a novel pyramidal deep learning model. It can effectively capture the global contextual features of the salient object, while preserving its local detailed features. The extensive experiments performed on six commonly- used public datasets can demonstrate the effectiveness of our framework and its superiority to 11 state-of-the-art approaches.

Multi-Level Context Aggregation Network With Channel-Wise Attention for Salient Object Detection

Fully convolutional neural networks (FCNs) have shown their advantages in the salient object detection task. However, the prediction results do not perform well in most existing FCN-based methods, such as coarse object boundaries or even getting wrong predictions, which resulted from ignoring the difference between multi-level features during feature aggregation or underutilizing the spatial details suitable for locating boundaries. In this paper, we propose a novel end-to-end multi-level context aggregation network (MLCANet) to solve the problem mentioned-above, in which both bottom-up and top-down message passing can cooperate in a joint manner.The bottom-up process that aggregates low-level fine details features into high-level semantically-richer features would enhance high-level features, and in turn the top-down process that passes refined features from deeper layers to the shallower ones could benefit from the enhanced high-level features. Also by considering that the features from different layers may not be equally important, a multi-level feature aggregation mechanism with channel-wise attention is proposed to aggregate multi-level features by flexibly adjusting their contributions and absorbing useful information to refine themselves. The features after message passing which simultaneously encode semantic information and spatial details are used to predict saliency maps in our network. Extensive experiments demonstrate that our method can obtain high quality saliency maps with clear boundaries, and perform favorably against the state-of-the-art methods without any pre-processing and post-processing.

Exploiting color name space for salient object detection

In this paper, we will investigate the contribution of color names for the task of salient object detection. An input image is first converted to color name space, which is consisted of 11 probabilistic channels. By exploiting a surroundedness cue, we obtain a saliency map through a linear combination of a set of sequential attention maps. To overcome the limitation of only using the surroundedness cue, two global cues with respect to color names are invoked to guide the computation of a weighted saliency map. Finally, we integrate the above two saliency maps into a unified framework to generate the final result. In addition, an improved post-processing procedure is introduced to effectively suppress image backgrounds while uniformly highlight salient objects. Experimental results show that the proposed model produces more accurate saliency maps and performs well against twenty-one saliency models in terms of three evaluation metrics on three public data sets.

Distribution equalization learning mechanism for road crack detection

Visual-based road crack detection becomes a hot research topic over the last decade because of its huge application demands. Road crack detection is actually a special form of salient object detection task, whose objects are small and distribute randomly in the image compared to the traditional ones, which increase the difficulty of detecting. Most conventional methods utilize bottom information, such as color, texture, and contrast, to extract the crack regions in the image. Even though these methods can achieve satisfactory performances for images with simple scenarios, they are easily interfered by some factors such as light and shadow, which may decrease the detection result directly. Inspired by the competitive performances of deep convolutional neural networks on many visual tasks, we propose a distribution equalization learning mechanism for road crack detection in this paper. Firstly, we consider the crack detection task as a pixel-level classification and use a U-Net based architecture to finalize it. Secondly, the occurrence probability of crack and non-crack are so different, which results in the ill-conditioned classifier and undesirable detection performance, especially the high false detection rate. In this case, we propose a weighted cross entropy loss term and a data augmentation strategy to avoid influence from imbalanced samples through emphasizing the crack regions. Additionally, we propose an auxiliary interaction loss, and combine it with the popular self-attention strategy to alleviate the fracture situations through considering relationships among different local regions in the image. Finally, we tested the proposed method on three public and challenging datasets, and the experimental results demonstrate its effectiveness.