Framework For Salient Object Detection Research Articles

ADRNet-S*: Asymmetric depth registration network via contrastive knowledge distillation for RGB-D mirror segmentation

Mirrors are frequently encountered in computer vision segmentation scenes and can considerably impact salient object detection (SOD) accuracy owing to their complex and variable appearances. In recent studies, several methods have achieved good performance. However, most of their structures are complex and heavy, while lightweight structures fall short in terms of accuracy. In this study, a novel asymmetric depth registration network student model trained with distilled knowledge (ADRNet-S*) is proposed to overcome these limitations. First, to filter redundant red–green–blue information from the image features, a depth maximal guidance module is introduced that orchestrates multimodal feature fusion. Next, a novel pixel-relation graph convolution module establishes global contextual relationships to further expand image understanding. Inspired by contrastive learning and knowledge distillation (KD), a contrastive KD framework is devised that enables the ADRNet-student (ADRNet-S) to significantly enhance its segmentation capability under the guidance of an experienced ADRNet teacher (ADRNet-T) to produce the ADRNet-S*. The proposed design compresses the number of parameters and floating-point operations from 25.93 M and 85.83 G for the ADRNet-T to 2.46 M and 7.32 G for the ADRNet-S, while maintaining 96.7 % accuracy. Finally, extensive experimentation demonstrates the robustness and effectiveness of the proposed method and its novel SOD framework.78.01

KD-SCFNet: Towards more accurate and lightweight salient object detection via knowledge distillation

Most existing salient object detection (SOD) models are difficult to apply due to the complex and huge model structures. Although some lightweight models are proposed, the accuracy is barely satisfactory. In this paper, we design a novel semantics-guided contextual fusion network (SCFNet) that focuses on the interactive fusion of multi-level features for accurate and efficient salient object detection. Furthermore, we apply knowledge distillation to SOD task and provide a sizeable dataset KD-SOD80K. In detail, we transfer the rich knowledge from a seasoned teacher to the untrained SCFNet through unlabeled images, enabling SCFNet to learn a strong generalization ability to detect salient objects more accurately. The knowledge distillation based SCFNet (KD-SCFNet) achieves comparable accuracy to the state-of-the-art heavyweight methods with less than 1M parameters and 174 FPS real-time detection speed. Extensive experiments demonstrate the robustness and effectiveness of the proposed distillation method and SOD framework.

Studies Advanced in Salient Object Detection based on Deep Learning

A very important job in the area of computer vision is salient object detection, its goal is to check salient or obvious objects in images or videos. Thanks to rapidly developing networks of convolutive neurons, especially generative adversarial networks, CGAN has become a popular framework for salient object detection, which has promoted breakthroughs in detection precision and velocity of protruding objects. Focusing on the topic of salient object detection, in this paper, we systematically introduce the research progress of salient object detection. Specifically, we first introduce methods for detecting protracted objects based on general object detection algorithms (such as Fast R-CNN, YOLO, and SSD), including their basic ideas, key steps, advantages and disadvantages, etc. We then focus on CGAN-based salient object detection methods, including method classification, typical method introduction, and application examples. Finally, we quantitatively compare the performance of different saliency detection algorithms on common datasets, and discuss the existing research problems and feasible solutions of the topic.

Salient detection via the fusion of background-based and multiscale frequency-domain features

Salient object detection is a fundamental problem in image processing and computer vision. Many saliency detection algorithms based on the background and frequency-domain are used to extract salient object clues. However, the former causes the real object to be submerged in the detected object areas, especially in complex or small object scenes. While the latter will lead to the loss of some object information when detecting large objects. To solve these problems and achieve better object detection results, we propose a fusion framework for salient object detection by fusing background and frequency-domain features. The background features of the image are extracted by an improved background model. This model represents the spatial layout of the image area with respect to the image boundaries. Meanwhile, we present a new frequency-domain processing method to obtain multiscale frequency-domain features and mark the saliency of the object at different scales. Within our framework, inspired by human visual attention, we use the idea of a self-attention mechanism to capture the intrinsic relation between background and multiscale frequency-domain features. In addition, this fusion framework provides a three-dimensional Gaussian convolution kernel, which expands two-dimensional local information to three dimensions for feature fusion, thus producing more accurate salient objects. Experiment results demonstrate that the proposed method consistently outperforms eleven state-of-the-art methods on five challenging and complicated datasets in terms of four evaluation metrics.

TF-SOD: a novel transformer framework for salient object detection

Most of existing salient object detection models are based on fully convolutional network (FCN), which learn multi-scale/level semantic information through convolutional layers to obtain high-quality predicted saliency maps. However, convolution is locally interactive, it is difficult to capture remote dependencies, and FCN-based methods suffer from coarse object boundaries. In this paper, to solve these problems, we propose a novel transformer framework for salient object detection (named TF-SOD), which mainly consists of the encoder part of the FCN, fusion module (FM), transformer module (TM) and feature decoder module (FDM). Specifically, FM is a bridge connecting the encoder and TM and provides some foresight for the non-local interaction of TM. Besides, FDM can efficiently decode the non-local features output by TM and achieve deep fusion with local features. This architecture enables the network to achieve a close integration of local and non-local interactions, making information complementary to each other, deeply mining the associated information between features. Furthermore, we also propose a novel edge reinforcement learning strategy, which can effectively suppress edge blurring from local and global aspects by means of powerful network architecture. Extensive experiments using five datasets demonstrate that the proposed method outperforms 19 state-of-the-art methods

Weakly-Supervised Salient Object Detection on Light Fields.

Most existing salient object detection (SOD) methods are designed for RGB images and do not take advantage of the abundant information provided by light fields. Hence, they may fail to detect salient objects of complex structures and delineate their boundaries. Although some methods have explored multi-view information of light field images for saliency detection, they require tedious pixel-level manual annotations of ground truths. In this paper, we propose a novel weakly-supervised learning framework for salient object detection on light field images based on bounding box annotations. Our method has two major novelties. First, given an input light field image and a bounding-box annotation indicating the salient object, we propose a ground truth label hallucination method to generate a pixel-level pseudo saliency map, to avoid heavy cost of pixel-level annotations. This method generates high quality pseudo ground truth saliency maps to help supervise the training, by exploiting information obtained from the light field (including depths and RGB images). Second, to exploit the multi-view nature of the light field data in learning, we propose a fusion attention module to calibrate the spatial and channel-wise light field representations. It learns to focus on informative features and suppress redundant information from the multi-view inputs. Based on these two novelties, we are able to train a new salient object detector with two branches in a weakly-supervised manner. While the RGB branch focuses on modeling the color contrast in the all-in-focus image for locating the salient objects, the Focal branch exploits the depth and the background spatial redundancy of focal slices for eliminating background distractions. Extensive experiments show that our method outperforms existing weakly-supervised methods and most fully supervised methods.

A salient object detection framework using linear quadratic regulator controller

In this paper, a novel salient object detection framework based on Linear Quadratic Regulator (LQR) controller is proposed. The major goal of this research is to take advantage of optimal control theory for improving the performance of detecting salient objects in images. In this regard, for the sake of detection of salient and non-salient regions, two LQR-based control systems are employed. In the proposed framework, for the initialization of the control systems, background and foreground estimations have been done with two different strategies. Doing so, we would ultimately have more effective distinction between those regions. After the initialization step, the control systems refine both estimations in parallel until reaching a steady state for each of them. Within the mentioned process, by using optimal control concept, specifically LQR controller (for the first time in the field), control signals which are in charge of determining saliency values, would be constantly optimized. At the end, the raw saliency map will be generated by combination of background and foreground optimized initial maps. Finally, the integrated saliency map will be refined by using angular embedding method. The experimental evaluations on three benchmark datasets shows that the proposed framework performs well and introduces comparable results with some deep learning based methods.

Progressive Self-Guided Loss for Salient Object Detection.

We present a simple yet effective progressive self-guided loss function to facilitate deep learning-based salient object detection (SOD) in images. The saliency maps produced by the most relevant works still suffer from incomplete predictions due to the internal complexity of salient objects. Our proposed progressive self-guided loss simulates a morphological closing operation on the model predictions for progressively creating auxiliary training supervisions to step-wisely guide the training process. We demonstrate that this new loss function can guide the SOD model to highlight more complete salient objects step-by-step and meanwhile help to uncover the spatial dependencies of the salient object pixels in a region growing manner. Moreover, a new feature aggregation module is proposed to capture multi-scale features and aggregate them adaptively by a branch-wise attention mechanism. Benefiting from this module, our SOD framework takes advantage of adaptively aggregated multi-scale features to locate and detect salient objects effectively. Experimental results on several benchmark datasets show that our loss function not only advances the performance of existing SOD models without architecture modification but also helps our proposed framework to achieve state-of-the-art performance.

Salient object detection via effective background prior and novel graph

Salient object detection is getting more and more attention in computer vision field. In this paper, we propose a novel and effective framework for salient object detection. Firstly, we develop a robust background-based map by using spatial prior to remove the foreground noises of image boundary regions. The proposed background-based map and Objectness map are integrated to obtain a coarse saliency map. Then, an effective saliency propagation mechanism is utilized to further highlight salient object and suppress background region by defining a novel graph model, each node connects to its more similar neighbors and nodes with low saliency values in the proposed graph. As a result, the coarse saliency map is optimized to the refined saliency map by novel graph based saliency propagation. Finally, we construct a novel integration framework to further integrate two saliency maps for performance improvement. Experiments on three benchmark datasets are tested, experimental results show the superiority of the proposed algorithm than other state-of-the-art methods.

Salient object detection based on distribution-edge guidance and iterative Bayesian optimization

Salient object detection has witnessed rapid progress, despite most existing methods still struggling in complex scenes, unfortunately. In this paper, we propose an efficient framework for salient object detection based on distribution-edge guidance and iterative Bayesian optimization. By considering color, spatial, and edge information, a discriminative metric is first constructed to measure the similarity between different regions. Next, boundary prior embedded with background scatter distribution is utilized to yield the boundary contrast map, and then a contour completeness map is derived through a wholly closed shape of the object. Finally, the above both maps are jointly integrated into an iterative Bayesian optimization framework to obtain the final saliency map. Results from an extensive number of experimentations demonstrate that the promising performance of the proposed algorithm against the state-of-the-art saliency detection methods in terms of different evaluation metrics on several benchmark datasets.

A mix-supervised unified framework for salient object detection

Recently, although deep learning network has shown its advantages in supervised salient object detection, supervised models often require massive pixel-wise annotations and learnable parameters, which seriously manacle training and testing of models. In this paper, we present a mix-supervised unified framework for salient object detection to avoid the insufficient training labels and speed training and testing up, which is composed of a region-wise stream and a pixel-wise stream. In the region-wise stream, to avoid the requirement of expensive pixel-wise annotations, an improved energy equation based manifold learning algorithm is employed, by which accurate object location and prior knowledge are introduced by the unsupervised learning. In the pixel-wise stream, to alleviate the problem of time-consuming, a simplified bi-directional reuse network is introduced, which can obtain clear object contour and competitive performance with fewer parameters. To relieve the bottleneck pressure of parallel training and testing, each steam is directly connected to its pre-processed color feature and post-processing refinement. Extensive experiments demonstrate that each component contributes to the final results and complement each other perfectly.

Salient Object Detection in the Distributed Cloud-Edge Intelligent Network

Intelligent network is crucial in the building of telecom networks because it utilizes artificial intelligent technologies to improve the performance. Salient object detection has increasingly attracted interest from intelligent network research since estimating human attention to objects is a crucial step in various surveillance applications. However, the computational-consuming and memory-consuming detection model is still less effective when it is deployed only either on the cloud or on the edge. In this article, we propose a specially-designed cloud-edge distributed framework for salient object detection based on the intelligent network. This framework can overcome the difficulty to transmit massive data in the cloud-only deployment scheme, as well as the difficulty to analyze massive data in the edge-only deployment scheme. However, the traditional cloud-edge distributed schemes are unsuitable to salient object detection task because of two challenges: 1) balance between the within-semantic knowledge and cross-semantic knowledge for the model training in different servers; 2) contradiction between extracting the semantic knowledge with global contextual information and local detailed information. To tackle the first challenge, our framework enables a hierarchical information allocation strategy in the cloud. It can prompt the salient object detection model in the edge to learn more from the similar scenes or semantics with where the edge server is located, while preserving the generalization ability of the model in the different scenes. To address the second challenge, our framework proposes a novel pyramidal deep learning model. It can effectively capture the global contextual features of the salient object, while preserving its local detailed features. The extensive experiments performed on six commonly- used public datasets can demonstrate the effectiveness of our framework and its superiority to 11 state-of-the-art approaches.

Embedding topological features into convolutional neural network salient object detection

Salient object detection can be applied as a critical preprocessing step in many computer vision tasks. Recent studies of salient object detection mainly employed convolutional neural networks (CNNs) for mining high-level semantic properties. However, the existing methods can still be improved to find precise semantic information in different scenarios. In particular, in the two main methods employed for salient object detection, the patchwise detection models might ignore spatial structures among regions and the fully convolution-based models mainly consider semantic features in a global manner. In this paper, we proposed a salient object detection framework by embedding topological features into a deep neural network for extracting semantics. We segment the input image and compute weight for each region with low-level features. The weighted segmentation result is called a topological map and it provides an additional channel for the CNN to emphasize the structural integrity and locality during the extraction of semantic features. We also utilize the topological map for saliency refinement based on a conditional random field at the end of our model. Experimental results on six benchmark datasets demonstrated that our proposed framework achieves competitive performance compared to other state-of-the-art methods.

SuperVAE: Superpixelwise Variational Autoencoder for Salient Object Detection

Image saliency detection has recently witnessed rapid progress due to deep neural networks. However, there still exist many important problems in the existing deep learning based methods. Pixel-wise convolutional neural network (CNN) methods suffer from blurry boundaries due to the convolutional and pooling operations. While region-based deep learning methods lack spatial consistency since they deal with each region independently. In this paper, we propose a novel salient object detection framework using a superpixelwise variational autoencoder (SuperVAE) network. We first use VAE to model the image background and then separate salient objects from the background through the reconstruction residuals. To better capture semantic and spatial contexts information, we also propose a perceptual loss to take advantage from deep pre-trained CNNs to train our SuperVAE network. Without the supervision of mask-level annotated data, our method generates high quality saliency results which can better preserve object boundaries and maintain the spatial consistency. Extensive experiments on five wildly-used benchmark datasets show that the proposed method achieves superior or competitive performance compared to other algorithms including the very recent state-of-the-art supervised methods.

A novel multi-graph framework for salient object detection

Graph-based methods have been widely adopted for predicting the most attractive region in an image. Most of the existing graph-based methods only utilize single graph to describe the image information, and thus cannot adapt for complex scenes. In this paper, a novel multi-graph framework for salient object detection is proposed. The proposed method is divided into three steps. Firstly, an image is divided into superpixels and described as a multi-graph, where superpixels are represented as nodes and their information is computed by color space and location space. Secondly, the multiple graphs are combined into a novel multi-graph-based manifold ranking propagation framework to obtain a coarse map. Finally, a map refinement model is developed to improve the quality of the coarse map. Experimental results on four challenging datasets show that the proposed method performs favorably against the state-of-the-art salient object detection methods.

Super Diffusion for Salient Object Detection.

One major branch of saliency object detection methods are diffusion-based which construct a graph model on a given image and diffuse seed saliency values to the whole graph by a diffusion matrix. While their performance is sensitive to specific feature spaces and scales used for the diffusion matrix definition, little work has been published to systematically promote the robustness and accuracy of salient object detection under the generic mechanism of diffusion. In this work, we firstly present a novel view of the working mechanism of the diffusion process based on mathematical analysis, which reveals that the diffusion process is actually computing the similarity of nodes with respect to the seeds based on diffusion maps. Following this analysis, we propose super diffusion, a novel inclusive learning-based framework for salient object detection, which makes the optimum and robust performance by integrating a large pool of feature spaces, scales and even features originally computed for non-diffusion-based salient object detection. A closed-form solution of the optimal parameters for the integration is determined through supervised learning. At the local level, we propose to promote each individual diffusion before the integration. Our mathematical analysis reveals the close relationship between saliency diffusion and spectral clustering. Based on this, we propose to re-synthesize each individual diffusion matrix from the most discriminative eigenvectors and the constant eigenvector (for saliency normalization). The proposed framework is implemented and experimented on prevalently used benchmark datasets, consistently leading to state-of-the-art performance.

Graph model-based salient object detection using objectness and multiple saliency cues

Recent years have witnessed increasing interest in salient object detection, which aims at stimulating the human visual attention mechanism to detect and segment the most attractive object in natural scenes, and can be widely applied in numerous computer vision tasks. In this paper, by considering both objectness cue and saliency detection, we propose a graph model-based bottom-up salient object detection framework by fusing multiple saliency maps using low-level features and objectness features under a manifold ranking framework. Specifically, for each feature, we utilize geodesic distance between any two superpixels to construct the affinity matrix and un-normalized Laplacian matrix of the graph. Then, we apply saliency optimization to refine each saliency map generated by manifold ranking with the first-stage query, and integrate saliency maps corresponding to different features by multilayer cellular automata in the final stage. Extensive experimental results demonstrate that our method can deliver promising performance in comparison to several state-of-the-art bottom-up methods on many benchmark datasets.

RGB-D Salient Object Detection via Minimum Barrier Distance Transform and Saliency Fusion

Automatic detection of salient objects in images has gained its popularity in computer vision field for its usage in numerous vision tasks in recent years. Depth information plays an important role in the human vision system while it is underutilized in most existing two-dimensional (2-D) saliency detection methods. In this letter, a multistage salient object detection framework via minimum barrier distance transform and saliency fusion based on multilayer cellular automata (MCA) is proposed. First, we independently generate the 3-D spatial prior, depth bias, and RGB-produced and depth-induced saliency maps. Next, the two saliency maps are weighted by depth bias to obtain two initial maps. Then, we adopt a saliency optimization step to generate more precise depth-induced saliency map. Moreover, the initial RGB-produced and the optimized depth-induced maps are further fused with 3-D spatial prior. Finally, we utilize MCA to fuse all saliency maps generated previously and obtain the final saliency result with complete salient object. The proposed method is evaluated on the publicly available benchmark dataset, RGBD1000. Compared to several state-of-the-art 2-D and depth-aware approaches, the experimental results demonstrate the effectiveness and superiority of our method, which can accurately detect the salient objects from RGB-D images, and has the most satisfactory overall performance.

A Two-Stage Bayesian Integration Framework for Salient Object Detection on Light Field

Unique visual features of 4D light field data have been shown to affect detection of salient objects. Nevertheless, only a few studies explore it yet. In this study, several helpful visual features extracted from light field data are fused in a two-stage Bayesian integration framework for salient object detection. First, background weighted color contrast is computed in high dimensional color space, which is more distinctive to identify object of interest. Second, focusness map of foreground slice is estimated. Then, it is combined with the color contrast results via first-stage Bayesian fusion. Third, background weighted depth contrast is computed. Depth contrast has been proved to be an extremely useful cue for salient object detection and complementary to color contrast. Finally, in the second-stage Bayesian fusion step, the depth-induced contrast saliency is further fused with the first-stage saliency fusion results to get the final saliency map. Experiments of comparing with eight existing state-of-the-art methods on light field benchmark datasets show that the proposed method can handle challenging scenarios such as cluttered background, and achieves the most visually acceptable salient object detection results.

Salient Object Detection Based on Background Feature Clustering

Automatic estimation of salient object without any prior knowledge tends to greatly enhance many computer vision tasks. This paper proposes a novel bottom-up based framework for salient object detection by first modeling background and then separating salient objects from background. We model the background distribution based on feature clustering algorithm, which allows for fully exploiting statistical and structural information of the background. Then a coarse saliency map is generated according to the background distribution. To be more discriminative, the coarse saliency map is enhanced by a two-step refinement which is composed of edge-preserving element-level filtering and upsampling based on geodesic distance. We provide an extensive evaluation and show that our proposed method performs favorably against other outstanding methods on two most commonly used datasets. Most importantly, the proposed approach is demonstrated to be more effective in highlighting the salient object uniformly and robust to background noise.