Saliency Detection Algorithms Research Articles

Saliency Detection Based on Multiple-Level Feature Learning.

Finding the most interesting areas of an image is the aim of saliency detection. Conventional methods based on low-level features rely on biological cues like texture and color. These methods, however, have trouble with processing complicated or low-contrast images. In this paper, we introduce a deep neural network-based saliency detection method. First, using semantic segmentation, we construct a pixel-level model that gives each pixel a saliency value depending on its semantic category. Next, we create a region feature model by combining both hand-crafted and deep features, which extracts and fuses the local and global information of each superpixel region. Third, we combine the results from the previous two steps, along with the over-segmented superpixel images and the original images, to construct a multi-level feature model. We feed the model into a deep convolutional network, which generates the final saliency map by learning to integrate the macro and micro information based on the pixels and superpixels. We assess our method on five benchmark datasets and contrast it against 14 state-of-the-art saliency detection algorithms. According to the experimental results, our method performs better than the other methods in terms of F-measure, precision, recall, and runtime. Additionally, we analyze the limitations of our method and propose potential future developments.

Studies Advanced in Salient Object Detection based on Deep Learning

A very important job in the area of computer vision is salient object detection, its goal is to check salient or obvious objects in images or videos. Thanks to rapidly developing networks of convolutive neurons, especially generative adversarial networks, CGAN has become a popular framework for salient object detection, which has promoted breakthroughs in detection precision and velocity of protruding objects. Focusing on the topic of salient object detection, in this paper, we systematically introduce the research progress of salient object detection. Specifically, we first introduce methods for detecting protracted objects based on general object detection algorithms (such as Fast R-CNN, YOLO, and SSD), including their basic ideas, key steps, advantages and disadvantages, etc. We then focus on CGAN-based salient object detection methods, including method classification, typical method introduction, and application examples. Finally, we quantitatively compare the performance of different saliency detection algorithms on common datasets, and discuss the existing research problems and feasible solutions of the topic.

What Catch Your Attention in SAR Images: Saliency Detection Based on Soft-Superpixel Lacunarity Cue

In existing superpixel-wise saliency detection algorithms, superpixel generation often is an isolated preprocessing step. The performance of saliency maps is determined by the accuracy of superpixels to a certain extent. However, it is still a challenge to develop a stable superpixel generation method. In this article, we attempt to incorporate the superpixel generation and saliency calculation steps into an end-to-end trainable deep network. First, we employ a recently proposed differentiable superpixel generation method to over-segment the synthetic aperture radar (SAR) images, which outputs the possibility that the pixels assigned to neighbor superpixels (soft superpixel). In saliency calculation part, as one of our main contributions, we propose a differentiable and computationally simple saliency model, i.e., lacunarity cue. It is inspired by the fact that generally the backscattering intensity of regions of interest (ROIs) in SAR images irregularly fluctuates, while the areas with consistent pixels are often ignored as the clusters. We improve the pixelwise box differential dimension algorithm to measure the irregularity of scattering points in a superpixel. The superpixel generation and saliency calculation can be implemented under a unified deep network. Hence, the shapes of the superpixels can be iteratively adjusted according to the saliency maps until the ROIs are correctly detected. Experiments on real SAR images with different sizes and scenes show that the saliency maps can effectively highlight the target areas, thus outperforming the state-of-the-art saliency detection models.

Online Learning And Saliency Effects On CNN-Based Gait Recognizers

Authentication through gait analysis offers a reliable and aneasy-to-use alternative to common authentication methods. This paper presents a novel gait recognizer that exploits online learning in Convolutional Neural Network, CNN. The features which make that algorithm promising are its high recognition accuracy and low computational cost, in addition to its adaptability, flexibility and applicability. Also, in a parallel line the effect of saliency as a means to generate global features is examined. In this paper, the inertial measures (instead of visual data) are utilized for person authentication. Thus the smartphone inertial sensors are used to continuously assess whether the mobile is actually in the hands of the right owner or not. Three different approaches (saliency detection, offline, and online learning) have been proposed, examined, and implemented. The last two of these approaches are based on the use of convolutional neural networks to shift the measured values of the sensors into a new vector that can be classified more reliably, while the first approach is based on the use of saliency detection algorithms to get the most salient regions of the gait. The models of these approaches are carried out and various experimentson such models are conducted. The results of these experiments were promising and showed the applicability of gait recognition to provide implicit continuous authentication, specially, when online learning is relied upon, since the identification accuracy reaches 98.7%.

Research on saliency detection method based on depth and width neural network

Image saliency detection is to segment the most important areas in the image. Solving the problem of image saliency detection usually involves knowledge in computer vision, neuroscience, cognitive psychology and other fields. In recent years, as deep learning has made great achievements in the field of computer vision, the application of deep learning has also played a good role in image saliency detection. Therefore, algorithms based on deep convolutional neural networks have become solutions to image saliency The most effective method of detection. For researchers, improving the computational efficiency of neural network-based saliency detection algorithms generally starts from two perspectives. One is to tailor the network structure and combine traditional feature extraction methods for processing. The other is to use a lighter network to solve the saliency detection problem. Based on these two points of thinking, this paper proposes two efficient and accurate neural network-based saliency detection algorithms. In recent years, with the rapid development of multimedia and Internet technologies, a huge amount of picture information is generated on blogs, social networking or shopping platforms every day. Such a lot of information not only enriches people's lives, but also provides efficient and accurate network management platforms. The management of these image information brings difficulties. Therefore, how to understand and process these image information more intelligently and efficiently has become a hot topic for many image processing and computer vision researchers. Among them, the saliency detection technology plays a key role in solving the problem of intelligent understanding and processing of images. To put it simply, saliency detection is a technology to automatically calculate or detect the most important areas in an image, and its processing results provide a basis for understanding and processing the image content. Saliency detection is a basic problem in computer vision, neuroscience and visual perception. The algorithm detects and extracts the most interesting or significant areas in the image.

Salient detection via the fusion of background-based and multiscale frequency-domain features

Salient object detection is a fundamental problem in image processing and computer vision. Many saliency detection algorithms based on the background and frequency-domain are used to extract salient object clues. However, the former causes the real object to be submerged in the detected object areas, especially in complex or small object scenes. While the latter will lead to the loss of some object information when detecting large objects. To solve these problems and achieve better object detection results, we propose a fusion framework for salient object detection by fusing background and frequency-domain features. The background features of the image are extracted by an improved background model. This model represents the spatial layout of the image area with respect to the image boundaries. Meanwhile, we present a new frequency-domain processing method to obtain multiscale frequency-domain features and mark the saliency of the object at different scales. Within our framework, inspired by human visual attention, we use the idea of a self-attention mechanism to capture the intrinsic relation between background and multiscale frequency-domain features. In addition, this fusion framework provides a three-dimensional Gaussian convolution kernel, which expands two-dimensional local information to three dimensions for feature fusion, thus producing more accurate salient objects. Experiment results demonstrate that the proposed method consistently outperforms eleven state-of-the-art methods on five challenging and complicated datasets in terms of four evaluation metrics.

A Fusion Model for Saliency Detection Based on Semantic Soft Segmentation

With the rapid development of neural networks in recent years, saliency detection based on deep learning has made great breakthroughs. Most deep saliency detection algorithms are based on convolutional neural networks, which still have great room for improvement in the edge accuracy of salient objects recognition, which may lead to fuzzy results in practical applications such as image matting. In order to improve the accuracy of detection, a saliency detection model based on semantic soft segmentation is proposed in this paper. Firstly, the semantic segmentation module combines spectral extinction and residual network model to obtain low-level color features and high-level semantic features, which can clearly segment all kinds of objects in the image. Then, the saliency detection module locates the position and contour of the main body of the object, and the edge accurate results are obtained after the processing of the two modules. Finally, compared with the other 11 algorithms on the DUTS-TEST data set, the weighted F-measure value of the proposed algorithm ranked first, which was 5.8% higher than the original saliency detection algorithm, and the accuracy was significantly improved.

Display Line Defect Detection Method Based on Color Feature Fusion

Display color line defect detection is an important step in the production quality inspection process. In order to improve the detection accuracy of low contrast line defects, we propose a display line defect detection method based on color feature fusion. The color saliency channels in the RG|GR and BY|YB channels were obtained using the relative entropy maximum criterion. Then, RG|GR were combined with the a channel and BY|YB with the b channel to calculate the red-green and the blue-yellow color fusion maps. The fusion color saliency map of the red-green and the blue-yellow color fusion maps was obtained by color feature fusion. Finally, the segmentation threshold was calculated according to the mean and standard deviation of the fusion color saliency map. The fused color saliency map was binarized and segmented to obtain a binary map of color line defects. The experimental results show that for the detection of multi-background offline defects, the detection accuracy of the algorithm in this paper is better than 90%, while other mainstreams fail to detect. Compared with state-of-the-art saliency detection algorithms, our method is capable of real-time low-contrast line defect detection.

Towards mesh saliency in 6 degrees of freedom

In this work, a novel 6DoF mesh saliency database is developed which provides both the subject’s 6DoF data and eye-movement data. Different from traditional databases, subjects in the experiment are allowed to move freely to view 3D meshes in the virtual reality environment. Based on the database, we first analyze the inter-observer variation and the influence of viewing direction toward subject’s visual attention, then we provide investigations about the subject’s visual attention bias and head movement during observation. As traditional 3D mesh saliency detection algorithms do not taking the subject’s head movement into consideration, we further propose a 6DoF mesh saliency detection algorithm based on the uniqueness measure and the bias preference. To evaluate the proposed approach, we design an evaluation metric accordingly which takes the 6DoF information into consideration, and extend some state-of-the-art 3D saliency detection methods to make comparisons. The experimental results demonstrate the superior performance of our approach for 6DoF mesh saliency detection, in addition to providing benchmarks for the presented 6DoF mesh saliency database. The database and proposed method will be made publicly available for research purposes.

In-Hand Object Localization Using a Novel High-Resolution Visuotactile Sensor

In-hand object localization and manipulation has always been a challenging task in robotic community. In this article, we address this problem by vision-based tactile sensing with high-spatial resolution. Specifically, we design a novel tactile sensor based on stereo vision, named GelStereo, which can perceive tactile point cloud with high-spatial resolution ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$&lt;\!1$</tex-math></inline-formula> mm). A tactile-based in-hand object localization pipeline composed of saliency detection and probabilistic point-set registration algorithms of the perceived contact point cloud is presented. Furthermore, extensive qualitative and quantitative analyses of perceived tactile point cloud and in-hand localization and insertion experiments of small parts are performed on our robot platform. The experimental results verify the accuracy and robustness of the tactile point cloud sensed by the novel GelStereo tactile sensor and the proposed in-hand object localization pipeline. This novel high-resolution visuotactile sensing technology has predictable application potential in the field of dexterous robotic manipulation.

Visual saliency detection via a recurrent residual convolutional neural network based on densely aggregated features

Current visual saliency detection algorithms based on deep learning suffer from reduced detection effect in complex scenes owing to ineffective feature expression and poor generalization. The present study addresses this issue by proposing a recurrent residual network based on dense aggregated features. Firstly, different levels of dense convolutional features are extracted from the ResNeXt101 network. Then, the features of all layers are aggregated under an Atrous spatial pyramid pooling operation, which makes comprehensive use of all possible saliency cues. Finally, the residuals are learned recurrently under a deep supervision mechanism to achieve continuous optimization of the saliency map. Application of the proposed algorithm to publicly available datasets demonstrates that the dense aggregation of features not only enhances the aggregation of effective information within a single layer, but also enhances external interactions between information at different feature levels. As a result, the proposed algorithm provides better detection ability than that of current state-of-the-art algorithms.

Research on saliency detection method based on depth and width neural network

Image saliency detection is to segment the most important areas in the image. Solving the problem of image saliency detection usually involves knowledge in computer vision, neuroscience, cognitive psychology and other fields. In recent years, as deep learning has made great achievements in the field of computer vision, the application of deep learning has also played a good role in image saliency detection. Therefore, algorithms based on deep convolutional neural networks have become solutions to image saliency The most effective method of detection. For researchers, improving the computational efficiency of neural network-based saliency detection algorithms generally starts from two perspectives. One is to tailor the network structure and combine traditional feature extraction methods for processing. The other is to use a lighter network to solve the saliency detection problem. Based on these two points of thinking, this paper proposes two efficient and accurate neural network-based saliency detection algorithms. In recent years, with the rapid development of multimedia and Internet technologies, a huge amount of picture information is generated on blogs, social networking or shopping platforms every day. Such a lot of information not only enriches people’s lives, but also provides efficient and accurate network management platforms. The management of these image information brings difficulties. Therefore, how to understand and process these image information more intelligently and efficiently has become a hot topic for many image processing and computer vision researchers. Among them, the saliency detection technology plays a key role in solving the problem of intelligent understanding and processing of images. To put it simply, saliency detection is a technology to automatically calculate or detect the most important areas in an image, and its processing results provide a basis for understanding and processing the image content. Saliency detection is a basic problem in computer vision, neuroscience and visual perception. The algorithm detects and extracts the most interesting or significant areas in the image.

RLP-AGMC: Robust label propagation for saliency detection based on an adaptive graph with multiview connections

There have been remarkable improvements in the salient object detection in the recent years. During the past few years, graph-based saliency detection algorithms have been proposed and made advances. Nevertheless, most of the state-of-the-art graph-based approaches are usually designed with low-level features, misleading assumption, fixed predefined graph structure and weak affinity matrix, which determine that they are not robust enough to handle images with complex or cluttered background.In this paper, we propose a robust label propagation-based mechanism for salient object detection throughout an adaptive graph to tackle above issues. Low-level features as well as deep features are integrated into the proposed framework to measure the similarity between different nodes. In addition, a robust mechanism is presented to calculate seeds based on the distribution of salient regions, which can achieve desirable results even if the object is in contact with the image boundary and the image scene is complex. Then, an adaptive graph with multiview connections is constructed based on different cues to learn the graph affinity matrix, which can better capture the characteristics between spatially adjacent and distant regions. Finally, a novel RLP-AGMC model, i.e. robust label propagation throughout an adaptive graph with multiview connections, is put forward to calculate saliency maps in combination with the obtained seed vectors. Comprehensive experiments on six public datasets demonstrate the proposed method outperforms fourteen existing state-of-the-art methods in terms of various evaluation metrics.

Salient target detection in hyperspectral image based on visual attention

Salient target detection in hyperspectral image is a significant task in image segmentation, target tracking, image classification and so on. Many existing saliency detection algorithms for hyperspectral image detection cannot present the boundary of the salient target well and the description of the target is not enough. A method based on visual attention to detect the salient target of hyperspectral image is proposed in this paper. In this method, frequency-tuned (FT) salient detection model is combined with spectral salient to detect target in hyperspectral image. FT model is used to get target with clear border, and spectral information is made full use of to improve the accuracy of target detection. Firstly, FT is used to detect saliency of hyperspectral image and the saliency map is generated. Then, spectral information of the hyperspectral image is measured by similarity, and the spectral saliency is obtained by calculating spectral angle distance between the spectral vectors. Finally, the FT's saliency map and the spectral saliency map are combined to form the final saliency target maps. Experimental results show that our method is superior to other methods in saliency target detection of hyperspectral image, and the precision-recall curve and F-measure are better as well.

A novel attention model for salient structure detection in seismic volumes

&lt;abstract&gt;&lt;p&gt;A new approach to seismic interpretation is proposed to leverage visual perception and human visual system modeling. Specifically, a saliency detection algorithm based on a novel attention model is proposed for identifying subsurface structures within seismic data volumes. The algorithm employs 3D-FFT and a multi-dimensional spectral projection, which decomposes local spectra into three distinct components, each depicting variations along different dimensions of the data. Subsequently, a novel directional center-surround attention model is proposed to incorporate directional comparisons around each voxel for saliency detection within each projected dimension. Next, the resulting saliency maps along each dimension are combined adaptively to yield a consolidated saliency map, which highlights various structures characterized by subtle variations and relative motion with respect to their neighboring sections. A priori information about the seismic data can be either embedded into the proposed attention model in the directional comparisons, or incorporated into the algorithm by specifying a template when combining saliency maps adaptively. Experimental results on two real seismic datasets from the North Sea, Netherlands and Great South Basin, New Zealand demonstrate the effectiveness of the proposed algorithm for detecting salient seismic structures of different natures and appearances in one shot, which differs significantly from traditional seismic interpretation algorithms. The results further demonstrate that the proposed method outperforms comparable state-of-the-art saliency detection algorithms for natural images and videos, which are inadequate for seismic imaging data.&lt;/p&gt;&lt;/abstract&gt;

Salient Object Detection Based on Multiscale Segmentation and Fuzzy Broad Learning

Abstract Saliency detection has been a hot topic in the field of computer vision. In this paper, we propose a novel approach that is based on multiscale segmentation and fuzzy broad learning. The core idea of our method is to segment the image into different scales, and then the extracted features are fed to the fuzzy broad learning system (FBLS) for training. More specifically, it first segments the image into superpixel blocks at different scales based on the simple linear iterative clustering algorithm. Then, it uses the local binary pattern algorithm to extract texture features and computes the average color information for each superpixel of these segmentation images. These extracted features are then fed to the FBLS to obtain multiscale saliency maps. After that, it fuses these saliency maps into an initial saliency map and uses the label propagation algorithm to further optimize it, obtaining the final saliency map. We have conducted experiments based on several benchmark datasets. The results show that our solution can outperform several existing algorithms. Particularly, our method is significantly faster than most of deep learning-based saliency detection algorithms, in terms of training and inferring time.

Revisiting Feature Fusion for RGB-T Salient Object Detection

While many RGB-based saliency detection algorithms have recently shown the capability of segmenting salient objects from an image, they still suffer from unsatisfactory performance when dealing with complex scenarios, insufficient illumination or occluded appearances. To overcome this problem, this article studies RGB-T saliency detection, where we take advantage of thermal modality's robustness against illumination and occlusion. To achieve this goal, we revisit feature fusion for mining intrinsic RGB-T saliency patterns and propose a novel deep feature fusion network, which consists of the multi-scale, multi-modality, and multi-level feature fusion modules. Specifically, the multi-scale feature fusion module captures rich contexture features from each modality feature, while the multi-modality and multi-level feature fusion modules integrate complementary features from different modality features and different level of features, respectively. To demonstrate the effectiveness of the proposed approach, we conduct comprehensive experiments on the RGB-T saliency detection benchmark. The experimental results demonstrate that our approach outperforms other state-of-the-art methods and the conventional feature fusion modules by a large margin.

Saliency detection hybrid information flows based on sub-network cascading

In view of the detail feature loss issue existing in the complex scenario of existing saliency detection algorithms, a fusion method of multi-layer sub-network cascade hybrid information flows is proposed in this paper. We first use the FCNs backbone network to obtain multi-scale features. Through the multi-layer sub-network layering mining to build a cascading network framework, the context information of the characteristic of each level is fully used. The detection and segmentation tasks are processed jointly. Multi-scale features are integrated by hybrid information flows, and more characteristic information with discernment is learned step by step. Finally, the embedded attention mechanism effectively compensates the deep semantic information as a mask, and further distinguishes the foreground and the messy background. Compared with the existing 9 algorithms on the basis of the 6 public datasets, the running speed of the proposed algorithm can reach 20.76 frames and the ex-perimental results are generally optimal on 5 evaluation indicators, even for the challenging new dataset SOC. The proposed method is obviously better than the classic algorithm. Experimental results were improved by 1.96%, 3.53%, 0.94%, and 0.26% for F-measure, weighted F-measure, S-measure, and E-measure, respectively. These experimental results show that the demonstrating the proposed model has higher accuracy and robustness and can be suitable for more complex environments, the proposed framework improves the performance significantly for state-of-the-art models on a series of datasets.

Saliency detection via multiple-morphological and superpixel based fast fuzzy C-mean clustering network

To model a human perception-based saliency detection algorithm in cluttered and noisy background images is a challenging problem in computer vision. Recently, many saliency detection algorithms have been proposed, which exploit the background information, or boundary priors of an image, to detect the salient object similar to the human attention system. These algorithms may not provide satisfying detection results for color images due to the assimilation of local spatial information. In this paper, we propose an unsupervised saliency detection technique, which uses multi-color space-based morphological gradient images. These gradient images contain different edge features, which are useful to obtain an accurate counter-based superpixel image containing both foreground and background clusters. A robust background measuring technique is implemented to remove background clusters, which describes the spatial information of an image cluster to image boundaries. This geometric clarification method effectively removes multiple low-level clues to produce a precise and uniform saliency map. These initially obtained saliency maps are fused using a multi-map fusion technique, and a compact saliency map prevails. The proposed algorithm is evaluated by executing different experiments on nine data sets. The results show that the proposed algorithm performs well for the detection of both single and multiple objects. The proposed algorithm is computationally efficient and provides better saliency detection results than state-of-the-art algorithms.

Video Saliency Detection via Pairwise Interaction

We propose a novel video saliency detection method based on pairwise interaction learning in this paper. Different from the traditional video saliency detection methods, which mostly combine spatial and temporal features, we adopt Least squares Conditional random field (LS-CRF) to capture the interaction information of regions within a frame or between video frames. Specifically, dual graph-connection models are built on superpixels structure of each frame for training and testing, respectively. In order to extract the essential scene structure from video sequences, LS-CRF is introduced to learn the background texture, object components and the various relationships between foreground and background regions through the training set, and each region will be distributed an inferred saliency value in testing phase. Benefitting from the learned diverse relations among scene regions, the proposed approach achieves reliable results especially on multiple objects scenes or under highly complicated scenes. Further, we substitute weak saliency maps for pixel-wise annotations in training phase to verify the expansibility and practicability of the proposed method. Extensive quantitative and qualitative experiments on various video sequences demonstrate that the proposed algorithm outperforms conventional saliency detection algorithms.