Generate Saliency Maps Research Articles

Saliency detection using multiple low-level priors and a propagation mechanism

The majority of existing methods for saliency detection based on low-level features failed to uniformly highlight the salient-object regions. In order to improve the accuracy and consistency of generated saliency maps, we propose a novel and efficient framework by combining low-level saliency priors and local similarity cues for image saliency detection. Firstly, we construct a multiple low-level prior map using location prior, color prior and background prior. Then, the prior maps employ a propagation mechanism based on Cellular Automata to enforce relevance of similar regions as a local similarity cue. Finally, a principle refinement framework by integrating multi-level prior maps and local similarity cue map are used to obtain an ultimate high-quality saliency map. Extensive experiments on publicly available datasets show that our designed approach is capable of producing accurate saliency maps compared with those generated results by the state-of-the-art saliency-detection methods.

Weakly Supervised Salient Object Detection by Learning A Classifier-Driven Map Generator.

Top-down saliency detection aims to highlight the regions of a specific object category, and typically relies on pixel-wise annotated training data. In this paper, we address the high cost of collecting such training data by a weakly supervised approach to object saliency detection, where only image-level labels, indicating the presence or absence of a target object in an image, are available. The proposed framework is composed of two collaborative CNN modules, an image-level classifier and a pixel-level map generator. While the former distinguishes images with objects of interest from the rest, the latter is learned to generate saliency maps by which the images masked by the maps can be better predicted by the former. In addition to the top-down guidance from class labels, the map generator is derived by also exploring other cues, including the background prior, superpixel- and object proposal-based evidence. The background prior is introduced to reduce false positives. Evidence from superpixels helps preserve sharp object boundaries. The clue from object proposals improves the integrity of highlighted objects. These different types of cues greatly regularize the training process and reduces the risk of overfitting, which happens frequently when learning CNN models with few training data. Experiments show that our method achieves superior results, even outperforming fully supervised methods.

Saliency for fine-grained object recognition in domains with scarce training data

This paper investigates the role of saliency to improve the classification accuracy of a Convolutional Neural Network (CNN) for the case when scarce training data is available. Our approach consists in adding a saliency branch to an existing CNN architecture which is used to modulate the standard bottom-up visual features from the original image input, acting as an attentional mechanism that guides the feature extraction process. The main aim of the proposed approach is to enable the effective training of a fine-grained recognition model with limited training samples and to improve the performance on the task, thereby alleviating the need to annotate a large dataset. The vast majority of saliency methods are evaluated on their ability to generate saliency maps, and not on their functionality in a complete vision pipeline. Our proposed pipeline allows to evaluate saliency methods for the high-level task of object recognition. We perform extensive experiments on various fine-grained datasets (Flowers, Birds, Cars, and Dogs) under different conditions and show that saliency can considerably improve the network’s performance, especially for the case of scarce training data. Furthermore, our experiments show that saliency methods that obtain improved saliency maps (as measured by traditional saliency benchmarks) also translate to saliency methods that yield improved performance gains when applied in an object recognition pipeline.

Saliency detection based on superpixel correlation and cosine window filtering

With the development of computer vision, image salient region detection plays an important role in the field of image processing. However, existing saliency detection methods can only generate saliency maps without achieving accuracy, thereby ignoring the integrity of the significant target. The present study proposed a saliency detection algorithm based on the correlation of superpixel and cosine window filtering (SPC) algorithm that introduced a variable weight superpixel segmentation method to keep the edge of information and enhance the integration of the salient target. Space distance and color distance are used to judge the correlation between two superpixels in the proposed algorithm, and based on the superpixel correlation, the saliency map is obtained by all superpixels’ voting. When the saliency map is corrected by a cosine window, we call this operation a cosine window filtering. Finally, we used the OSTU method for binarization and obtained the target position and size, using connected domain detection. The experiment results showed that the SPC algorithm can describe superpixels’ significance accurately and extract salient object. As compared with other algorithms, our algorithm offers higher accurate results, regarding the accurate position and size of salient targets.

Structural Correlates of Personality Dimensions in Healthy Aging and MCI.

The revised NEO Personality Inventory (NEOPI-R), popularly known as the five-factor model, defines five personality factors: Neuroticism, Extraversion, Openness to Experience, Agreeableness, and Conscientiousness. The structural correlates of these personality factors are still a matter of debate. In this work, we examine the impact of subtle cognitive deficits on structural substrates of personality in the elderly using DTI derived white matter (WM) integrity measure, Fractional Anisotropy (FA). We employed canonical correlation analysis (CCA) to study the relationship between personality factors of the NEOPI-R and FA measures in two population groups: healthy controls and MCI. Agreeableness was the only personality factor to be associated with FA patterns in both groups. Openness was significantly related to FA data in the MCI group and the inverse was true for Conscientiousness. Furthermore, we generated saliency maps using bootstrapping strategy which revealed a larger number of positive correlations in healthy aging in contrast to the MCI status. The MCI group was found to be associated with a predominance of negative correlations indicating that higher Agreeableness and Openness scores were mostly related to lower FA values in interhemispheric and cortico-spinal tracts and a limited number of higher FA values in cortico-cortical and cortico-subcortical connection. Altogether these findings support the idea that WM microstructure may represent a valid correlate of personality dimensions and also indicate that the presence of early cognitive deficits led to substantial changes in the associations between WM integrity and personality factors.

Single object tracking using particle filter framework and saliency-based weighted color histogram

Despite many years of research, object tracking remains a challenging problem, not only because of the variety of object appearances, but also because of the complexity of surrounding environments. In this research, we present an algorithm for single object tracking using a particle filter framework and color histograms. Particle filters are iterative algorithms that perform predictions in each iteration using particles, which are samples drawn from a statistical distribution. Color histograms are embedded in these particles, and the distances between histograms are used to measure likelihood between targets and observations. One downside of color histograms is that they ignore spatial information, which may produce tracking failure when objects appear that are similar in color. To overcome this disadvantage, we propose a saliency-based weighting scheme for histogram calculation. Given an image region, first its saliency map is generated. Next, its histogram is calculated based on the generated saliency map. Pixels located in salient regions have higher weights than those in others, which helps preserve the spatial information. Experimental results showed the efficiency of the proposed appearance model in object tracking under various conditions.

Eye Fixation Assisted Video Saliency Detection via Total Variation-based Pairwise Interaction.

As human visual attention is naturally biased towards foreground objects in a scene, it can be used to extract salient objects in video clips. In this work, we proposed a weakly supervised learning based video saliency detection algorithm utilizing eye fixations information from multiple subjects. Our main idea is to extend eye fixations to saliency regions step by step. First, visual seeds are collected using multiple color space geodesic distance based seed region mapping with filtered and extended eye fixations. This operation helps raw fixation points spread to the most likely salient regions, namely, visual seed regions. Second, in order to seize the essential scene structure from video sequences, we introduce the total variance based pairwise interaction model to learn the potential pairwise relationship between foreground and background within a frame or across video frames. In this vein, visual seed regions eventually grow into salient regions. Compared with previous approaches the generated saliency maps has two most outstanding properties: integrity and purity, which are conductive to segment the foreground and significant to the follow-up tasks. Extensive quantitative and qualitative experiments on various video sequences demonstrate that the proposed method outperforms state-of-theart image and video saliency detection algorithms.

SalNet360: Saliency maps for omni-directional images with CNN

The prediction of Visual Attention data from any kind of media is of valuable use to content creators and used to efficiently drive encoding algorithms. With the current trend in the Virtual Reality (VR) field, adapting known techniques to this new kind of media is starting to gain momentum. In this paper, we present an architectural extension to any Convolutional Neural Network (CNN) to fine-tune traditional 2D saliency prediction to Omnidirectional Images (ODIs) in an end-to-end manner. We show that each step in the proposed pipeline works towards making the generated saliency map more accurate with respect to ground truth data.

A novel method of optic disk segmentation based on visual saliency and rotary scanning

Fast optic disk localization and boundary segmentation is an important research topic in computer aided diagnosis. This paper proposes a novel method to effectively segment optic disk by using human visual characteristics in analyzing and processing fundus image. After a general analysis of optic disk features in fundus images, the target of interest could be located quickly, and intensity, color and spatial distribution of the disc are used to generate saliency map based on pixel distance. Then the adaptive threshold is used to segment optic disk. Moreover, to reduce the influence of vascular, a rotary scanning method is devised to achieve complete and continuous contour of optic disk boundary. Tests in the public fundus images database Drishti-GS have good performances, which mean that the proposed method is simple and rapid, and it meets the standard of the eye specialists. It is hoped that the method could be conducive to the computer aided diagnosis of eye diseases in the future.

Research on insect pest image detection and recognition based on bio-inspired methods

Insect pest recognition and detection are vital for food security, a stable agricultural economy and quality of life. To realise rapid detection and recognition of insect pests, methods inspired by human visual system were proposed in this paper. Inspired by human visual attention, Saliency Using Natural statistics model (SUN) was used to generate saliency maps and detect region of interest (ROI) in a pest image. To extract the invariant features for representing the pest appearance, we extended the bio-inspired Hierarchical Model and X (HMAX) model in the following ways. Scale Invariant Feature Transform (SIFT) was integrated into the HMAX model to increase the invariance to rotational changes. Meanwhile, Non-negative Sparse Coding (NNSC) is used to simulate the simple cell responses. Moreover, invariant texture features were extracted based on Local Configuration Pattern (LCP) algorithm. Finally, the extracted features were fed to Support Vector Machines (SVM) for recognition. Experimental results demonstrated that the proposed method had an advantage over the compared methods: HMAX, Sparse Coding and Natural Input Memory with Bayesian Likelihood Estimation (NIMBLE), and was comparable to the Deep Convolutional Network. The proposed method has achieved a good result with a recognition rate of 85.5% and could effectively recognise insect pest under complex environments. The proposed method has provided a new approach for insect pest detection and recognition.

Discrete stationary wavelet transform based saliency information fusion from frequency and spatial domain in low contrast images

Due to degraded visibility and low signal-to-noise ratio properties, traditional saliency detection models face great challenges toward low contrast images. In this circumstance, it is difficult to extract effective visual features to describe saliency information. To cope with this problem, this paper proposes a salient object detection model utilizing efficient features both from frequency domain and spatial domain in low contrast images. The discrete stationary wavelet transform (DSWT) is used to fuse the saliency information from frequency and spatial domain. The input image is firstly converted into HSV color space, where each color channel is transformed into frequency domain to adjust the amplitude spectrum by a median filter. Then, a superpixel-level feature extraction is utilized to generate saliency map from both local and global spatial information. Finally, the frequency and spatial domain saliency maps are fused via DSWT to obtain the final result. Experiments are carried out on three public datasets containing visible light condition and our low contrast image dataset to demonstrate the effectiveness of the proposed saliency detection model over other ten state-of-the-art saliency models.

Salient region detection and object segmentation in color images using dynamic mode decomposition

Estimation of visual saliency in images has become an important tool since it allows the processing of images without knowing the actual contents. In this paper we introduce a novel method to detect salient regions of an image using dynamic mode decomposition (DMD). The key idea is to utilize the analytical power of DMD, which is a powerful tool evolving in data science. The applicability of DMD in static image processing applications is made possible by developing a new way of image representation. The proposed algorithm utilizes color and luminance information to generate a full resolution saliency map. In order to model the non-linear behavior of human visual system we exploited the power of different color spaces including CIELab, YCbCr, YUV and RGB. The proposed method is computationally less expensive, simple and generates full resolution saliency maps.The effectiveness of the generated saliency map is evaluated and confirmed on three benchmark data sets across fourteen existing algorithms based on the standard performance measures such as F-measure, precision and recall curve, mean absolute error (MAE), area under ROC curve (AUC-Borji), normalized scanpath saliency (NSS) and Pearson's correlation coefficient (CC). We also propose a saliency driven transition region [SDTR] based segmentation to segment the salient object from images.

Edge Preserving and Multi-Scale Contextual Neural Network for Salient Object Detection.

In this paper, we propose a novel edge preserving and multi-scale contextual neural network for salient object detection. The proposed framework is aiming to address two limits of the existing CNN based methods. First, region-based CNN methods lack sufficient context to accurately locate salient object since they deal with each region independently. Second, pixel-based CNN methods suffer from blurry boundaries due to the presence of convolutional and pooling layers. Motivated by these, we first propose an end-to-end edge-preserved neural network based on Fast R-CNN framework (named RegionNet) to efficiently generate saliency map with sharp object boundaries. Later, to further improve it, multi-scale spatial context is attached to RegionNet to consider the relationship between regions and the global scenes. Furthermore, our method can be generally applied to RGB-D saliency detection by depth refinement. The proposed framework achieves both clear detection boundary and multi-scale contextual robustness simultaneously for the first time, and thus achieves an optimized performance. Experiments on six RGB and two RGB-D benchmark datasets demonstrate that the proposed method achieves state-of-the-art performance.In this paper, we propose a novel edge preserving and multi-scale contextual neural network for salient object detection. The proposed framework is aiming to address two limits of the existing CNN based methods. First, region-based CNN methods lack sufficient context to accurately locate salient object since they deal with each region independently. Second, pixel-based CNN methods suffer from blurry boundaries due to the presence of convolutional and pooling layers. Motivated by these, we first propose an end-to-end edge-preserved neural network based on Fast R-CNN framework (named RegionNet) to efficiently generate saliency map with sharp object boundaries. Later, to further improve it, multi-scale spatial context is attached to RegionNet to consider the relationship between regions and the global scenes. Furthermore, our method can be generally applied to RGB-D saliency detection by depth refinement. The proposed framework achieves both clear detection boundary and multi-scale contextual robustness simultaneously for the first time, and thus achieves an optimized performance. Experiments on six RGB and two RGB-D benchmark datasets demonstrate that the proposed method achieves state-of-the-art performance.

Enhanced decision fusion of semantically segmented images via local majority saliency map

Decision fusion is the most important step in ensemble machine learning schemes. One of the greatest challenges of decision fusion is the discrete nature of decisions. This challenge causes decision fusion solutions to become variations of the voting algorithm from statistical perspective. However, increasing the redundancy of decisions imposes a serious computational challenge for real-time systems. Resorting to fewer decisions imposes uncertainty challenges. In this Letter, the authors present a methodology to generate saliency maps for decision fusion. Specifically, they propose a local saliency map for decision fusion using a local majority filter. They choose semantic segmentation via pixel labelling produced from a random decision forest model as a case study. The local saliency map is used to derive three intermediate labelled images that are added to the voting pool and hence rectifying the final decision. The results of the proposed solution reduced the error by 26% and increased robustness by 16% with only two decisions.

Fabric defect detection via learned dictionary-based visual saliency

Purpose – Fabric defect detection plays an important role in textile quality control. The purpose of this paper is to propose a fabric defect detection algorithm using learned dictionary-based visual saliency. Design/methodology/approach – First, the test fabric image is splitted into image blocks, and the learned dictionary with normal samples and defective sample is constructed by selecting the image block local binary pattern features with highest or lowest similarity comparing with the average feature vector; second, the first L largest correlation coefficients between each test image block and the dictionary are calculated, and other correlation coefficients are set to zeros; third, the sum of the non-zeros coefficients corresponding to defective samples is used to generate saliency map; finally, an improve valley-emphasis method can efficiently segment the defect region. Findings – Experimental results demonstrate that the generated saliency map by the proposed method can efficiently outstand defect region comparing with the state-of-the-art, and segment results can precisely localize defect region. Originality/value – In this paper, a novel fabric defect detection scheme is proposed via learned dictionary-based visual saliency.

The Research on Lung Cancer Significant Detection Combined with Shape Feature of Target

At present, the research on detection of lung cancer includes the sample image segmentation, extracting visual features of lung cancer and generating the classification model by training learning, then according to the classification model generated to classify the inspected images. But this kind of method usually needs a large amount of calculation and speed is slow. In order to find the region of interest as soon as possible and improve the detection speed, this paper attempts to introduce the current popular Itti visual attention model into the lung cancer detection. However, because medical images usually have low contrast, the Itti method is not directly applied to extract the region of interest in medical image. Therefore the selective visual attention mechanism combined with shape feature of target is proposed. Firstly some primary features are chosen, such as gray scale, direction, corner point and edge to generate saliency map, and then the significant regions are segmented and judged. Compared to popular lung cancer detection method, this method can improve the detection rate of suspected lung cancer and has great significance for the early detection, early diagnosis and treatment of lung cancer.

Saliency Map Generation by the Convolutional Neural Network for Real-Time Traffic Light Detection Using Template Matching

A critical issue in autonomous vehicle navigation and advanced driver assistance systems (ADAS) is the accurate real-time detection of traffic lights. Typically, vision-based sensors are used to detect the traffic light. However, the detection of traffic lights using computer vision, image processing, and learning algorithms is not trivial. The challenges include appearance variations, illumination variations, and reduced appearance information in low illumination conditions. To address these challenges, we present a visual camera-based real-time traffic light detection algorithm, where we identify the spatially constrained region-of-interest in the image containing the traffic light. Given, the identified region-of-interest, we achieve high traffic light detection accuracy with few false positives, even in adverse environments. To perform robust traffic light detection in varying conditions with few false positives, the proposed algorithm consists of two steps, an offline saliency map generation and a real-time traffic light detection. In the offline step, a convolutional neural network, i.e., a deep learning framework, detects and recognizes the traffic lights in the image using region-of-interest information provided by an onboard GPS sensor. The detected traffic light information is then used to generate the saliency maps with a modified multidimensional density-based spatial clustering of applications with noise (M-DBSCAN) algorithm. The generated saliency maps are indexed using the vehicle GPS information. In the real-time step, traffic lights are detected by retrieving relevant saliency maps and performing template matching by using colour information. The proposed algorithm is validated with the datasets acquired in varying conditions and different countries, e.g., USA, Japan, and France. The experimental results report a high detection accuracy with negligible false positives under varied illumination conditions. More importantly, an average computational time of 10 ms/frame is achieved. A detailed parameter analysis is conducted and the observations are summarized and reported in this paper.

Motion saliency detection based on temporal difference

We focus on motion saliency detection, which has attracted much attention in recent years. Distinct from conventional algorithms, without considering the spatial information of the input frames, our method is solely based on the temporal difference of corresponding pixels. To be specific, the difference is modeled as two parts, i.e., symmetric frame difference and background sample difference. The first term calculates the differences of adjacent frames, including not only the previous n frames but also the backward n frames. To obtain the background sample difference, N samples taken from the previous frames are stored, and the differences between the coming pixel and all its corresponding samples are accumulated. The initialization and updating policies of the samples follow the off-the-shelf visual background extractor method. Finally, the motion saliency map is generated by fusing the aforementioned two features. We describe our method in full detail and compare it with three other state-of-the-art motion detection techniques. Experiments on a variety of visual surveillance databases verify that the proposed algorithm consistently outperformed the compared methods. We also demonstrate how the generated saliency maps can be used to create high-quality segmentation masks for moving object detection.

Hierarchical salient object detection model using contrast-based saliency and color spatial distribution

Visual saliency is an important cue in human visual system to detect salient objects in natural scenes. It has attracted a lot of research focus in computer vision, and has been widely used in many applications including image retrieval, object recognition, image segmentation, and etc. However, the accuracy of salient object detection model remains a challenge. Accordingly, a hierarchical salient object detection model is presented in this paper. In order to accurately interpret object saliency in image, we propose to investigate distinctive features from a global perspective. Image contrast and color distribution are calculated to generate saliency maps respectively, which are then fused using the principal component analysis. Compared with state-of-the-art models, the proposed model can accurately detect the salient object which conform with the human visual principle. The experimental results from the MSRA database validate the effectiveness of our proposed model.

A generalized nonlocal mean framework with object-level cues for saliency detection

Nonlocal mean (NM) is an efficient method for many low-level image processing tasks. However, it is challenging to directly utilize NM for saliency detection. This is because that conventional NM method can only extract the structure of the image itself and is based on regular pixel-level graph. However, saliency detection usually requires human perceptions and more complex connectivity of image elements. In this paper, we propose a novel generalized nonlocal mean (GNM) framework with the object-level cue which fuses the low-level and high-level cues to generate saliency maps. For a given image, we first use uniqueness to describe the low-level cue. Second, we adopt the objectness algorithm to find potential object candidates, then we pool the object measures onto patches to generate two high-level cues. Finally, by fusing these three cues as an object-level cue for GNM, we obtain the saliency map of the image. Extensive experiments show that our GNM saliency detector produces more precise and reliable results compared to state-of-the-art algorithms.