Image Saliency Detection Research Articles

Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks.

Eye movements in the case of freely viewing natural scenes are believed to be guided by local contrast, global contrast, and top-down visual factors. Although a lot of previous works have explored these three saliency cues for several years, there still exists much room for improvement on how to model them and integrate them effectively. This paper proposes a novel computation model to predict eye fixations, which adopts a multiresolution convolutional neural network (Mr-CNN) to infer these three types of saliency cues from raw image data simultaneously. The proposed Mr-CNN is trained directly from fixation and nonfixation pixels with multiresolution input image regions with different contexts. It utilizes image pixels as inputs and eye fixation points as labels. Then, both the local and global contrasts are learned by fusing information in multiple contexts. Meanwhile, various top-down factors are learned in higher layers. Finally, optimal combination of top-down factors and bottom-up contrasts can be learned to predict eye fixations. The proposed approach significantly outperforms the state-of-the-art methods on several publically available benchmark databases, demonstrating the superiority of Mr-CNN. We also apply our method to the RGB-D image saliency detection problem. Through learning saliency cues induced by depth and RGB information on pixel level jointly and their interactions, our model achieves better performance on predicting eye fixations in RGB-D images.

Salient object detection with spatiotemporal background priors for video.

Saliency detection for images has been studied for many years, for which a lot of methods have been designed. In saliency detection, background priors which are often regarded as pseudo-background are effective clues to find salient objects in images. Although image boundary is commonly used background priors, it doesn't work well for images of complex scenes and videos. In this paper, we explore how to identify the background priors for a video and propose a saliency based method to detect the visual objects by using background priors. For a video, we integrate multiple pairs of SIFT flows from long-range frames and a bidirectional consistency propagation is conducted to obtain the accurate and sufficient temporal background priors, which are combined with spatial background priors to generate spatiotemporal background priors. Next, a novel dual-graph based structure using spatiotemporal background priors is put forward in computation of saliency maps, fully taking advantage of appearance and motion information in videos. Experimental results on different challenging datasets show that the proposed method robustly and accurately detect the video objects in both simple and complex scenes and achieve better performance compared with other state-of-the-art video saliency models.

Salient object detection with low-rank approximation and ℓ2,1-norm minimization

Salient object detection is an important issue in computer vision and image procession in that it can facilitate humans to locate conspicuous visual regions in complex scenes rapidly and improve the performance of object detection and video tracking. In recent years, low-rank matrix approximation has been proved to be favorable in image saliency detection and gained a great deal of attention. An underlying assumption of low-rank recovery is that an image is a combination of background regions being low-rank and salient objects being sparse, which corresponds to tough non-smooth optimization problems. In this paper, by incorporating ℓ2,1-norm minimization, we obtain the corresponding smooth optimization problems and propose two effective algorithms with proved convergence. To guarantee the robustness of the proposed methods, the input image is divided into patches and each patch is approximately represented by its mean value. Besides, multi-scale visual features of each patch of the given image are extracted to capture common low-level features such as color, edge, shape and texture. The salient objects of a given image are indicated with sparse coefficients solved by the low-rank matrix approximation problem. Saliency maps are further produced with integration of the high-level prior knowledge. Finally, extensive experiments in four real-world datasets demonstrate that the proposed methods come with competitive performance over the eight compared state-of-the-arts.

Correspondence Driven Saliency Transfer.

In this paper, we show that large annotated data sets have great potential to provide strong priors for saliency estimation rather than merely serving for benchmark evaluations. To this end, we present a novel image saliency detection method called saliency transfer. Given an input image, we first retrieve a support set of best matches from the large database of saliency annotated images. Then, we assign the transitional saliency scores by warping the support set annotations onto the input image according to computed dense correspondences. To incorporate context, we employ two complementary correspondence strategies: a global matching scheme based on scene-level analysis and a local matching scheme based on patch-level inference. We then introduce two refinement measures to further refine the saliency maps and apply the random-walk-with-restart by exploring the global saliency structure to estimate the affinity between foreground and background assignments. Extensive experimental results on four publicly available benchmark data sets demonstrate that the proposed saliency algorithm consistently outperforms the current state-of-the-art methods.

Fast full resolution saliency detection based on incoherent imaging system

Image saliency detection is widely applied in many tasks in the field of the computer vision. In this paper, we combine the saliency detection with the Fourier optics to achieve acceleration of saliency detection algorithm. An actual optical saliency detection system is constructed within the framework of incoherent imaging system. Additionally, the application of our system to implement the bottom-up rapid pre-saliency process of primate visual saliency is discussed with dual-resolution camera. A set of experiments over our system are conducted and discussed. We also demonstrate the comparisons between our method and pure computer methods. The results show our system can produce full resolution saliency maps faster and more effective.

53-4: Image Saliency Detection based Depth Adjustment for Stereoscopic Image

53-4: Image Saliency Detection based Depth Adjustment for Stereoscopic Image

Saliency Detection With Spaces of Background-Based Distribution

In this letter, an effective image saliency detection method is proposed by constructing some novel spaces to model the background and redefine the distance of the salient patches away from the background. Concretely, given the backgroundness prior, eigendecomposition is utilized to create four spaces of background-based distribution (SBD) to model the background, in which a more appropriate metric (Mahalanobis distance) is quoted to delicately measure the saliency of every image patch away from the background. After that, a coarse saliency map is obtained by integrating the four adjusted Mahalanobis distance maps, each of which is formed by the distances between all the patches and background in the corresponding SBD. To be more discriminative, the coarse saliency map is further enhanced into the posterior probability map within Bayesian perspective. Finally, the final saliency map is generated by properly refining the posterior probability map with geodesic distance. Experimental results on two usual datasets show that the proposed method is effective compared with the state-of-the-art algorithms.

RODS: Rarity based Outlier Detection in a Sparse Coding Framework

Outlier detection has been an active area of research for a few decades. We propose a new definition of outlier that is useful for high-dimensional data. According to this definition, given a dictionary of atoms learned using the sparse coding objective, the outlierness of a data point depends jointly on two factors: the frequency of each atom in reconstructing all data points (or its negative log activity ratio, NLAR) and the strength by which it is used in reconstructing the current point. A R arity based O utlier D etection algorithm in a S parse coding framework (RODS) that consists of two components, NLAR learning and outlier scoring, is developed. This algorithm is unsupervised; both the offline and online variants are presented. It is governed by very few manually-tunable parameters and operates in linear time. We demonstrate the superior performance of the RODS in comparison with various state-of-the-art outlier detection algorithms on several benchmark datasets. We also demonstrate its effectiveness using three real-world case studies: saliency detection in images, abnormal event detection in videos, and change detection in data streams. Our evaluations shows that RODS outperforms competing algorithms reported in the outlier detection, saliency detection, video event detection, and change detection literature.

Multi-Scale Blobs for Saliency Detection in Satellite Images

Saliency can be modeled as spatially localized and contrasted structures with higher blob density and higher blob evenness across scales in images. And it is likely to contain regions and objects of interest. So saliency detection is desired before further image processing and analysis. This paper presented an automatic and effective method for saliency detection in satellite images, based on multi-scale blob information. Firstly, multi-scale blob information were extracted from input images, to produce a blob map. Then, in the blob map, multi-level distance transform spread the blob information to the entire image, to generate a saliency map. Finally the saliency map was segmented to detect salient regions and to locate object centers. The experimental results illustrated its accuracy and stability for detecting salient regions (such as residential areas, parking lots and airplane docks) and for locating object centers in various satellite images.

A hierarchical visual saliency detection method by combining distinction and background probability maps

We propose a bottom-up method for image saliency detection by combining the distinction map and the background probability map. The contributions of our work are as follows. First, a novel distinction measure is proposed by weighted combination of the color contrast and the spatial distance distribution criterions in previous works. Second, a background pixel distribution approximation method using patches sampled near the image borders is introduced. Finally, the distinction map and the background probability map are incorporated into a hierarchical framework to generate the final saliency map. We have compared our method with several recent works experimentally and observe that competitive results can be achieved.

Moving Target Detection Algorithm Based on Image Saliency Detection

Moving Target Detection Algorithm Based on Image Saliency Detection

Gradient Domain Guided Image Filtering.

Guided image filter (GIF) is a well-known local filter for its edge-preserving property and low computational complexity. Unfortunately, the GIF may suffer from halo artifacts, because the local linear model used in the GIF cannot represent the image well near some edges. In this paper, a gradient domain GIF is proposed by incorporating an explicit first-order edge-aware constraint. The edge-aware constraint makes edges be preserved better. To illustrate the efficiency of the proposed filter, the proposed gradient domain GIF is applied for single-image detail enhancement, tone mapping of high dynamic range images and image saliency detection. Both theoretical analysis and experimental results prove that the proposed gradient domain GIF can produce better resultant images, especially near the edges, where halos appear in the original GIF.

A flexible framework of adaptive method selection for image saliency detection

For most of the data analysis tasks (e.g. visual saliency detection), there are usually plenty of candidate methods to be selected. However, it is very difficult to choose a proper one for new instances, especially when the performances of these methods are with little difference overall. Though aggregation strategy aims to take advantage of the different methods, it often has the following weaknesses. Firstly, these methods often tend to combine the results from these candidate methods. Therefore, they suffer from high computation cost. Secondly, the performance may significantly degrade when there are obviously poor results. To address the two limitations above, we propose an instance-aware method selection approach which aims to select a single method instead of aggregating the results of all candidate ones. The proposed approach is based on the following observations: different methods often perform differently and the performance of a method often varies with respect to different instances. Hence, we devise the method selection manner to adaptively choose the best method for a specific instance. We transform the method selection problem into a multi-label annotation problem, which makes it general for many applications and flexible to employ metric learning technique.

Image saliency detection based on rectangular-wave spectrum analysis

Saliency detection is widely used in the fields of computer graphics and multimedia processing. Many computer graphics tasks, such as image segmentation, image labeling, and tracking, rely on the accurate generation of saliency maps. However, most current methods lack the ability to generate a fine boundary between the foreground and background while also providing a high recall rate. The saliency detection algorithm proposed in this paper is based on rectangular-wave spectrum analysis. In this method, we divide a given image into several regions, which are then convoluted using a pre-set rectangular-wave template. We determine the final saliency value by calculating the difference between a region and its adjacent regions, and its uniqueness compared with the entire image. Repeated tests using different data sets produced a high accuracy-recall rate. Moreover, the boundary in our saliency map is clear and fine.

Online Detection of Abnormal Events Using Incremental Coding Length

We present an unsupervised approach for abnormal event detection in videos. We propose, given a dictionary of features learned from local spatiotemporal cuboids using the sparse coding objective, the abnormality of an event depends jointly on two factors: the frequency of each feature in reconstructing all events (or, rarity of a feature) and the strength by which it is used in reconstructing the current event (or, the absolute coefficient). The Incremental Coding Length (ICL) of a feature is a measure of its entropy gain. Given a dictionary, the ICL computation does not involve any parameter, is computationally efficient and has been used for saliency detection in images with impressive results. In this paper, the rarity of a dictionary feature is learned online as its average energy, a function of its ICL. The proposed approach is applicable to real world streaming videos. Experiments on three benchmark datasets and evaluations in comparison with a number of mainstream algorithms show that the approach is comparable to the state-of-the-art.

Visual saliency and potential field data enhancements: Where is your attention drawn?

Interpretation of gravity and magnetic data for exploration applications may be based on pattern recognition in which geophysical signatures of geologic features associated with localized characteristics are sought within data. A crucial control on what comprises noticeable and comparable characteristics in a data set is how images displaying those data are enhanced. Interpreters are provided with various image enhancement and display tools to assist their interpretation, although the effectiveness of these tools to improve geologic feature detection is difficult to measure. We addressed this challenge by analyzing how image enhancement methods impact the interpreter’s visual attention when interpreting the data because features that are more salient to the human visual system are more likely to be noticed. We used geologic target-spotting exercises within images generated from magnetic data to assess commonly used magnetic data visualization methods for their visual saliency. Our aim was achieved in two stages. In the first stage, we identified a suitable saliency detection algorithm that can computationally predict visual attention of magnetic data interpreters. The computer vision community has developed various image saliency detection algorithms, and we assessed which algorithm best matches the interpreter’s data observation patterns for magnetic target-spotting exercises. In the second stage, we applied this saliency detection algorithm to understand potential visual biases for commonly used magnetic data enhancement methods. We developed a guide to choosing image enhancement methods, based on saliency maps that minimize unintended visual biases in magnetic data interpretation, and some recommendations for identifying exploration targets in different types of magnetic data.

Image Saliency Detection Based on Local and Regional Features

This paper proposes a model of salient region detection based on local and regional features in color space. Firstly, the image is divided into 8 × 8 sub-blocks, in each sub-block, then the multi-scale local and regional features are calculated and combined by weighted summation as the sub-block s salient value; secondly, the salient edge is calculated by the color contrast of the four color channels; finally, the salient map can be extracted by combining the salient features and salient edge together. The experiment results show that our model can extract salient objects in images fast and exactly.

Saliency detection for stereoscopic images.

Many saliency detection models for 2D images have been proposed for various multimedia processing applications during the past decades. Currently, the emerging applications of stereoscopic display require new saliency detection models for salient region extraction. Different from saliency detection for 2D images, the depth feature has to be taken into account in saliency detection for stereoscopic images. In this paper, we propose a novel stereoscopic saliency detection framework based on the feature contrast of color, luminance, texture, and depth. Four types of features, namely color, luminance, texture, and depth, are extracted from discrete cosine transform coefficients for feature contrast calculation. A Gaussian model of the spatial distance between image patches is adopted for consideration of local and global contrast calculation. Then, a new fusion method is designed to combine the feature maps to obtain the final saliency map for stereoscopic images. In addition, we adopt the center bias factor and human visual acuity, the important characteristics of the human visual system, to enhance the final saliency map for stereoscopic images. Experimental results on eye tracking databases show the superior performance of the proposed model over other existing methods.

Research on Image Saliency Detection Model Based on Calculating the Probability of Objectness Likelihood

Establishing calculation model on the probability of saliency objectness likelihood based on superpixels is introduced to dectect image saliency. At first,analysing factors which affect the size of visual attention according to the law of universal gravitation; And then use the SLIC algorithm to divide image into superpixels; Next, according to the texture, color,and gradient feature information, establish calculation models on probability saliency object under different rules: Including compactness in class color ,spatial distribution estimation and edge continuity; And then refer to the principle that activity in cells responding to stimuli,.a new feature combination theory is proposed to deal with the relationship of the characteristics of independence and mutual interaction for achieving features fusion accurately. Afterwards, the proposed algorithm applied in virtual and reality interaction to detect the effective region and eliminate noise area.

Saliency-guided improvement for hand posture detection and recognition

To detect and recognise hand postures against complex backgrounds, we propose a novel model that is constructed by the integration of image saliency and skin information. Although a skin model is a simple and efficient strategy by which to locate skin regions within images, it is easily affected by complex backgrounds, e.g. skin-like background regions and various lighting conditions. To solve this problem, we propose a general image saliency detection method that is then integrated with skin information to improve the performance of hand posture detection. Lastly, a linear Support Vector Machine (SVM) is adopted to recognise hand postures according to the results of hand posture detection. In the experiment, we tested the performance of the proposed image saliency detection method over seven state-of-the-art methods. The saliency-based hand posture detection and recognition model was also evaluated. These experiments show that the proposed model has stable performance for a wide range of images.