Salient Region Detection Method Research Articles

CSWS—Suppress confusion subregion in indoor image fingerprint localization

AbstractImage data can provide rich content information, which has attracted a lot of attention in the field of indoor fingerprint positioning. However, indoor image information from different locations is characterized by high content repetition, and these repetitive regions can cause the problem of insufficient differentiation of adjacent image fingerprints or even fingerprint misclassification. To solve this problem, this article proposes a confusion subregion weighted suppression strategy in the image fingerprint database. First, duplicate regions (confusion subregions) are extracted from the fingerprint database using an appropriate salient region detection method. Then the similarity of these duplicate regions in Euclidean space is defined, and the degree of influence of these regions on the distinguishability of the fingerprint database is measured. Finally, the suppression of these duplicate regions is achieved in the original image fingerprint database by introducing weighted suppression coefficients. Experiments show that the algorithm proposed in this article can achieve significant results in the fingerprint localization task of real indoor scenes and effectively improve the localization accuracy.

VCSNet: V‐fused color saliency net

SummaryImage salient region detection methods, which can detect and extract interesting regions in pictures, have been a hot research direction in recent years. Most of current salient region detection algorithms and corresponding training datasets originated from general visual attention processes and primarily reflected attention to object shape in pictures. While color vision provided more useful information in vision systems, the color effect of visual attention should be and must be considered. First, we collected cue data of painting pictures from several observers using eye‐track recording technology when observers were asked to pay attention to color information of various paintings. Second, we constructed a color attention dataset, color saliency dataset (CSD), from the cue data and pictures. Thirdly, we designed a V‐fused color saliency net (VCSNet) model which included three modules: a color information fusion module, a prediction module, and an optimization module and trained the model using the CSD. Finally, we compared our method with previous algorithms on the CSD, and results showed that our method outperformed the previous algorithms in color saliency detection with MAE of 0.057 and Fmax of 0.265. We open source part of the self‐created dataset: https://github.com/InfiniteEM/ColorSaliencyDataSet.

Structure‐aware multiple salient region detection and localization for autonomous robotic manipulation

This paper proposes a multiple salient region detection and localization approach for unstructured industrial robot work environments with arbitrarily located and orientated objects. Different from the existing, the authors' novel technique to detect multiple salient regions performs locally adaptive center-surround operations on proto-object partitions obtained through color consistency and spatial proximity analysis. The multi-scale center-surround operations are done by masks that are local structure-aware yielding regions with precise and accurate boundaries as required for robotic manipulation. First, experiments to evaluate the multiple salient region detection performance are carried out using four standard databases having images with multiple salient objects. Quantitative result analysis using F-measure, shuffled F-measure, shuffled AUC and MAE, and subjective result inspection suggests that the proposed approach is in general better at collectively detecting multiple salient regions than the state-of-the-art, including those based on deep learning. Then, real-life experiments involving robotic manipulation are carried out to demonstrate the utility of the multiple salient region detection method. For robotic manipulation, object localization is improved after salient region detection by employing a fast shadow detection algorithm proposed based on hue analysis, and recognition through existing matching techniques is applied only at the localized salient regions. The benefit of the novel multiple salient region detection approach in the robotic manipulation system is shown using localization and pose estimation accuracy, rates of detection and recognition, positional and angular errors, and processing speed.

Salient Region Guided Blind Image Sharpness Assessment.

Salient regions provide important cues for scene understanding to the human vision system. However, whether the detected salient regions are helpful in image blur estimation is unknown. In this study, a salient region guided blind image sharpness assessment (BISA) framework is proposed, and the effect of the detected salient regions on the BISA performance is investigated. Specifically, three salient region detection (SRD) methods and ten BISA models are jointly explored, during which the output saliency maps from SRD methods are re-organized as the input of BISA models. Consequently, the change in BISA metric values can be quantified and then directly related to the difference in BISA model inputs. Finally, experiments are conducted on three Gaussian blurring image databases, and the BISA prediction performance is evaluated. The comparison results indicate that salient region input can help achieve a close and sometimes superior performance to a BISA model over the whole image input. When using the center region input as the baseline, the detected salient regions from the saliency optimization from robust background detection (SORBD) method lead to consistently better score prediction, regardless of the BISA model. Based on the proposed hybrid framework, this study reveals that saliency detection benefits image blur estimation, while how to properly incorporate SRD methods and BISA models to improve the score prediction will be explored in our future work.

Exploiting Surroundedness and Superpixel cues for salient region detection

In this paper, we will present a new salient region detection method by exploiting its surrounding and superpixel cues. Its main highlights are: 1) An input image is quantized to 256 colors by using minimum variance quantization; 2) Saliency maps is computed based on the figure-ground segregation of the quantized image; 3) Mean saliency value of each superpixel is employed to refine saliency maps further. This can highlight salient objects robustly and suppress backgrounds evenly. Experimental results show that the proposed method produces more accurate saliency maps and performs well against twenty-one saliency models concerning three evaluation metrics on two public datasets.

Significant obstacle location with ultra-wide FOV LWIR stereo vision system

Intelligent driving is an active area of research in both industry and academia. In order to overcome the shortcomings of traditional machine vision such as visibility is easily affected by illumination conditions, the blind area of infrared small field of view (FOV) is too large and could not provide depth information, this paper proposes a method for detecting significant obstacles based on ultra-wide FOV long-wave infrared (LWIR) stereo vision system. The stereo vision positioning location with ultra-wide FOV is established by the generalized fisheye camera model. On the basis of analyzing obstacle imaging scale and the structure characteristics of the proposed stereo vision system, a multi-scale salient region detection method based on composite pattern is proposed, and its implementation process is described in detail. Experiment shows that the proposed ultra-wide FOV LWIR stereo vision system is able to detect and locate significant obstacles in ultra-wide FOV and the detection rate of pedestrians and vehicles in real complex street scenes is over 92.6%. At the same time, the relative error of pedestrian positioning with a distance of 5 m to 30 m near the central FOV is between 1.6%-10.3%, and its spatial location ability and advantages are verified. The proposed stereo vision system effectively overcomes the shortcomings of existing vision systems, expands the scope of machine vision, and can be used in the field of assistant driving and intelligent driving.

Salient region detection through salient and non-salient dictionaries.

Low-rank representation-based frameworks are becoming popular for the saliency and the object detection because of their easiness and simplicity. These frameworks only need global features to extract the salient objects while the local features are compromised. To deal with this issue, we regularize the low-rank representation through a local graph-regularization and a maximum mean-discrepancy regularization terms. Firstly, we introduce a novel feature space that is extracted by combining the four feature spaces like CIELab, RGB, HOG and LBP. Secondly, we combine a boundary metric, a candidate objectness metric and a candidate distance metric to compute the low-level saliency map. Thirdly, we extract salient and non-salient dictionaries from the low-level saliency. Finally, we regularize the low-rank representation through the Laplacian regularization term that saves the structural and geometrical features and using the mean discrepancy term that reduces the distribution divergence and connections among similar regions. The proposed model is tested against seven latest salient region detection methods using the precision-recall curve, receiver operating characteristics curve, F-measure and mean absolute error. The proposed model remains persistent in all the tests and outperformed against the selected models with higher precision value.

Graph-based particular object discovery

Severe background clutter is challenging in many computer vision tasks, including large-scale image retrieval. Global descriptors, which are popular due to their memory and search efficiency, are especially prone to corruption by such a clutter. Eliminating the impact of the clutter on the image descriptor increases the chance of retrieving relevant images and prevents topic drift due to actually retrieving the clutter in the case of query expansion. In this work, we propose a novel salient region detection method. It captures, in an unsupervised manner, patterns that are both discriminative and common in the dataset. Saliency is based on a centrality measure of a nearest neighbor graph constructed from regional CNN representations of dataset images. The proposed method exploits recent CNN architectures trained for object retrieval to construct the image representation from the salient regions. We improve particular object retrieval on challenging datasets containing small objects.

Salient Region Detection Using Diffusion Process with Nonlocal Connections

Diffusion-based salient region detection methods have gained great popularity. In most diffusion-based methods, the saliency values are ranked on 2-layer neighborhood graph by connecting each node to its neighboring nodes and the nodes sharing common boundaries with its neighboring nodes. However, only considering the local relevance between neighbors, the salient region may be heterogeneous and even wrongly suppressed, especially when the features of salient object are diverse. In order to address the issue, we present an effective saliency detection method using diffusing process on the graph with nonlocal connections. First, a saliency-biased Gaussian model is used to refine the saliency map based on the compactness cue, and then, the saliency information of compactness is diffused on 2-layer sparse graph with nonlocal connections. Second, we obtain the contrast of each superpixel by restricting the reference region to the background. Similarly, a saliency-biased Gaussian refinement model is generated and the saliency information based on the uniqueness cue is propagated on the 2-layer sparse graph. We linearly integrate the initial saliency maps based on the compactness and uniqueness cues due to the complementarity to each other. Finally, to obtain a highlighted and homogeneous saliency map, a single-layer updating and multi-layer integrating scheme is presented. Comprehensive experiments on four benchmark datasets demonstrate that the proposed method performs better in terms of various evaluation metrics.

Spatial–temporal multi-task learning for salient region detection

This paper proposes a novel multi-task learning based salient region detection method by fusing spatial and temporal features. Salient region detection has been widely used in various computer vision tasks, being as a general preprocessor to identify interest objects. Despite the recent successes, existing saliency models still lag behind the performance of human when visually perceives dynamic scenes. Most of the existing models largely rely on various spatial features. However, these spatial feature based methods have several deficiencies: (i) they can hardly adapt to the situation where moving objects are included, and (ii) they cannot model the human vision process in dynamic scenes. Recently, some saliency models introduce temporal features in their detecting process, such as the optical flow and stacking frames. The potential of temporal features for saliency optimization has been demonstrated. However, since temporal features in these models are merely used as a compensation to static features, the advantages of temporal features have not yet been fully explored. Aiming to comprehensively address these issues above, our method fuses spatial and temporal features, and learns the mapping relationship from various features to salient regions using our multi-task learning framework. The final salient region is generated by our unified Bayesian framework. The experimental results demonstrated that our proposed approach outperforms previous methods.

Salient region detection using feature extraction in the non‐subsampled contourlet domain

The human visual system is attracted to the most dominant part of the image which is called salient region. There has been a surge of interest in the past few years to efficiently detect the salient regions of images. In this study, a new salient region detection method is proposed using the non-subsampled contourlet transform. It is known that this transform is capable of providing a multiscale, multi-directional and translation invariant decomposition of images. The proposed saliency detection method is realised by extracting various local and global features from the non-subsampled contourlet coefficients of the colour channels. A saliency map is obtained based on a linear combination of the local features and the distribution of the global features. In order to provide a better preservation of the structure and boundary of the objects and to obtain a more uniformly highlighted salient region, the saliency map is abstracted using an optimisation framework. Several experiments are conducted on sets of natural images to evaluate the performance of the proposed method. The results show that the performance of the proposed method is superior to that of the other existing methods in terms of precision-recall performance, F-measure, and mean absolute error values.

Object recognition in clutter color images using Hierarchical Temporal Memory combined with salient-region detection

Abstract The essential goal of this paper is to extend the functionality of the bio-inspired intelligent HTM (Hierarchical Temporal Memory) network towards two capabilities: (i) object recognition in color images, (ii) detection of multiple objects located in clutter color images. The former extension is based on the development of a novel scheme for the application of three parallel HTM networks that separately process color, texture, and shape information in color images. For the latter HTM extension, we propose a novel system in which HTM is combined with a modified model of computational visual attention. We adopt the results of Bi et al. (2010), Hu et al. (2005), and Kucerova (2011) and add new elements for the calculation of image saliency maps. The proposed algorithm enables one to automatically locate individual objects in clutter images. For computer experiments, a special image database is created to simulate ideal single object images and cluttered images with multiple objects on an inhomogeneous background. The recognition performance of HTM alone and in combination with the salient-region detection method is evaluated. We show that the attention subsystem is able to satisfactorily locate multiple objects in clutter color images with an inhomogeneous background. We also perform benchmark calculations for two selected computer vision methods used for object detection in color clutter images. Namely, the cascade detector and template matching methods are used. Our study confirms that the proposed attention system can improve the capabilities of HTM for object classification in cluttered images. The compound system of visual attention and HTM outperforms the compared methods in both criteria (recall and correct detection rate). However, as expected, the system cannot match the recognition accuracy achieved by HTM for single object images, and thus, further research is needed.

A Method for Salient Region Detection by Fusing Prior Information

Human visual system has tendency to concentrate on some region in an image. Such regions which draw most of the attention are known as salient region. Finding these regions from an image is known as salient region detection. Saliency maps are built to represent such regions. Because of the wide applicability in computer vision, pattern recognition and guiding robots, researchers have gained lot of interest in this field. Various parameters are used for salient region detection like color, contrast, frequency, location etc. In this paper, contrast based method is proposed and two facts are utilized that generally centre of an image gains most of the attention (centre prior) and salient region rarely touches the image boundary (boundary prior). Intermediate maps are built using centre prior and boundary prior. Final saliency map is obtained by combining two intermediate maps. The results of proposed approach are validated by performing experiments on MSRA dataset and comparing it with 3 state-of-arts.

Color-Boosted Saliency-Guided Rotation Invariant Bag of Visual Words Representation with Parameter Transfer for Cross-Domain Scene-Level Classification

Scene classification on remote sensing imagery is usually based on supervised learning but collecting labelled data in remote sensing domains is expensive and time-consuming. Bag of Visual Words (BOVW) achieves great success in scene classification but there exist problems in domain adaptation tasks, such as the influence of background and the rotation transformation on BOVW representation, and the transfer of SVM parameters from the source domain to the target domain, which may lead to decreased cross-domain scene classification performance. In order to solve the three problems, Color-boosted saliency-guided rotation invariant bag of visual words representation with parameter transfer is proposed for cross-domain scene classification. The global contrast-based salient region detection method is combined with the color-boosted method to increase the accuracy of detected salient regions and reduce the effect of background information on the BOVW representation. The rotation invariant BOVW representation is also proposed by sorting the BOVW representation in each patch in order to decrease the effect of rotation transformation. The several best configurations in the source domain are also applied to the target domain so as to reduce the distribution bias between scenes in the source and target domain. These configurations deliver the top classification performance the optimal parameter in the target domain. The experimental results on two benchmark datasets confirm that the proposed method outperforms most previous methods in scene classification when instances in the target domain are limited. It is also proved that color boosted global contrast-based salient region detection (CBGCSRD) method, rotation invariant BOVW representation, and transfer of SVM parameters from the source to the target domain are all effective in improving the classification accuracy with 2.5%, 3.3%, and 3.1%. These three contributions may increase about 7.5% classification accuracy in total.

Adaptive recommendation for photo pose via deep learning

With the development of image acquisition devices and the popularity of smart phones, more and more people would like to upload their photos to diverse social networks. It is hard to guarantee the quality and artistry of these photos because of not everyone is a professional photographer. In order to handle this problem and further help each common user to improve the beauty of photos, we propose an intelligent photo pose recommendation method to recommended professional photo pose according to everyone’s posture in viewfinder. Firstly, the CNN model (VGG-16) is utilized to extract the global features for each photo. Secondly, the salient region detection method is leveraged to extract the regions of interest in each photo. To represent the edge distribution in the local regions, we extract the histogram of oriented gradients. Finally, we propose an effective feature fusion method based on CCA to generate the global visual features for each photo. We implement the Euclidean distance to handle the similarity measure between uploaded photos and the professional photos. The most similar professional photo will be utilized to guide user photo composition. In order to evaluate the performance of the proposed method, we collected a set of professional photos form some professional photography websites. The comparison experiments and user study demonstrate the superiority of the proposed approach.

Salient region detection using efficient wavelet-based textural feature maps

A salient region is part of an image that captures the highest level of attention by the human visual system. In this paper, a new salient region detection method is proposed by linearly combining the feature maps for the L, a and b color channels. Since, the wavelet transform is capable of providing a multi-scale spatial-frequency decomposition of the image, the color feature maps are obtained using this transform. A scheme is proposed whereby the channel feature maps are linearly combined. The weights for the linear combination are determined by making use of the entropy of the channel feature maps and a Gaussian kernel, utilizing the fact that the salient objects are generally clustered and scene-centric. The salient region is further refined by making use of the proximity of the pixels to the centers of gravity in the image feature map. Extensive simulations are conducted in order to evaluate the performance of the proposed saliency detection scheme by applying it to the natural images from several datasets. Experimental results show that the proposed method provides values of precision, recall and F-measure larger than and that of the mean absolute error smaller than those provided by other existing methods. The performance of the proposed salient region detection method is also evaluated on noisy images and it is shown to be robust against noise.

Joint upsampling of random color distance maps for fast salient region detection

The human visual system is capable of rapid response, even in the presence of massive quantities of visual information. This is possible because it restricts the operation of further processing stages to a small, potentially important, subset of the incoming information. This mechanism is called visual attention and is drawn by distinctive, visually salient, regions of the scene. Detection of visually salient regions is widely employed in vision-based applications, since a reduction in visual search space can lead to significant improvement in computational performance. Despite recent advances in salient region detection, most efforts have focused on improving accuracy, at the expense of increased execution time, significantly hindering their applicability. To address this, a fast and accurate salient region detection method is presented in this work, based on an efficient saliency estimate called random color distance map. This estimate is joint upsampled into an accurate saliency map, which is assessed and compared to saliency maps obtained by other four state-of-the-art methods on the MSRA1K, MSRA10K and SED2 datasets, showing that it is highly competitive in both accuracy and execution time.

Maximum mean discrepancy regularized sparse reconstruction for robust salient regions detection

Sparse reconstruction based saliency detection methods are gaining popularity for object detection and content-based image recovery due to its simplicity and easy understandability. It helps in extracting concise representations of the stimuli and capture high-level semantics in visual information with a few active coefficients. Different from the conventional sparse representation techniques that highlight the borders of the salient objects only, we propose a novel regularized sparse coding method that preserves the similarity and locality to achieve smoothness in sparse representation to evenly highlight the entire salient object. In this study, we propose a novel Maximum Mean Discrepancy (MMD) regularized sparse representation method for salient region detection. Initially, the resemblance and locality of super pixels are conserved by constructing a graph regularization term to increase the fidelity of the salient coefficient score of the visual part. Secondly, the distributions divergence among the similar regions is alleviated by constructing a MMD regularized term. Furthermore, the re-constructive background dictionary is extracted from background pixels that are enriched with visual and geometrical information. The results computed through this dictionary are more accurate in terms of background suppression. We analyze our model on four largest benchmark datasets using five evaluation metrics that reveal the fact that the performance of the proposed model is satisfactory and favorably against the existing state-of-the-art schemes.

A Novel Salient Region Detection Method Based on Hierarchical Spatial Information

Different patterns in one object will cause unequal saliency degree which makes it hard to highlight the object region uniformly. We propose a salient region detection method which mainly includes image abstraction, saliency calculation and integration. Under the detection framework, the hierarchical spatial information is introduced to improve the performance. The image abstraction with “pixel level” spatial information is applied to capture some meaningful elements. The local contrast is calculated with the “element level” spatial information. The “object level” spatial information is represented as compactness and background possibility, which further help to better pop out the object region and suppress the background. The results show that our method has a good performance even though the object consists of complex patterns.

Complex networks driven salient region detection based on superpixel segmentation

In this paper, we propose an efficient method for salient region detection. First, the image is decomposed by using superpixel segmentation which groups similar pixels and generates compact regions. Based upon the generated superpixels, similarity between the regions is calculated by benefiting from color, location, histogram, intensity, and area information of each region as well as community identification via complex networks theory in the over-segmented image. Then, contrast, distribution and complex networks based saliency maps are generated by using the mentioned features. These saliency maps are used to create a final saliency map. The applicability, effectiveness and consistency of the proposed approach are illustrated by conducting some experiments using publicly available datasets. The tests have been used to compare the proposed method with some state-of-the-art methods. The reported results cover qualitative and quantitative assessments which demonstrate that our approach outputs high quality saliency maps and mostly achieves the highest precision rate compared to the other methods.