Visual Salient Region Research Articles

Regional importance detection of 3D mesh via fusion of local color difference and curvature entropy

Predicting relative visual salient regions on three-dimensional (3D) meshes benefits many computer graphics applications. Most computation models for mesh saliency focus on geometrical information alone. Nevertheless, the ignored texture, lighting and material also provide more detailed appearance information, especially in the context of static scene rendering. In this paper, we propose a mesh saliency detection algorithm considering both geometrical and colorimetric information to address this challenge. Our model first computes the local curvature entropy at multi-scale to capture the geometrical details. Second, a set of images are projected onto the screen at several viewpoints with specified material and lighting model. Potentially salient regions on the rendered images are detected by fusion of multiple color difference maps measured with an approximated multi-scale Laplacian of Gaussian filter. A Gaussian distribution-based central bias model is applied to the image saliency map to emphasize the global rarity of salient regions. Third, the saliency maps of rendered images are projected back to the 3D mesh via the ray casting method. In the end, both saliency maps are combined linearly as the saliency map of 3D mesh. Experiment on the human fixation database demonstrates the performance of our method compared to the classic methods in terms of linear correlation coefficient and AUC.

Depth-aware salient object segmentation

Depth-aware salient object segmentation

Coarse-to-fine salient object detection with low-rank matrix recovery

Low-rank matrix recovery (LRMR) has recently been applied to saliency detection by decomposing image features into a low-rank component associated with background and a sparse component associated with visual salient regions. Despite its great potential, existing LRMR-based saliency detection methods seldom consider the inter-relationship among elements within these two components, thus are prone to generating scattered or incomplete saliency maps. In this paper, we introduce a novel and efficient LRMR-based saliency detection model under a coarse-to-fine framework to circumvent this limitation. First, we roughly measure the saliency of image regions with a baseline LRMR model that integrates a ℓ1-norm sparsity constraint and a Laplacian regularization smooth term. Given samples from the coarse saliency map, we then learn a projection that maps image features to refined saliency values, to significantly sharpen the object boundaries and to preserve the object entirety. We evaluate our framework against existing LRMR-based methods on three benchmark datasets. Experimental results validate the superiority of our method as well as the effectiveness of our suggested coarse-to-fine framework, especially for images containing multiple objects. MATLAB code for implementing our method is available at https://github.com/qizhust/HLRSaliency.

DAA: Dual LSTMs with adaptive attention for image captioning

Image captioning enables people to better understand images through fine-grained analysis. Recently the encoder-decoder architecture with attention mechanism has achieved great achievements in image captioning and visual question answering. In this paper, we propose a new captioning algorithm that integrates two separate LSTM (Long-short Term Memory) networks through an adaptive semantic attention model. Within our approach, the first LSTM network is followed by an attention model, which serves as a visual sentinel can flexibly make a trade off between the visual semantic region and textual content. Another LSTM is used as a language model, which combines the hidden state representation of the first LSTM and attention context vector, then outputs the word sequence. The proposed model has been extensively evaluated on two large-scale datasets: MSCOCO and Flickr30k. Experimental results show that the proposed method pays more attention to visual salient regions and achieves significant performance of prior state-of-the-art approaches on multiple evaluation metrics.

Infrared and Visible Image Fusion Under Different Illumination Conditions Based on Illumination Effective Region Map

It is well known that the quality of visible images taken under different illumination conditions varies greatly, so the illumination factors will affect infrared and visible image fusion effects inevitably. This paper proposes an infrared and visible image fusion algorithm that satisfies poor illumination conditions. The algorithm is divided into three stages: image preprocessing, multiscale decomposition and image fusion. The purpose of image preprocessing is to improve the contrast of visible image and extract the visual salient regions of infrared image. In the multiscale analysis stage, infrared and visible images are decomposed into different scales by combining Gaussian transform and Rolling Guidance Filter, which effectively avoids the halo artifacts. In the image fusion stage, the fusion weights of base layer coefficients are determined based on the combination of the saliency map of infrared image and the Illumination Effective Region Map of visible image. The detail layer coefficients of images are fused by choose-max fusion rule based on the local variance of detail features of the original images. Experiments show that the fusion effects of the proposed algorithm are robust to illumination variations, and the fused images have good detail clarity and can preserve the effective information of the original images well in an unsatisfactory illumination condition.

Dim and small infrared target fast detection guided by visual saliency

In order to detect dim and small infrared targets from a mass of high-resolution images of omni-directional Infrared Search and Track (IRST) systems rapidly and accurately, a fast target detection method guided by visual saliency (TDGS) is proposed. In this method, a coarse-to-fine detection strategy is used. First, in the stage of coarse-detection, according to the differences of global features between targets and backgrounds, a global saliency model based on fast spectral scale space (FSSS) is constructed to suppress complex background regions rapidly. And visual salient regions which contain dim and small targets are extracted from the original image. Then, in the stage of fine-detection, according to differences of local contrast between targets and background, an adaptive local contrast method (ALCM) is applied to finely improve contrast of targets in visual salient regions. Candidate targets can be further extracted through the adaptive threshold segmentation. Finally, dim and small targets are detected by their temporal relativity in multi-frames. Experimental results on four typical image sequences have indicated that the proposed method can not only detect dim and small infrared targets with small amount of computation, high detection probability, and low false alarm rate, but also adapt to various complex backgrounds. It is suitable for dim and small targets detection in omni-directional IRST systems and other practical applications.

A New Method Based on Two-Stage Detection Mechanism for Detecting Ships in High-Resolution SAR Images

Ship detection in synthetic aperture radar (SAR) remote sensing images, being a fundamental but challenging problem in the field of satellite image analysis, plays an important role for a wide range of applications and is receiving significant attention in recent years. Aiming at the requirements of ship detection in high-resolution SAR images, the accuracy, the intelligent level, a better real-time operation and processing efficiency, The characteristics of ocean background and ship target in high-resolution SAR images were analyzed, we put forward a ship detection algorithm in high-resolution SAR images. The algorithm consists of two detection stages: The first step designs a pre-training classifier based on improved spectral residual visual model to obtain the visual salient regions containing ship targets quickly, then achieve the purpose of probably detection of ships. In the second stage, considering the Bayesian theory of binary hypothesis detection, a local maximum posterior probability (MAP) classifier is designed for the classification of pixels. After the parameter estimation and judgment criterion, the classification of pixels are carried out in the target areas to achieve the classification of two types of pixels in the salient regions. In the paper, several types of satellite image data, such as TerraSAR-X (TS-X), Radarsat-2, are used to evaluate the performance of detection methods. Comparing with classical CFAR detection algorithms, experimental results show that the algorithm can achieve a better effect of suppressing false alarms, which caused by the speckle noise and ocean clutter background inhomogeneity. At the same time, the detection speed is increased by 25% to 45%.

A Hybrid of Local and Global Saliencies for Detecting Image Salient Region and Appearance

This paper presents a visual saliency detection approach, which is a hybrid of local feature-based saliency and global feature-based saliency (simply called local saliency and global saliency, respectively, for short). First, we propose an automatic selection of smoothing parameter scheme to make the foreground and background of an input image more homogeneous. Then, we partition the smoothed image into a set of regions and compute the local saliency by measuring the color and texture dissimilarity in the smoothed regions and the original regions, respectively. Furthermore, we utilize the global color distribution model embedded with color coherence, together with the multiple edge saliency, to yield the global saliency. Finally, we combine the local and global saliencies, and utilize the composition information to obtain the final saliency. Experimental results show the efficacy of the proposed method, featuring: 1) the enhanced accuracy of detecting visual salient region and appearance in comparison with the existing counterparts, 2) the robustness against the noise and the low-resolution problem of images, and 3) its applicability to multisaliency detection task.

Visual enhancement based on salient region detection and layered difference representation

We propose a novel image enhancement method based on salient region detection and a layered difference representation of 2D histograms. We first obtain the visual salient region corresponding to maximal human attention using saliency filters. Then, we obtain a difference vector for the visual salient region by solving a constrained optimization problem of the layered difference representation at a specified layer. Finally, the new difference vector and the difference vector of the original image are aggregated to enhance the salient region and protect other regions from overstretching or brightness shift. Experimental results including comparisons with other methods show that our proposed algorithm produces more suitable enhanced images compared with the results of existing algorithms.

Inverse Bilateral Filter for Saliency

analysis and automatic detection of visual salient image regions has been the subject of considerable research useful in object segmentation, adaptive compression and re-targeting. However, the nature of the essential mechanisms intervening human visual saliency remains elusive. To assess the validity of salient regions some kind of prior model is consistently required. This paper proposes a new model using inverse bilateral fitter that allows the system to output saliency maps with salient objects in their context. The filter is described firstly for automatically learning local contrast distribution to accurately predict salient image regions. Along with the contrast distribution checking, local opposition is analyzed by the second application of the inverse bilateral filter to establish fuzzy boundary of salient regions in form of trimap. This approach is shown to increase the reliability of identifying visual salient objects. Output from the research has potential applications in the areas of object detection and recognition.

Building up a Bio-Inspired Visual Attention Model by Integrating Top-Down Shape Bias and Improved Mean Shift Adaptive Segmentation

The driver-assistance system (DAS) becomes quite necessary in-vehicle equipment nowadays due to the large number of road traffic accidents worldwide. An efficient DAS detecting hazardous situations robustly is key to reduce road accidents. The core of a DAS is to identify salient regions or regions of interest relevant to visual attended objects in real visual scenes for further process. In order to achieve this goal, we present a method to locate regions of interest automatically based on a novel adaptive mean shift segmentation algorithm to obtain saliency objects. In the proposed mean shift algorithm, we use adaptive Bayesian bandwidth to find the convergence of all data points by iterations and the k-nearest neighborhood queries. Experiments showed that the proposed algorithm is efficient, and yields better visual salient regions comparing with ground-truth benchmark. The proposed algorithm continuously outperformed other known visual saliency methods, generated higher precision and better recall rates, when challenged with natural scenes collected locally and one of the largest publicly available data sets. The proposed algorithm can also be extended naturally to detect moving vehicles in dynamic scenes once integrated with top-down shape biased cues, as demonstrated in our experiments.

Measuring Image Similarity Based on Shape Context

Measuring image similarity is important for a number of image processing applications. The goal of research in objective image similarity assessment is to develop quantitative measures that can automatically predict perceived image similarity. In this paper, we propose a new objective approach of measuring image similarity based on shape context. We take the geometric structures of objects into account during measuring the image similarity by virtue of shape context which is a robust and compact, yet highly discriminative descriptor. Firstly we find visual salient regions of images by virtue of a regional contrast based saliency extraction algorithm and employ shape context to describe the shape of visual salient region. Then we detect shape deformations of visual salient regions between two images through estimating shape context distances, and accordingly compute the image similarity values. Real data have been used to test the proposed approach and very good results have been achieved, validating it.

Exploiting local and global characteristics for contrast based visual saliency detection

Visual saliency is an important cue in human visual system to identify salient region in the image; it can be useful in many applications including image retrieval, object recognition, image segmentation, etc. Image contrast has been used as an effective feature to detect visual salient region. However, the conventional contrast measures either in spectral domain or in spatial domain fail to give sufficient consideration towards the local and global characteristics of the image. This paper presents a visual saliency detection algorithm based on a novel contrast measurement. This measurement extracts the spectral information of image block using the 2D discrete Fourier transform (DFT), and combines with the total variation (TV) of image block in spatial domain. The proposed algorithm is used to perform salient region detection in the image, and compared with state-of-the-art algorithms. The experimental results from the MSRA dataset validate the effectiveness of the proposed algorithm.

A New Similarity Measure with Deformation Detection of Visual Salient Regions for Image Retargeting

Measurement of image similarity is a fundamental issue in both image processing and computer vision and is very important for image retargeting applications, which have drawn much attention in recent years. In this paper, we propose a new similarity measure with deformation detection of visual salient regions for image retargeting. According to the fact that the human visual system is sensitive to content changes as well as visual effect variations, the proposed approach combines content similarity and visual effect similarity. Firstly, a content correspondence between original and retargeted images is established, which aims to assess their content similarity. Secondly, deformation detection of visual salient regions between two images is conducted to measure their visual effect similarity. Real images have been used to test the proposed method and very good results have been achieved, validating it.

Image Fusion Metric Based on Visual Saliency Weighted Structural Similarity

In this paper, a novel non-reference objective quality metric for night vision image fusion is presented. The metric extracts visual salient regions of the source images based on the mechanism of human visual attention, calculates the visual saliency weights with consideration of the physical characteristics of the input images, and evaluates the quality of the fused image with a visual saliency weighted structural similarity. Experimental results show that the proposed approach is consistent with human visual inspection better than recent state of the art image fusion evaluation methods, and can be used to assess the fusion schemes that are not performed at the same level.

Local Precise Image Deformation on Visual Salient Region

Local Precise Image Deformation on Visual Salient Region