Visual Saliency Map Research Articles

Orientation-Aware Saliency Guided JND Model for Robust Image Watermarking

The visual saliency-based just noticeable distortion (VS-JND) model has been widely used in the quantization-based watermarking framework, but it remains a challenge that how to reflect the characteristics of the human visual system (HVS) to the greatest extent in the processing of detecting salient objects. In this paper, given that the HVS has a different perception sensitivity in the different orientation of the image, a compound orientation feature map, which contains the vertical, horizontal, and diagonal information for directional patch extraction, is computed to improve the visual saliency map within just noticeable distortion (JND) profile. First, the DC and three low-frequency coefficients of each block are used to calculate the luminance and texture feature map, respectively. Then, the feature map based on the complex directional feature includes the horizontal, vertical, and diagonal information contained in the image is calculated by three low-frequency coefficients. Finally, we will be able to make a linear fusion of the same three different visual features to get the final VS map, and the JND model is improved according to the new VS model. Later, the new proposed VS-JND model is employed in the quantization watermarking (Q-watermarking) framework. The simulation results show that the proposed watermarking scheme is effective and produces superior robustness compared with existing monochrome image watermarking schemes.

Optimization Method of IR Thermography Facial Image Registration

So far, there have been many types of researches subject to technical requirements due to image registration. Using image registration can lower deviation from sequential images and make it possible to analyze the information variation of the particular area subsequently. This study provides a procedure creating fixed image based on the data of facial IR thermography, where its methods include the visual saliency map by detected image, as well as cluster algorithm. Comparison is also made here to solve the merits and demerits by affine parameter to reach the optimum measure among Genetic Algorithm, Particle Swarm Optimization, and Simulated Annealing Algorithm, where there are two control parameters concerning this experiment: one is the calculation of time confined each alignment, while the other one is to use parallel computing toolbox or not. The optimum method will be chosen by the values of the objective function based on the control parameters. Afterward, the optimal internal parameter is to be verified through the Taguchi experiment and the validity of this procedure in this study will be built following the parameter result as above. Therefore, the difference of images before and after alignment can be validated by overlapping the images before and after alignment as well as the image quality measurements, where its results reveal that the alignment procedure of IR thermography in this study is capable of performing human face alignment precisely, and subsequently, do help data statistics and analysis concerning temperature area interdependence.

Tree DBH measurement method based on smartphone and machine vision technology.

Aiming at the problems of high labor intensity and low efficiency, complicated operation and high cost of advanced equipment in the measurement of DBH in existing traditional forest resources, and proposing a method for measuring diameter at breast height (DBH) based on smartphone and machine vision technology, and combining with camera calibration, 3D reconstruction, machine vision, close-range photogrammetry, and other technologies. First, The original image is obtained by smartphone, and the visual saliency map of the preprocessed image is processing by using the lab color space model and 3×3 operator to convolving with image. Combining with the H component of the HSV (Hue, Saturation, Value) color model to enhance the color contrast part of the tree trunk in the image. The image segmentation algorithm is used to identify and obtain the target tree's contour in a natural environment. Then, an improved camera calibration model with nonlinear distortion terms is used to calibrate the camera's internal and external parameters. Finally, achieving reconstruction of a 3D world with camera parameters and two-dimensional image information is carried out to measure DBH at a height of 1.3 m. Experimental results indicated that within a certain range, the relative error of the DBH measurements was lower than 2.50%. This method has a high measurement accuracy, meeting the accuracy requirement of DBH measurement in forest resources investigation and could be applied in forestry resource surveys.

A Novel Ship-Tracking Method for GF-4 Satellite Sequential Images.

The geostationary remote sensing satellite has the capability of wide scanning, persistent observation and operational response, and has tremendous potential for maritime target surveillance. The GF-4 satellite is the first geostationary orbit (GEO) optical remote sensing satellite with medium resolution in China. In this paper, a novel ship-tracking method in GF-4 satellite sequential imagery is proposed. The algorithm has three stages. First, a local visual saliency map based on local peak signal-to-noise ratio (PSNR) is used to detect ships in a single frame of GF-4 satellite sequential images. Second, the accuracy positioning of each potential target is realized by a dynamic correction using the rational polynomial coefficients (RPCs) and automatic identification system (AIS) data of ships. Finally, an improved multiple hypotheses tracking (MHT) algorithm with amplitude information is used to track ships by further removing the false targets, and to estimate ships’ motion parameters. The algorithm has been tested using GF-4 sequential images and AIS data. The results of the experiment demonstrate that the algorithm achieves good tracking performance in GF-4 satellite sequential images and estimates the motion information of ships accurately.

Multi-Focus Fusion Technique on Low-Cost Camera Images for Canola Phenotyping.

To meet the high demand for supporting and accelerating progress in the breeding of novel traits, plant scientists and breeders have to measure a large number of plants and their characteristics accurately. Imaging methodologies are being deployed to acquire data for quantitative studies of complex traits. Images are not always good quality, in particular, they are obtained from the field. Image fusion techniques can be helpful for plant breeders with more comfortable access plant characteristics by improving the definition and resolution of color images. In this work, the multi-focus images were loaded and then the similarity of visual saliency, gradient, and color distortion were measured to obtain weight maps. The maps were refined by a modified guided filter before the images were reconstructed. Canola images were obtained by a custom built mobile platform for field phenotyping and were used for testing in public databases. The proposed method was also tested against the five common image fusion methods in terms of quality and speed. Experimental results show good re-constructed images subjectively and objectively performed by the proposed technique. The findings contribute to a new multi-focus image fusion that exhibits a competitive performance and outperforms some other state-of-the-art methods based on the visual saliency maps and gradient domain fast guided filter. The proposed fusing technique can be extended to other fields, such as remote sensing and medical image fusion applications.

Infrared and visible images fusion using visual saliency and optimized spiking cortical model in non-subsampled shearlet transform domain

Aiming at some problems in existing infrared and visible image fusion methods such as edge blurring, low contrast, loss of details, a novel fusion scheme based on non-subsampled shearlet transform (NSST), visual saliency and multi-objective artificial bee colony (MOABC) optimizing spiking cortical mode (SCM) is proposed. NSST has many advantages such as multi-scale features and sparse representation. Moreover, the visual saliency map can improve the low frequency fusion strategy, and SCM has coupling and pulse synchronization properties. Firstly, NSST is utilized to decompose the source image into a low-frequency subband and a series of high-frequency subbands. Secondly, the low-frequency subband is fused by SCM, where SCM is motivated by the edge saliency map of the low-frequency subband of the source image, and then the high-frequency subbands are also fused by SCM, where the modified spatial frequency of the high-frequency subbands of the source image is adopted as the input stimulus of SCM, the parameters of SCM are optimized by the novel multi-objective artificial bee colony technique. Finally, the fused image is reconstructed by inverse NSST. Experimental results indicate that the proposed scheme performs well and has obvious superiorities over other current typical ones in both subjective visual performance and objective criteria.

The visual attention saliency map for movie retrospection

Abstract The visual saliency map is becoming important and challenging for many scientific disciplines (robotic systems, psychophysics, cognitive neuroscience and computer science). Map created by the model indicates possible salient regions by taking into consideration face presence and motion which is essential in motion pictures. By combining we can obtain credible saliency map with a low computational cost.

Real‐time segmentation of various insulators using generative adversarial networks

The conventional inspection of fragile insulators is critical to grid operation and insulator segmentation is the basis of inspection. However, the segmentation of various insulators is still difficult because of the great differences in colour and shape, as well as the cluttered background. Traditional insulator segmentation algorithms need many artificial thresholds, thereby limiting the adaptability of algorithms. A compact end‐to‐end neural network, which is trained in the framework of conditional generative adversarial networks, is proposed for the real‐time pixel‐level segmentation of insulators. The input image is mapped to a visual saliency map, and various insulators with different poses are filtered out at the same time. The proposed two‐stage training and empty samples are also used to improve the segmentation quality. Extensive experiments and comparisons are performed on many real‐world images. The experimental results demonstrate superior segmentation and real‐time performance. Meanwhile, the effectiveness of the proposed training strategies and the trade‐off between performance and speed are analysed in detail.

Visual saliency estimation using constraints

Abstract In this paper, we propose visual saliency estimation using constraints. Based on the observations that salient regions are generally distinctive from the background, we define visual saliency as the possibility of being assigned to the label of the most salient region. First, we generate an initial saliency map for a given image at the superpixel level using superpixel segmentation and three common priors. Then, we select salient and non-salient seeds from the initial saliency map to generate adaptive constraints. Adaptive constraints are able to propagate the seed information adaptively by their correlations. Finally, we produce the visual saliency map by propagating saliency seeds to the whole image using a learned non-linear kernel mapping. Experimental results demonstrate that kernel learning and seed propagation are capable of learning distinctive saliency from images.

A framework for fast automatic image cropping based on deep saliency map detection and gaussian filter

ABSTRACTExtracting robust visual saliency map and image cropping are fundamental problems in computer vision, graphics, and so on. It is not easy task to accurately detect and crop the entire salient object from images with complex background. In this paper, a deep learning strategy is adopted to train a large data-set of images, to get saliency map from the input image using graph-based segmentation and gray level adjustment to enhance and extract more accurate and clear saliency map. Furthermore, the Gaussian filter and image scaling used along with cropping method to keep better presentation of the visual object. The important task of overall framework is to take care about relevant image contents as well as to identify more region of interest and get optimum rectangle from the saliency map with minimum and maximum rectangular windows. Quality and low computational complexity have been focused while performing the cropping operation because automatic and efficient cropping technique should not only rely on geometric constraints, but it should be fast enough to consider the important image contents. The applied method use different data-set of images, to ensure the efficiency of this technique and the experimental results show that the framework is not only fast as well as much better for image cropping. We used Matlab and Caffe framework for efficient experimental results.

Toward a Cluttered Environment for Learning-Based Multi-Scale Overhead Ground Wire Recognition

In this paper, we propose a learning-based real-time method to recognize and segment an overhead ground wire (OGW) from an image, which is mainly applied to the multi-scale features in a cluttered environment. The recognition and segmentation are implemented under the framework of conditional generative adversarial nets. The generator is an end-to-end convolutional neural network (CNN) with skip connection. The discriminator is a multi-stage CNN and learns the loss function to train the generator. The OGW is recognized and shown in the downsampling visual saliency map. Thus, the location and existence of OGW are verified, which is crucial for the detection in the cluttered environment with structural lines. Detailed experiments and comparisons are performed on real-world images to demonstrate that the proposed method significantly outperforms the traditional method. Additionally, the optimized network achieves approximately 200 fps on a graphics card (GTX970) and 30 fps on an embedded platform (Jetson TX1).

Effects Selection Tool with Visual Saliency Map

ポスタやパッケージなどに使用する画像の制作過程において，特定オブジェクトの誘目性を向上させる場面が数多く存在する．本論文では，このような場面において適切な画像加工を行うエフェクト列を自動選択するツールを提案する．

Quaternion representation based visual saliency for stereoscopic image quality assessment

In this paper, a novel visual saliency detection method for stereoscopic images is proposed for the stereoscopic image quality assessment (SIQA) by considering the disparity map and difference image between the stereo image pairs. Firstly, a new quaternion representation (QR) of each stereo image (left/right view image) is constructed, which comprises the image content, the inter-view disparity, and the difference map. The quaternion Fourier transform (QFT) is performed on the constructed QR to generate the visual saliency maps for left and right views of stereoscopic image pairs, respectively. The generated visual saliency maps are further incorporated into the quality metrics for SIQA. Experimental results demonstrate that the visual saliency maps generated by the proposed method can help significantly boost the performance of SIQA, comparing with other visual saliency models proposed for stereoscopic images. It further confirms that the proposed visual saliency model can accurate depict the acuity property of human visual system (HVS) in judging the perceptual quality of stereoscopic images.

RETRACTED ARTICLE: Multimedia and multi-feature cluster fusion model based on saliency for mobile network applications

This paper introduces the concept, advantages, structure, method and application of multisensor integration and data fusion, and lists four different integration characteristics of sensors. Data fusion technology combines data from different sensors or other information sources in order to improve the accuracy of location and feature estimation. In the process of data fusion, modeling includes signal model, noise model, converter model, data transformation model and fusion model. The data fusion model includes the fusion method and the structure. This paper introduces the integrated, distributed and hybrid fusion structures, and compares them. The visual saliency map to image processing technology depends on the quality of the obtained good results, the existing visual saliency detection method is usually only detected by visual saliency map attribute rough, seriously affected the image processing results. Therefore, a visual saliency detection method based on Bayesian theory and statistical learning is proposed to detect the visual saliency of the image. The method is based on Bayesian theory of the significance of static images. According to the bottom-up visual saliency model, the ROC curve was used for quantitative evaluation in the two standard data sets. The results show that the nonlinear combination effect is better than the linear combination.

Perceptually Driven Nonuniform Asymmetric Coding of Stereoscopic 3D Video

Asymmetric stereoscopic video coding has already proven its effectiveness in reducing the bandwidth required for stereoscopic 3D delivery without degrading the visual quality. This approach, in which the left and right views are encoded with different levels of quality, relies on the perceptual theory of binocular suppression. However, to ensure comfortable 3D viewing, the just-noticeable level of asymmetry, i.e., the maximum quality gap between views, has to be carefully defined. Both subjectively and empirically fixed thresholds of asymmetry demonstrated either the maladjustment to content or dependency to the experimental design. This paper describes a new nonuniform asymmetric stereoscopic video coding method adaptively adjusting the level of asymmetry for each region of the image based on its perceptual significance. The proposed method uses a fully automated model that dynamically determines the best bounds of asymmetry for which the 3D viewing experience will not be altered. This is achieved by exploiting several human-visual-system-inspired models, namely, the binocular just-noticeable difference, and the visual saliency map and depth information. The simulation results show that the proposed method results in bit rate saving of up to 26% and provides better 3D visual quality compared with state-of-the-art asymmetric coding methods.

Visual Attention Saccadic Models Learn to Emulate Gaze Patterns From Childhood to Adulthood.

How people look at visual information reveals fundamental information about themselves, their interests and their state of mind. While previous visual attention models output static 2D saliency maps, saccadic models aim to predict not only where observers look at but also how they move their eyes to explore the scene. In this paper, we demonstrate that saccadic models are a flexible framework that can be tailored to emulate observer's viewing tendencies. More specifically, we use fixation data from 101 observers split into five age groups (adults, 8-10 y.o., 6-8 y.o., 4-6 y.o., and 2 y.o.) to train our saccadic model for different stages of the development of human visual system. We show that the joint distribution of saccade amplitude and orientation is a visual signature specific to each age group, and can be used to generate age-dependent scan paths. Our age-dependent saccadic model does not only output human-like, age-specific visual scan paths, but also significantly outperforms other state-of-the-art saliency models. We demonstrate that the computational modeling of visual attention, through the use of saccadic model, can be efficiently adapted to emulate the gaze behavior of a specific group of observers.

A Method of Ship Detection under Complex Background

The detection of ships in optical remote sensing images with clouds, waves, and other complex interferences is a challenging task with broad applications. Two main obstacles for ship target detection are how to extract candidates in a complex background, and how to confirm targets in the event that targets are similar to false alarms. In this paper, we propose an algorithm based on extended wavelet transform and phase saliency map (PSMEWT) to solve these issues. First, multi-spectral data fusion was utilized to separate the sea and land areas, and the morphological method was used to remove isolated holes. Second, extended wavelet transform (EWT) and phase saliency map were combined to solve the problem of extracting regions of interest (ROIs) from a complex background. The sea area was passed through the low-pass and high-pass filter to obtain three transformed coefficients, and the adjacent high frequency sub-bands were multiplied for the final result of the EWT. The visual phase saliency map of the product was built, and locations of ROIs were obtained by dynamic threshold segmentation. Contours of the ROIs were extracted by texture segmentation. Morphological, geometric, and 10-dimensional texture features of ROIs were extracted for target confirmation. Support vector machine (SVM) was used to judge whether targets were true. Experiments showed that our algorithm was insensitive to complex sea interferences and very robust compared with other state-of-the-art methods, and the recall rate of our algorithm was better than 90%.

Comparative study of computational visual attention models on two-dimensional medical images.

Computational modeling of visual attention is an active area of research. These models have been successfully employed in applications such as robotics. However, most computational models of visual attention are developed in the context of natural scenes, and their role with medical images is not well investigated. As radiologists interpret a large number of clinical images in a limited time, an efficient strategy to deploy their visual attention is necessary. Visual saliency maps, highlighting image regions that differ dramatically from their surroundings, are expected to be predictive of where radiologists fixate their gaze. We compared 16 state-of-art saliency models over three medical imaging modalities. The estimated saliency maps were evaluated against radiologists' eye movements. The results show that the models achieved competitive accuracy using three metrics, but the rank order of the models varied significantly across the three modalities. Moreover, the model ranks on the medical images were all considerably different from the model ranks on the benchmark MIT300 dataset of natural images. Thus, modality-specific tuning of saliency models is necessary to make them valuable for applications in fields such as medical image compression and radiology education.

Fast content-aware resizing of multi-layer information visualization via adaptive triangulation

Visual graphics and image-based content have become the pervasive modes of interaction with the digital information flow. With the immense proliferation of display systems and devices, visual content representation has become increasingly challenging. Classical static image resizing algorithms are not directly suitable for the current dynamic information visualization of streaming data flows and processes because most of the visual content often consists of superimposed, multi-layered, multi-scale structure. In this paper, we propose a new adaptive method for content-aware resizing of visual information flow. Scaling is performed by deforming a hierarchical triangle mesh that matches the visual saliency map (VSM) of the streaming data. The VSM is generated automatically based on a series of predefined rules operating on a triangular mesh representation of visual features. We present a linear energy function to minimize distortions of the triangular deformations to perceptually preserve informative content. Through multiple experiments on real datasets, we show that the method has both high performance as well as high robustness in the presence of large differences in the visual aspect ratios between target displays.

The Top-down Visual Attention Method with Known Interference Target

The simulation of visual attention can improve the efficiency of relevant intelligent applications by finding out most salient locations quickly in images or videos while ignoring redundant information. The paper can reason key features of target based on task scenario, filter the bottom-up visual attention feature channel by top-down method, and propose the top-down visual attention method with known interference target. The proposed method can minimize the time of generating visual saliency map by ensuring the hit rate of mission targets, and improve the efficiency of the search targets based on visual attention.