Saliency-based Visual Attention Model Research Articles

Depth-aware salient object segmentation

Depth-aware salient object segmentation

Research on Region Selection Strategy for Visible Watermark Embedding

ABSTRACT Visible watermarking is one of the most direct and effective copyright protection methods for digital images and videos, but it faces the problems of easy detection of watermarking location and watermarking removal attacks. It requires that the visible watermarking not only can not affect visual quality of original digital products but also can resist accidental editing and malicious attacks. In this paper, an adaptive region selection method for visible watermark embedding is proposed. First, an improved region selection method based on the model of saliency-based visual attention (MSVA) is used to more accurately distinguish salient and non-salient regions of host image; then, the non-salient regions of image are divided into sub-blocks to calculate texture complexity respectively; finally, a moderate image block is selected as the embedding region of visible watermark image according to the distribution of image texture complexity. The experimental results show that the proposed method can identify saliency regions more accurately, thus ensuring that the important contents of host image will not be occluded. The adaptive selection of embedding region based on image complexity not only ensures the recognizability and aesthetics of visible watermark embedding effect but also takes into account the security of watermark embedding, and especially reduces the risk of batch removal attacks.

Saliency-based gaze prediction based on head direction

Despite decades of attempts to create a model for predicting gaze locations by using saliency maps, a highly accurate gaze prediction model for general conditions has yet to be devised. In this study, we propose a gaze prediction method based on head direction that can improve the accuracy of any model. We used a probability distribution of eye position based on head direction (static eye–head coordination) and added this information to a model of saliency-based visual attention. Using empirical data on eye and head directions while observers were viewing natural scenes, we estimated a probability distribution of eye position. We then combined the relationship between eye position and head direction with visual saliency to predict gaze locations. The model showed that information on head direction improved the prediction accuracy. Further, there was no difference in the gaze prediction accuracy between the two models using information on head direction with and without eye–head coordination. Therefore, information on head direction is useful for predicting gaze location when it is available. Furthermore, this gaze prediction model can be applied relatively easily to many daily situations such as during walking.

Autonomous Detection of Weld Seam Profiles via a Model of Saliency-Based Visual Attention for Robotic Arc Welding

This paper presents a method of autonomously detecting weld seam profiles from molten pool background in metal active gas (MAG) arc welding using a novel model of saliency-based visual attention. First, a vision sensor based on structured light is employed to capture laser stripes and molten pools simultaneously in the same frame. Second, to effectively detect the weld seam profile from molten pool background for next autonomous guidance of initial welding positions and seam tracking, a model of visual attention based on saliency is proposed. With respect to the enhanced effect of saliency, the proposed model is much better than the classic models in the field. According to the comprehensive saliency map created by the proposed model, the weld seam profile can be extracted after threshold segmentation and clustering are applied to it in turn. Third, different weld seam images are used to demonstrate the robustness of the proposed methodology and last, to evaluate the performance of the proposed method, a measure called profile extraction rate (PER) is computed, which shows that the extracted weld seam profile can basically meet the requirements of seam tracking and the guidance of welding torches.

Recognition of suspicious behavior using case-based reasoning

A novel method case-based reasoning was proposed for suspicious behavior recognition. The method is composed of three departs: human behavior decomposition, human behavior case representation and case-based reasoning. The new approach was proposed to decompose behavior into sub-behaviors that are easier to recognize using a saliency-based visual attention model. New representation of behavior was introduced, in which the sub-behavior and the associated time characteristic of sub-behavior were used to represent behavior case. In the process of case-based reasoning, apart from considering the similarity of basic sub-behaviors, order factor was proposed to measure the similarity of a time order among the sub-behaviors and span factor was used to measure the similarity of duration time of each sub-behavior, which makes the similarity calculations more rational and comprehensive. Experimental results show the effectiveness of the proposed method in comparison with other related works and can run in real-time for the recognition of suspicious behaviors.

Biological edge detection for UCAV via improved artificial bee colony and visual attention

Purpose – The purpose of this paper is to propose a biological edge detection approach for aircraft such as unmanned combat air vehicle (UCAV), with the objective of making the UCAV recognize targets, especially in complex noisy environment. Design/methodology/approach – The hybrid model of saliency-based visual attention and artificial bee colony (ABC) algorithm is established for edge detection of UCAV. Visual attention can extract the region of interesting objects, and this approach can narrow the searching region for object segmentation, which can reduce the computational complexity. An improved ABC algorithm is applied in edge detection of the salient region. Findings – This work improved ABC algorithm by modifying the search strategy and adding some limits, so that it can be applied to edge detection problem. A hybrid model of saliency-based visual attention and ABC algorithm is developed. Experimental results demonstrated the feasibility and effectiveness of the proposed method: it can guarantee ef...

A New Approach of Feature Combination for Object Detection in Saliency-based Visual Attention

This paper presents a fuzzy approach of feature maps combination in saliency-based visual attention model proposed by Itti. This strategy applies fuzzy rules to combine three conspicuity maps instead of linear combination in the basic model of visual attention that does not seem reasonable biologically. In this method, in addition to bottom-up features, top-down cues are also considered in the model. As fuzzy rules are designed using target mask information, top-down characteristics of the target are considered helping the model to make the target more conspicuous in the final saliency map. This can be applied in further processing such as object detection and recognition application. The experimental results show the effectiveness of our new fuzzy approach in finding the target in the first hit. A database of emergency triangle in natural environment background is used in this paper to show the results. Moreover, the comparison of this fuzzy combination approach with some other combination methods also proved the priority of the approach over other combination strategies.

Color coding in the cortex: a modified approach to bottom-up visual attention

Itti and Koch's (Vision Research 40:1489-1506, 2000) saliency-based visual attention model is a broadly accepted model that describes how attention processes are deployed in the visual cortex in a pure bottom-up strategy. This work complements their model by modifying the color feature calculation. Evidence suggests that S-cone responses are elicited in the same spatial distribution and have the same sign as responses to M-cone stimuli; these cells are tentatively referred to as red-cyan. For other cells, the S-cone input seems to be aligned with the L-cone input; these cells might be green-magenta cells. To model red-cyan and green-magenta double-opponent cells, we implement a center-surround difference approach of the aforementioned model. The resulting color maps elicited enhanced responses to color salient stimuli when compared to the classic ones at high statistical significance levels. We also show that the modified model improves the prediction of locations attended by human viewers.

Predicting visual fixations on video based on low-level visual features

To what extent can a computational model of the bottom–up visual attention predict what an observer is looking at? What is the contribution of the low-level visual features in the attention deployment? To answer these questions, a new spatio-temporal computational model is proposed. This model incorporates several visual features; therefore, a fusion algorithm is required to combine the different saliency maps (achromatic, chromatic and temporal). To quantitatively assess the model performances, eye movements were recorded while naive observers viewed natural dynamic scenes. Four completing metrics have been used. In addition, predictions from the proposed model are compared to the predictions from a state of the art model [Itti’s model (Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254–1259)] and from three non-biologically plausible models (uniform, flicker and centered models). Regardless of the metric used, the proposed model shows significant improvement over the selected benchmarking models (except the centered model). Conclusions are drawn regarding both the influence of low-level visual features over time and the central bias in an eye tracking experiment.

Bottom-up spatiotemporal visual attention model for video analysis

The human visual system (HVS) has the ability to fixate quickly on the most informative (salient) regions of a scene and therefore reducing the inherent visual uncertainty. Computational visual attention (VA) schemes have been proposed to account for this important characteristic of the HVS. A video analysis framework based on a spatiotemporal VA model is presented. A novel scheme has been proposed for generating saliency in video sequences by taking into account both the spatial extent and dynamic evolution of regions. To achieve this goal, a common, image-oriented computational model of saliency-based visual attention is extended to handle spatiotemporal analysis of video in a volumetric framework. The main claim is that attention acts as an efficient preprocessing step to obtain a compact representation of the visual content in the form of salient events/objects. The model has been implemented, and qualitative as well as quantitative examples illustrating its performance are shown.

Assessing the contribution of color in visual attention

Visual attention is the ability of a vision system, be it biological or artificial, to rapidly detect potentially relevant parts of a visual scene, on which higher level vision tasks, such as object recognition, can focus. The saliency-based model of visual attention represents one of the main attempts to simulate this visual mechanism on computers. Though biologically inspired, this model has only been partially assessed in comparison with human behavior. Our methodology consists in comparing the computational saliency map with human eye movement patterns. This paper presents an in-depth analysis of the model by assessing the contribution of different cues to visual attention. It reports the results of a quantitative comparison of human visual attention derived from fixation patterns with visual attention as modeled by different versions of the computer model. More specifically, a one-cue gray-level model is compared to a two-cues color model. The experiments conducted with over 40 images of different nature and involving 20 human subjects assess the quantitative contribution of chromatic features in visual attention.

Empirical Validation of the Saliency-based Model of Visual Attention

Visual attention is the ability of the human vision system to detect salient parts of the scene, on which higher vision tasks, such as recognition, can focus. In human vision, it is believed that visual attention is intimately linked to the eye movements and that the fixation points correspond to the location of the salient scene parts. In computer vision, the paradigm of visual attention has been widely investigated and a saliency-based model of visual attention is now available that is commonly accepted and used in the field, despite the fact that its biological grounding has not been fully assessed. This work proposes a new method for quantitatively assessing the plausibility of this model by comparing its performance with human behavior. The basic idea is to compare the map of attention - the saliency map - produced by the computational model with a fixation density map derived from eye movement experiments. This human attention map can be constructed as an integral of single impulses located at the positions of the successive fixation points. The resulting map has the same format as the computer-generated map, and can easily be compared by qualitative and quantitative methods. Some illustrative examples using a set of natural and synthetic color images show the potential of the validation method to assess the plausibility of the attention model.

Real-time visual attention on a massively parallel SIMD architecture

Visual attention is the ability to rapidly detect the visually salient parts of a given scene on which higher level vision tasks, such as object recognition, can focus. Found in biological vision, this mechanism represents a fundamental tool for computer vision. This paper reports the first real-time implementation of the complete visual attention mechanism on a compact and low-power architecture. Specifically, the saliency-based model of visual attention was implemented on a highly parallel single instruction, multiple data architecture called ProtoEye. Conceived for general purpose low-level image processing, ProtoEye consists of a 2D array of mixed analog–digital processing elements. To reach real-time, the operations required for visual attention computation were optimally distributed on the analog and digital parts. The currently available prototype runs at a frequency of 14 images/ s and operates on 64×64 gray level images. Extensive testing and run-time analysis of the system stress the strengths of the architecture.

A model of saliency-based visual attention for rapid scene analysis

A visual attention system, inspired by the behavior and the neuronal architecture of the early primate visual system, is presented. Multiscale image features are combined into a single topographical saliency map. A dynamical neural network then selects attended locations in order of decreasing saliency. The system breaks down the complex problem of scene understanding by rapidly selecting, in a computationally efficient manner, conspicuous locations to be analyzed in detail.