Saliency Model Research Articles

Priorização de Stakeholders em Empresas Familiares: Um Ensaio Teórico

Objetivo do estudo: identificar os fatores que influenciam a priorização de stakeholders em empresas familiares.Metodologia/Abordagem: para a condução do estudo, foi desenvolvido um ensaio teórico, baseado no “modelo de saliência de stakeholders”, que aborda critérios de priorização, e no “modelo de três círculos”, que apresenta as interseções entre família, gestão e propriedade, peculiares em empresas familiares.Principais resultados: os resultados apontam a saliência de três grupos de stakeholders internos na empresa familiar (o fundador, os membros da próxima geração e os funcionários sem vínculo familiar), a partir dos atributos de poder, de legitimidade e de urgência, e considerando a simultaneidade dos papéis desempenhados.Contribuições teóricas: o estudo contribui para a gestão e a priorização de stakeholders, por considerar as semelhanças e as diferenças entre empresas familiares e não familiares; e por incluir o atributo “simultaneidade de papéis” como fator de influência importante nesse fenômeno.Relevância/Originalidade: a inclusão do atributo “simultaneidade de papéis” é relevante para o avanço da “teoria dos stakeholders”, no contexto de empresas familiares, pois oferece maior abrangência e precisão ao modelo de saliência; e o estudo fomenta a discussão sobre como alocar os recursos organizacionais diante das diversas demandas existentes.

Saliency and priority modulation in a pop-out paradigm: Pupil size and microsaccades

A salient stimulus can trigger a coordinated orienting response consisting of a saccade, pupil, and microsaccadic responses. Saliency models predict that the degree of visual conspicuity of all visual stimuli guides visual orienting. By presenting a multiple-item array that included an oddball colored item (pop-out), randomly mixed colored items (mixed-color), or single-color items (single-color), we examined the effects of saliency and priority (saliency + relevancy) on pupil size and microsaccade responses. Larger pupil responses were produced in the pop-out compared to the mixed-color or single-color conditions after stimulus presentation. However, the saliency modulation on microsaccades was not significant. Furthermore, although goal-relevancy information did not modulate pupil responses and microsaccade rate, microsaccade direction was biased toward the pop-out item when it was the subsequent saccadic target. Together, our results demonstrate saliency modulation on pupil size and priority effects on microsaccade direction during visual pop-out.

Ship Target Automatic Detection Based on Hypercomplex Flourier Transform Saliency Model in High Spatial Resolution Remote-Sensing Images.

Efficient ship detection is essential to the strategies of commerce and military. However, traditional ship detection methods have low detection efficiency and poor reliability due to uncertain conditions of the sea surface, such as the atmosphere, illumination, clouds and islands. Hence, in this study, a novel ship target automatic detection system based on a modified hypercomplex Flourier transform (MHFT) saliency model is proposed for spatial resolution of remote-sensing images. The method first utilizes visual saliency theory to effectively suppress sea surface interference. Then we use OTSU methods to extract regions of interest. After obtaining the candidate ship target regions, we get the candidate target using a method of ship target recognition based on ResNet framework. This method has better accuracy and better performance for the recognition of ship targets than other methods. The experimental results show that the proposed method not only accurately and effectively recognizes ship targets, but also is suitable for spatial resolution of remote-sensing images with complex backgrounds.

Mobile Interface Attentional Priority Model

Mobile devices are now the most popular computing technology for accessing the Internet. This has resulted in designers devoting more of their time to improving users’ mobile experiences. This can be achieved in part by helping users find content quickly and easily. Understanding what supports or harms a user’s visual search is key to creating displays that meet usability efficiency requirements. Broadly, searches are guided by a combination of visual salience and previous experiences. We recommend revealing these influences by employing a computational saliency model and by employing our mobile spatial convention map. The research presented extends the previous attentional priority (AP) modeling work, from web pages to mobile interfaces. Notably, we reveal that users typically search a mobile web page by using a “railroad-like” viewing pattern rather than the “F” pattern that is typically described in web page research. Also, we propose a mobile web page-specific attentional priority (AP) model. The AP model combines our experience-based spatial convention map with a saliency model map. We examined the predictive performance of a saliency model, compared to the mobile-specific convention map. It was discovered that the convention map better predicted the initial deployment of attention, and the saliency map better accounted for later selection.

Explore double-opponency and skin color for saliency detection

Recent advances in salient detection have exploited the foreground or background information to assist other saliency cues such as contrast to achieve state-of-the-art results. However the problem remains challenging. For example, human skin color is easily overlooked during saliency detection. Simulating the human visual mechanism to improve the current algorithm, we propose a saliency model based on the color-opponent mechanisms of a certain type of color-sensitive double-opponent (DO) cells in the primary visual cortex (V1) of human visual system. Firstly, DO cells with concentric receptive fields (RFs) can detect region contrast for yielding foreground saliency map. Our approach is intuitive and easy to interpret, and it allows fast implementation. Then, skin saliency map is built in a way that is different from high-level factors such as face detection, combining skin region and spatial Euclidean distance weight in the RGB space, and the significant skin features can be obtained effectively. Finally, a linear fusion strategy is proposed to integrate different saliency maps. Experimental results with three well-known benchmark databases demonstrate that the proposed method can achieve competitive performance when compared to state-of-the-art methods. Saliency regions contain important skin features compared to other traditional methods.

Viewport-Dependent Saliency Prediction in 360° Video

Saliency prediction in traditional images and videos has drawn extensive research interests in recent years. Few works have been proposed for saliency prediction over 360° videos. They focus on directly predicting fixations over the whole panorama. When viewing 360° videos, a person can only observe the content in her viewport, which means that only a fraction of the 360° scene can be seen at any given time. In this paper, we study human attention over viewport of 360° videos and propose a novel visual saliency model, dubbed viewport saliency, to predict fixations over 360° videos. Two contributions are introduced. First, we find that where people look is affected by the content and location of the viewport in 360° video. We study this over 200+ 360° videos viewed by 30+ subjects over two recent benchmark databases. Second, we propose a Multi-Task Deep Neural Network (MT-DNN) method for Viewport Saliency (VS) prediction in 360° video, which considers the input content and location of the viewport. Extensive experiments and analyses show that our method outperforms other state-of-the-art methods in this task. In particular, over the two recent 360° video databases, our MT-DNN raises the average CC score by 0.149 and 0.205, compared to SalGAN and DeepVS methods, respectively.

A brief survey of visual saliency detection

Salient object detection models mimic the behavior of human beings and capture the most salient region/object from the images or scenes, this field contains many important applications in both computer vision and pattern recognition tasks. Despite hundreds of models that have been proposed in this field, but still, it requires a large room for research. This paper demonstrates a detailed overview of the recent progress of saliency detection models in terms of heuristic-based techniques and deep learning-based techniques. we have discussed and reviewed its co-related fields, such as Eye-fixation-prediction, RGBD salient-object-detection, co-saliency object detection, and video-saliency-detection models. We have reviewed the key issues of the current saliency models and discussed future trends and recommendations. The broadly utilized datasets and assessment strategies are additionally investigated in this paper.

Towards Making Videos Accessible for Low Vision Screen Magnifier Users.

People with low vision who use screen magnifiers to interact with computing devices find it very challenging to interact with dynamically changing digital content such as videos, since they do not have the luxury of time to manually move, i.e., pan the magnifier lens to different regions of interest (ROIs) or zoom into these ROIs before the content changes across frames. In this paper, we present SViM, a first of its kind screen-magnifier interface for such users that leverages advances in computer vision, particularly video saliency models, to identify salient ROIs in videos. SViM's interface allows users to zoom in/out of any point of interest, switch between ROIs via mouse clicks and provides assistive panning with the added flexibility that lets the user explore other regions of the video besides the ROIs identified by SViM. Subjective and objective evaluation of a user study with 13 low vision screen magnifier users revealed that overall the participants had a better user experience with SViM over extant screen magnifiers, indicative of the former's promise and potential for making videos accessible to low vision screen magnifier users.

Latitude and binocular perception based blind stereoscopic omnidirectional image quality assessment for VR system

With the rapid development of 5G technologies, it is possible to perform high-speed transmission of virtual reality (VR) visual contents. As one of techniques related to VR visual contents, omnidirectional image/video quality assessment has been becoming one of important issues. In this paper, based on latitude characteristics of omnidirectional image and binocular perception of human visual system, we propose a new blind stereoscopic omnidirectional image quality assessment (BSOIQA) method. Firstly, with the latitude characteristics of omnidirectional image, an improved omnidirectional image saliency model is established to estimate the visual behavior of users. Secondly, to describe the perceptual characteristics of the left and right views of stereoscopic omnidirectional image (SOI), Weber local descriptor is used to obtain differential excitation image and orientation image. The differential excitation image is combined with saliency map to obtain the statistical features of quantized orientation image, and multi-scale statistical features are extracted from the left and right views of SOI. Thirdly, tensor decomposition is used to fuse the left and right views of SOI to extract binocular perception features. Dual-tree dual-density complex wavelet transform, image information entropy and singular value decomposition are used to extract binocular fusion features, and the latitude characteristics of SOI are taken into account in image blocking. The gradient map of the absolute difference image of the left and right views of SOI is calculated, and its histogram statistical features are extracted to represent binocular rivalry. In addition, the statistical characteristics of the binocular product image are used to represent the local structure information of binocular perception. Finally, the feature vector of SOI is created with feature screening and used to predict the objective quality of SOI, and an SOI subjective assessment dataset (NBU-SOID) is also established to evaluate the proposed BSOIQA method. Experimental results on NBU-SOID and other databases show that the proposed BSOIQA method is in better consistence with human visual perception, compared with the other state-of-the-art methods.

CMP-based saliency model for stereoscopic omnidirectional images

Although many saliency models for images had been proposed, the saliency model for stereoscopic omnidirectional image (SOI) is still a very important but not yet deeply studied topic. In addition, one of main disadvantages of the existing saliency models is difficult to simulate the characteristics of a wide field of view (FOV) of SOIs. To solve the problems, this paper proposes a novel cube map projection (CMP) based saliency model for SOIs. Considering that the equi-rectangular projection (ERP) representation of SOI will result in obvious stretch distortion, and what the human eyes view are the viewport images by using head mounted display (HMD), we construct the saliency model of the SOI in its less shape distorted CMP plane. First, the SOI with ERP format is converted into the CMP format. In order to reduce the boundary effect between the faces in CMP plane, its four horizontal faces are moved to obtain the horizontal global face and the horizontal local faces for which the corresponding saliency maps are subsequently computed by combining color distance and depth distance in the graph model. Then, to establish the correlations of these faces, the horizontal face saliency map is calculated by weighting the horizontal global face saliency map to the horizontal local face saliency map. Meanwhile, the vertical face saliency map is defined and obtained. Finally, the horizontal face saliency map and vertical face saliency map are spliced and re-projected to ERP plane to obtain the final saliency map of the ERP-mapped SOI. The experiments on the public ODI database are performed to compare the proposed methods with the state-of-the-art methods, and the results show that the proposed method achieves better performance of estimating SOI saliency map in terms of six well-known quantitative metrics.

Exploiting Surroundedness and Superpixel cues for salient region detection

In this paper, we will present a new salient region detection method by exploiting its surrounding and superpixel cues. Its main highlights are: 1) An input image is quantized to 256 colors by using minimum variance quantization; 2) Saliency maps is computed based on the figure-ground segregation of the quantized image; 3) Mean saliency value of each superpixel is employed to refine saliency maps further. This can highlight salient objects robustly and suppress backgrounds evenly. Experimental results show that the proposed method produces more accurate saliency maps and performs well against twenty-one saliency models concerning three evaluation metrics on two public datasets.

Robust image hashing with visual attention model and invariant moments

Image hashing is an efficient technique of multimedia processing for many applications, such as image copy detection, image authentication, and social event detection. In this study, the authors propose a novel image hashing with visual attention model and invariant moments. An important contribution is the weighted DWT (discrete wavelet transform) representation by incorporating a visual attention model called Itti saliency model into LL sub-band. Since the Itti saliency model can efficiently extract saliency map reflecting regions of attention focus, perceptual robustness of the proposed hashing is achieved. In addition, as invariant moments are robust and discriminative features, hash construction with invariant moments extracted from the weighted DWT representation ensures good classification performance between robustness and discrimination. Extensive experiments with open image datasets are done to validate the performances of the proposed hashing. The results demonstrate that the proposed hashing is robust and discriminative. Performance comparisons with some hashing algorithms are also conducted, and the receiver operating characteristic results illustrate that the proposed hashing outperforms the compared hashing algorithms in classification performance between robustness and discrimination.

Big Decisions in Small Business Ecosystems: Stakeholder Priority

In an increasingly competitive environment, engaging with stakeholders is no longer an option but a necessity to ensure short-term and long-term success. This is particularly true in small business, yet little is known about how small business managers make those decisions, often under uncertainty and time pressure. Small businesses operate in networks of interdependent entities (ecosystems) where individuals are known to each other. This supports building of relationships, trust and reputation and removes the separation between 'business' and 'ethics' by promoting a long-term focus. Yet, limited resources and conflicting demands result in a need to prioritise some stakeholder demands. As suggested by Stakeholder Salience, decision makers may prioritise stakeholder claims based on perceived levels of power, legitimacy and urgency. The key decision makers of nine Australian small businesses were asked about their relationships with important stakeholders, how decisions regarding the priority of various stakeholders and their expectations were made and what factors may affect such decisions. Relationships with stakeholders were identified as dynamic and strategic resources, interviewees invested considerable time and effort to build and maintain positive relationships. Consistent with Stakeholder Salience model, decisions were affected by perceptions of stakeholders' level of power, legitimacy and urgency. This research also found that intuitive perceptions about the stakeholder affected the decision-making process, and identified the following additional factors: commitment, dependence and potential exit costs as well as the decision maker's perception of alignment with own values, thereby introducing a moral and ethical consideration that would sometimes take priority over other considerations.

Detecting visually salient scene areas and deriving their relative spatial relations from continuous street-view panoramas

ABSTRACT A salient scene is an area within an image that contains visual elements that stand out from surrounding areas. They are important for distinguishing landmarks in first-person-view (FPV) applications and determining spatial relations in images. The relative spatial relation between salient scenes acts as a visual guide that is easily accepted and understood by users in FPV applications. However, current digitally navigable maps and location-based services fall short of providing information on visual spatial relations for users. This shortcoming has a critical influence on the popularity and innovation of FPV applications. This paper addresses the issue by proposing a method for detecting visually salient scene areas (SSAs) and deriving their relative spatial relationships from continuous panoramas. This method includes three critical steps. First, an SSA detection approach is introduced by fusing region-based saliency derived from super-pixel segmentation and the frequency-tuned saliency model. The method focuses on a segmented landmark area in a panorama. Secondly, a street-view-oriented SSA generation method is introduced by matching and merging the visual SSAs from continuous panoramas. Thirdly, a continuous geotagged panorama-based referencing approach is introduced to derive the relative spatial relationships of SSAs from continuous panoramas. This information includes the relative azimuth, elevation angle, and the relative distance. Experiment results show that the error for the SSA relative azimuth angle is approximately ± 6° (with an average error of 2.67°), and the SSA relative elevation angle is approximately ± 4° (with an average error of 1.32°) when using Baidu street-view panoramas. These results demonstrate the feasibility of the proposed approach. The method proposed in this study can facilitate the development of FPV applications such as augmented reality (AR) and pedestrian navigation using proper spatial relation.

Rule-based aggregation driven by similar images for visual saliency detection

The visual saliency detection consists in determining the relevant visual information in a scene to segment it from the background. This paper proposes visual saliency detection in a rule-based approach using the image similarity cue to improve saliency detection performance. Our system induces rules for saliency detection, and for a given input image, determines the subset of rules to be used from a set of candidate rules. The proposed approach consists of two main stages: training and testing. Firstly, during the training stage, our system learns an ensemble of rough-set-based rules by combining knowledge extracted from outputs of four state-of-the-art saliency models. Secondly, our system determines the most suitable subset of induced rules for binary detection of pixels of a salient object in an image. The decision of the best subset of rules is based on the image similarity cue. The binary determination of saliency in the output image, exempts us from performing a post-processing stage as is needed in most saliency approaches. The proposed method is evaluated quantitatively on three challenging databases designed for the saliency detection task. The results obtained from the performed experiments indicate that the proposed method outperforms the state-of-the-art approaches used for comparison.

Understanding Eye Movements: Psychophysics and a Model of Primary Visual Cortex

Humans move their eyes in order to learn visual representations of the world. These eye movements depend on distinct factors, either by the scene that we perceive or by our own decisions. To select what is relevant to attend is part of our survival mechanisms and the way we build reality, as we constantly react both consciously and unconsciously to all the stimuli that is projected into our eyes. In this thesis we try to explain (1) how we move our eyes, (2) how to build machines that understand visual information and deploy eye movements, and (3) how to make these machines understand tasks in order to decide for eye movements.(1) We provided the analysis of eye movement behavior elicited by low-level feature distinctiveness with a dataset of 230 synthetically-generated image patterns. A total of 15 types of stimuli has been generated (e.g. orientation, brightness, color, size, etc.), with 7 feature contrasts for each feature category. Eye-tracking data was collected from 34 participants during the viewing of the dataset, using Free-Viewing and Visual Search task instructions. Results showed that saliency is predominantly and distinctively influenced by: 1. feature type, 2. feature contrast, 3. temporality of fixations, 4. task difficulty and 5. center bias. From such dataset (SID4VAM), we have computed a benchmark of saliency models by testing performance using psychophysical patterns. Model performance has been evaluated considering model inspiration and consistency with human psychophysics. Our study reveals that state-of-the-art Deep Learning saliency models do not perform well with synthetic pattern images, instead, models with Spectral/Fourier inspiration outperform others in saliency metrics and are more consistent with human psychophysical experimentation.(2) Computations in the primary visual cortex (area V1 or striate cortex) have long been hypothesized to be responsible, among several visual processing mechanisms, of bottom-up visual attention (also named saliency). In order to validate this hypothesis, images from eye tracking datasets have been processed with a biologically plausible model of V1 (named Neurodynamic Saliency Wavelet Model or NSWAM). Following Li's neurodynamic model, we define V1's lateral connections with a network of firing rate neurons, sensitive to visual features such as brightness, color, orientation and scale. Early subcortical processes (i.e. retinal and thalamic) are functionally simulated. The resulting saliency maps are generated from the model output, representing the neuronal activity of V1 projections towards brain areas involved in eye movement control. We want to pinpoint that our unified computational architecture is able to reproduce several visual processes (i.e. brightness, chromatic induction and visual discomfort) without applying any type of training or optimization and keeping the same parametrization. The model has been extended (NSWAM-CM) with an implementation of the cortical magnification function to define the retinotopical projections towards V1, processing neuronal activity for each distinct view during scene observation. Novel computational definitions of top-down inhibition (in terms of inhibition of return and selection mechanisms), are also proposed to predict attention in Free-Viewing and Visual Search conditions. Results show that our model outpeforms other biologically-inpired models of saliency prediction as well as to predict visual saccade sequences, specifically for nature and synthetic images. We also show how temporal and spatial characteristics of inhibition of return can improve prediction of saccades, as well as how distinct search strategies (in terms of feature-selective or category-specific inhibition) predict attention at distinct image contexts.(3) Although previous scanpath models have been able to efficiently predict saccades during Free-Viewing, it is well known that stimulus and task instructions can strongly affect eye movement patterns. In particular, task priming has been shown to be crucial to the deployment of eye movements, involving interactions between brain areas related to goal-directed behavior, working and long-term memory in combination with stimulus-driven eye movement neuronal correlates. In our latest study we proposed an extension of the Selective Tuning Attentive Reference Fixation Controller Model based on task demands (STAR-FCT), describing novel computational definitions of Long-Term Memory, Visual Task Executive and Task Working Memory. With these modules we are able to use textual instructions in order to guide the model to attend to specific categories of objects and/or places in the scene. We have designed our memory model by processing a visual hierarchy of low- and high-level features. The relationship between the executive task instructions and the memory representations has been specified using a tree of semantic similarities between the learned features and the object category labels. Results reveal that by using this model, the resulting object localization maps and predicted saccades have a higher probability to fall inside the salient regions depending on the distinct task instructions compared to saliency.

Research on Generating an Indoor Landmark Salience Model for Self-Location and Spatial Orientation from Eye-Tracking Data

Landmarks play an essential role in wayfinding and are closely related to cognitive processes. Eye-tracking data contain massive amounts of information that can be applied to discover the cognitive behaviors during wayfinding; however, little attention has been paid to applying such data to calculating landmark salience models. This study proposes a method for constructing an indoor landmark salience model based on eye-tracking data. First, eye-tracking data are taken to calculate landmark salience for self-location and spatial orientation tasks through partial least squares regression (PLSR). Then, indoor landmark salience attractiveness (visual, semantic and structural) is selected and trained by landmark salience based on the eye-tracking data. Lastly, the indoor landmark salience model is generated by landmark salience attractiveness. Recruiting 32 participants, we designed a laboratory eye-tracking experiment to construct and test the model. Finding 1 proves that our eye-tracking data-based modelling method is more accurate than current weighting methods. Finding 2 shows that significant differences in landmark salience occur between two tasks; thus, it is necessary to generate a landmark salience model for different tasks. Our results can contribute to providing indoor maps for different tasks.

Attention-guided RGBD saliency detection using appearance information

Most of the deep convolutional neural networks (CNNs) based RGBD saliency models either regard the RGB and depth cues as the same status or trust the depth information excessively. However, they ignore that the low-quality depth map is an interference factor and the multi-level deep features that originated from RGB images contain abundant appearance information. Therefore, we propose a novel RGBD saliency model, where the attention-guided bottom-up and top-down modules are powerfully combined by using multi-level deep RGB features, to utilize the deep RGB and depth features in a sufficient and appropriate way. Concretely, a two-stream structure based bottom-up module is first constructed to dig and fuse the RGB and depth information, yielding the fused deep feature. Besides, the module embeds the depth cue based attention maps to guide the indication of salient objects. Then, considering the abundant appearance information, a top-down module is deployed to perform coarse-to-fine saliency inference, where the fused deep feature is progressively integrated with appearance information. Similarly, the attention map is also inserted into this module for locating salient objects. Extensive experiments are performed on five public RGBD datasets and the corresponding experimental results firmly demonstrate the effectiveness and superiority of our model when compared with the state-of-the-art RGBD saliency models.

A novel image recognition approach using multiscale saliency model and GoogLeNet

It is very good to apply the saliency model in the visual selective attention mechanism to the preprocessing process of image recognition. However, the mechanism of visual perception is still unclear, so this visual saliency model is not ideal. To this end, this paper proposes a novel image recognition approach using multiscale saliency model and GoogLeNet. First, a multi-scale convolutional neural network was taken advantage of constructing multiscale salient maps, which could be used as filters. Second, an original image was combined with the salient maps to generate the filtered image, which highlighted the salient regions and suppressed the background in the image. Third, the image recognition task was implemented by adopting the classical GoogLeNet model. In this paper, many experiments were completed by comparing four commonly used evaluation indicators on the standard image database MSRA10K. The experimental results show that the recognition results of the test images based on the proposed method are superior to some stateof- the-art image recognition methods, and are also more approximate to the results of human eye observation.

Saliency detection using adversarial learning networks

This paper proposes a novel model for saliency detection using the adversarial learning networks, in which the generator is used to generate the saliency map and the discriminator is deployed to guide the training process of overall network. Concretely, the training procedure of our model consists of three steps including the training of generator, the training of discriminator, and the training throughout the overall network. The key point of training process lies in the discriminator, which is designed to provide the feedback information for the acceleration of the generator and the refinement of saliency map. Therefore, during the training stage of overall network, the output of the generator, i.e. the coarse saliency map, is fed into the discriminator, yielding the corresponding feedback information. Following this way, we can obtain the final generator with a higher performance. For testing, the obtained generator is employed to perform saliency detection. Extensive experiments on four challenging saliency detection datasets show that our model not only achieves the favorable performance against the state-of-the-art saliency models, but also possesses the faster convergence speed when training the proposed model.