Saliency Model Research Articles

Saliency Model Extension for Sensorless Initial Position and Polarity Detection of Permanent Magnet Synchronous Motors

In this paper, we propose an extended permanent magnet synchronous motor model that incorporates a quadratic flux-current function to determine the polarity-dependent saliency and develop a sensorless initial position detection method that is capable of polarity detection. The quadratic model extension is based on the second-order Taylor polynomial of the flux-current function.We present the model equations in both the stator- and the rotor-oriented reference frames as well as the Park and inverse Park transforms of the quadratic terms and Hessian matrices. The transient behavior and the dependency of the extended model on the rotor position have been validated. The proposed initial position detection method injects non-modulated even square wave voltage signals and relies only on phase-current measurements. We analyzed the effect of the pulse length on the errors of the position estimation and polarity detection. We provide machine parameter-based approximations of the current components dependent on the rotor position and, based on them, present a quantitative pulse-length design method.

Dynamic Selective Network for RGB-D Salient Object Detection.

RGB-D saliency detection is receiving more and more attention in recent years. There are many efforts have been devoted to this area, where most of them try to integrate the multi-modal information, i.e. RGB images and depth maps, via various fusion strategies. However, some of them ignore the inherent difference between the two modalities, which leads to the performance degradation when handling some challenging scenes. Therefore, in this paper, we propose a novel RGB-D saliency model, namely Dynamic Selective Network (DSNet), to perform salient object detection (SOD) in RGB-D images by taking full advantage of the complementarity between the two modalities. Specifically, we first deploy a cross-modal global context module (CGCM) to acquire the high-level semantic information, which can be used to roughly locate salient objects. Then, we design a dynamic selective module (DSM) to dynamically mine the cross-modal complementary information between RGB images and depth maps, and to further optimize the multi-level and multi-scale information by executing the gated and pooling based selection, respectively. Moreover, we conduct the boundary refinement to obtain high-quality saliency maps with clear boundary details. Extensive experiments on eight public RGB-D datasets show that the proposed DSNet achieves a competitive and excellent performance against the current 17 state-of-the-art RGB-D SOD models.

Selective focus saliency model driven by object class‐awareness

Current many salient object detection (SOD) models only focus on highlighting visual conspicuous region but fail to make saliency detection for specific targets. In this paper, a selective focus saliency model driven by object class-awareness (SF-OCA) to run saliency detection is proposed. The framework consists of a visual saliency detection flow, a segmentation-classification flow, and a class-awareness selection module. It combines bottom-up visual perception with a top-down task-driven manner, which is capable of detecting specific category salient targets and eliminating the interference from other saliency areas, providing a new idea for saliency detection. Experimental results show that the method achieves comparable performance with state-of-the-art models on four public saliency datasets. In addition, a new dataset was also built to test the proposed framework for the selective focus saliency detection. Compared with other SOD methods, the method not only highlights visual saliency regions but can choose more important or more noteworthy targets in a class-awareness manner. The method also shows better robustness under a variety of conditions including multi-targets, small targets and complex background.

Audiovisual saliency prediction via deep learning

Neuroscience study verifies that synchronized audiovisual stimuli would make a stronger response of visual perception than an independent stimulus. Many researches show that audio signals would affect human gaze behavior in the viewing of natural video scenes. Thus in this paper, we propose a multi-sensory framework of audio and visual signals for video saliency prediction. It mainly includes four modules: auditory feature extraction, visual feature extraction, semantic interaction between auditory feature and visual feature, and feature fusion. With the inputs of audio and visual signals, we present a network architecture of deep learning to undertake the tasks of these four modules. It is an end-to-end architecture that could interact the semantics from its learned features of audio and visual stimuli. The numerical and visual results show our method achieves a significant improvement over eleven recent saliency models that are regardless of the audio stimuli, even some of them are state-of-the-art deep learning models.

Revisiting Video Saliency Prediction in the Deep Learning Era.

Predicting where people look in static scenes, a.k.a visual saliency, has received significant research interest recently. However, relatively less effort has been spent in understanding and modeling visual attention over dynamic scenes. This work makes three contributions to video saliency research. First, we introduce a new benchmark, called DHF1K (Dynamic Human Fixation 1K), for predicting fixations during dynamic scene free-viewing, which is a long-time need in this field. DHF1K consists of 1K high-quality elaborately-selected video sequences annotated by 17 observers using an eye tracker device. The videos span a wide range of scenes, motions, object types and backgrounds. Second, we propose a novel video saliency model, called ACLNet (Attentive CNN-LSTM Network), that augments the CNN-LSTM architecture with a supervised attention mechanism to enable fast end-to-end saliency learning. The attention mechanism explicitly encodes static saliency information, thus allowing LSTM to focus on learning a more flexible temporal saliency representation across successive frames. Such a design fully leverages existing large-scale static fixation datasets, avoids overfitting, and significantly improves training efficiency and testing performance. Third, we perform an extensive evaluation of the state-of-the-art saliency models on three datasets : DHF1K, Hollywood-2, and UCF sports. An attribute-based analysis of previous saliency models and cross-dataset generalization are also presented. Experimental results over more than 1.2K testing videos containing 400K frames demonstrate that ACLNet outperforms other contenders and has a fast processing speed (40 fps using a single GPU). Our code and all the results are available at https://github.com/wenguanwang/DHF1K.

Validity and measurement invariance of the Addictions Neuroclinical Assessment incentive salience domain among treatment-seekers with alcohol use disorder

Incentive salience, or the attribution of motivational value to stimuli, is a biopsychological process that is disrupted in alcohol use disorder (AUD). The Addictions Neuroclinical Assessment (ANA) is a framework to characterize heterogeneity in addiction and establish a common assessment battery for research and clinical use. The ANA framework hypothesizes three constructs that correspond to processes in the etiology, course, and treatment of addiction: incentive salience, negative emotionality, and executive function. The current study extends prior findings on the ANA by validating the incentive salience construct among participants (n = 563) in a multisite prospective study of individuals entering treatment for AUD. We used confirmatory factor analysis to test a one-factor model of incentive salience. Indicators included items assessing perception of urges to drink from the Alcohol Dependence Scale, Impaired Control Scale, and Marlatt Relapse Interview. Results indicated the one-factor model fit the data well (χ2 (12) = 19.42, p = .08; RMSEA = 0.034 [90% CI: 0.000, 0.060], CFI = 0.992) and was measurement invariant across sex. Incentive salience was associated with drinking patterns (e.g., drinks per day, r = 0.447 [95% CI: 0.379, 0.514]); reasons for drinking (urges/temptation r = 0.529 [95% CI: 0.460, 0.599]); testing personal control, r = 0.384 (95% CI: 0.308, 0.461); social pressure, r = 0.549 (95% CI: 0.481, 0.617); and family history of AUD, r = 0.134. The incentive salience factor demonstrated greater predictive validity for drinking outcomes compared to alternative preexisting scales. Overall, this study provides support for the construct validity and measurement invariance of the ANA incentive salience construct in a sample of individuals seeking AUD treatment.

Integrating saliency with fuzzy thresholding for brain tumor extraction in MR images

The automatic detection and extraction of tumor area in Magnetic Resonance Imaging (MRI) is an important and challenging task. This paper presents a fully automatic and unsupervised method for fast and accurate extraction of brain tumor area from MR images. The proposed method named as Saliency Based Segmentation (SBS) is based on visual saliency. The saliency model detects the pathologically important area and then fuzzy thresholding is used for extraction of the detected region. The performance of SBS is compared with Adaptively Regularized Kernel-Based Fuzzy C-Means Clustering, Mean Shift and Fuzzy C-Means clustering with Level Set Method. The experimental evaluation validated on BRATS database using Jaccard index (0.84 ± 0.04), Dice Index (0.91 ± 0.02), Execution time (2.99 ± 0.29), Precision (0.82 ± 0.16), Recall (0.97 ± 0.03) and F-measure (0.88 ± 0.10) demonstrates that SBS achieves better segmentation results even in the presence of noise and uneven illumination in images.

A Spatial-Temporal Recurrent Neural Network for Video Saliency Prediction.

In this paper, a recurrent neural network is designed for video saliency prediction considering spatial-temporal features. In our work, video frames are routed through the static network for spatial features and the dynamic network for temporal features. For the spatial-temporal feature integration, a novel select and re-weight fusion model is proposed which can learn and adjust the fusion weights based on the spatial and temporal features in different scenes automatically. Finally, an attention-aware convolutional long short term memory (ConvLSTM) network is developed to predict salient regions based on the features extracted from consecutive frames and generate the ultimate saliency map for each video frame. The proposed method is compared with state-of-the-art saliency models on five public video saliency benchmark datasets. The experimental results demonstrate that our model can achieve advanced performance on video saliency prediction.

Contrast-weighted dictionary learning based saliency detection for VHR optical remote sensing images

Object detection in very high resolution (VHR) optical remote sensing (RS) images is one of the most fundamental but challenging tasks in the field of RS image analysis. To reduce the computational complexity of redundant information and improve the efficiency of image processing, visual saliency models have been widely applied in this field. In this paper, a novel saliency detection model based on Contrast-weighted Dictionary Learning (CDL) is proposed for VHR optical RS images. Specifically, the proposed CDL learns salient and non-salient atoms from positive and negative samples to construct a discriminant dictionary, in which a contrast-weighted term is proposed to encourage the contrast-weighted patterns to be present in the learned salient dictionary while discouraging them from being present in the non-salient dictionary. Then, we measure the saliency by combining the coefficients of the sparse representation (SR) and reconstruction errors. Furthermore, by using the proposed joint saliency measure, a variety of saliency maps are generated based on the discriminant dictionary. Finally, a fusion method based on global gradient optimization is proposed to integrate multiple saliency maps. Experimental results on four datasets demonstrate that the proposed model outperforms other state-of-the-art methods.

Attention-SLAM: A Visual Monocular SLAM Learning From Human Gaze

This paper proposes a novel simultaneous localization and mapping (SLAM) approach, namely Attention-SLAM, which simulates human navigation mode by combining a visual saliency model (SalNavNet) with traditional monocular visual SLAM. Firstly a visual saliency model namely SalNavNet is proposed. In SalNavNet, we introduce a correlation module and propose an adaptive Exponential Moving Average (EMA) module. These modules mitigate the center bias, which most current saliency models have. This novel idea enables the saliency maps generated by SalNavNet to pay more attention to the same salient object. An open-source saliency SLAM dataset namely Salient-Euroc is published, it consists of Euroc dataset and corresponding saliency maps. Moreover, we propose a new optimization method called Weighted Bundle Adjustment (Weighted BA) in Attention-SLAM. Most SLAM methods treat all the features extracted from the images as equal importance during the optimization process. In weighted BA, the feature points extracted from the salient regions have greater importance. Comprehensive test results prove that Attention-SLAM outperforms benchmarks such as Direct Sparse Odometry (DSO), ORB-SLAM, and Salient DSO in the 7 of 11 test cases. The test cases are all indoor scenes, with varying brightness, speed, and image distortion. Compared with ORB-SLAM, our method improves the accuracy by 4% and efficiency by 6.5% on average.

Saliency-driven rate-distortion optimization for 360-degree image coding

360-degree images allow an immersive experience. They offer multiple views of a scene and the viewpoint can be selected by the user. However, the huge amount of data that is necessary for real-time transmission of 360-degree image and video requires efficient coding techniques, particularly for virtual reality (VR) and augmented reality (AR) applications. The viewer is only interested in a part of the scene so compressing the entire scene with equal quality is inefficient. This study initially constructs a saliency model of the 360-degree image and then a visual attention guided coding scheme is developed using a predicted saliency map. For saliency prediction, two methods of saliency prediction are used and the results are fused, to address the problem of geometry distortion in the ERP (Equirectangular Projection) format. A smoothing-based optimization is then realized in the spherical domain to improve the saliency map. Using the saliency map of the 360-degree image, the distortion of the rate-distortion optimization is modified to ensure a better visual experience. The experimental results show that the viewports of greatest interest are rendered with the highest quality and there is a maximum of 14.33% reduction in the bitrate when the quality measurement is performed in these regions.

Functional connectivity in a triple-network saliency model is associated with real-life self-control

Despite its significance for health and education, the neurocognitive mechanism of real-life self-control remains unclear. While recent studies focused on task-related brain activation patterns as predictors of self-control, the contribution and relevance of functional connectivity between large-scale brain networks mediating higher-order cognition is largely unknown. Using a saliency-based triple-network model of cognitive control, we tested the hypothesis that cross-network interactions among the salience network (SN), the central executive network (CEN), and the default mode network (DMN) are associated with real-life self-control. To this end, a large community sample (N = 294) underwent ecological momentary assessment of daily self-control as well as task-free fMRI to examine intrinsic inter-network organization and determine a SN-centered network interaction index (NII). Logistic multilevel regression analysis showed that higher NII scores were associated with increased real-life self-control. This suggests that the assumed role of the SN in initiating switching between the DMN and CEN is an important part of self-control.

Development of a bionic interactive interface for Owl robot using stereo vision algorithms

With the requirements for improving life quality, companion robots have gradually become hotspot of application for healthy home living. In this article, a novel bionic human‐robot interaction (HRI) strategy using stereo vision algorithms has been developed to imitate the animal vision system on the Owl robot. Depth information of a target is found via two methods, vergence and disparity. Vergence requires physical tracking of the target, moving each camera to align with a chosen object, and through successive camera movements (saccades) a sparse depth map of the scene can be built up. Disparity however requires the cameras to be fixed and parallel, using the position of the target within the field of view, of a stereo pair of cameras, to calculate distance. As disparity does not require the cameras to move, multiple targets can be chosen to build up a disparity map, providing depth information for the whole scene. In addition, a salience model is implemented imitating how people explore a scene. This is achieved with feature maps, which apply filtering to the scene to highlight areas of interest, for example color and edges, which is purely a bottom‐up approach based on Itti and Koch's saliency model. A series of experiments have been conducted on Plymouth Owl robot to validate the proposed interface.

Meaning maps and saliency models based on deep convolutional neural networks are insensitive to image meaning when predicting human fixations

Eye movements are vital for human vision, and it is therefore important to understand how observers decide where to look. Meaning maps (MMs), a technique to capture the distribution of semantic information across an image, have recently been proposed to support the hypothesis that meaning rather than image features guides human gaze. MMs have the potential to be an important tool far beyond eye-movements research. Here, we examine central assumptions underlying MMs. First, we compared the performance of MMs in predicting fixations to saliency models, showing that DeepGaze II – a deep neural network trained to predict fixations based on high-level features rather than meaning – outperforms MMs. Second, we show that whereas human observers respond to changes in meaning induced by manipulating object-context relationships, MMs and DeepGaze II do not. Together, these findings challenge central assumptions underlying the use of MMs to measure the distribution of meaning in images.

How much time do you have? Introducing a multi-duration saliency model

How much time do you have? Introducing a multi-duration saliency model

Salient Object Detection Techniques in Computer Vision-A Survey.

Detection and localization of regions of images that attract immediate human visual attention is currently an intensive area of research in computer vision. The capability of automatic identification and segmentation of such salient image regions has immediate consequences for applications in the field of computer vision, computer graphics, and multimedia. A large number of salient object detection (SOD) methods have been devised to effectively mimic the capability of the human visual system to detect the salient regions in images. These methods can be broadly categorized into two categories based on their feature engineering mechanism: conventional or deep learning-based. In this survey, most of the influential advances in image-based SOD from both conventional as well as deep learning-based categories have been reviewed in detail. Relevant saliency modeling trends with key issues, core techniques, and the scope for future research work have been discussed in the context of difficulties often faced in salient object detection. Results are presented for various challenging cases for some large-scale public datasets. Different metrics considered for assessment of the performance of state-of-the-art salient object detection models are also covered. Some future directions for SOD are presented towards end.

A heuristic framework for perceptual saliency prediction

Saliency prediction can be regarded as the human spontaneous activity. The most effective saliency model should highly approximate the response of viewers to the perceived information. In the paper, we exploit the perception response for saliency detection and propose a heuristic framework to predict salient region. First, to find the perceptually meaningful salient regions, an orientation selectivity based local feature and a visual Acuity based global feature are proposed to jointly predict candidate salient regions. Subsequently, to further boost the accuracy of saliency map, we introduce a visual error sensitivity based operator to activate the meaningful salient regions from a local and global perspective. In addition, an adaptive fusion method based on free energy principle is designed to combine the sub-saliency maps from each image channel to obtain the final saliency map. Experimental results on five natural and emotional datasets demonstrate the superiority of the proposed method compared to twelve state-of-the-art algorithms.

How stakeholders\u2019 data literacy contributes to student success in higher education: a goal-oriented analysis

Student success is becoming a shared vision for quality in higher education. Majority data in higher education have not been transformed into actionable insights for quality enhancement. Data are dispersed among stakeholders, and stakeholders’ data literacy influences the effectiveness of using data for student success. However, existing studies mainly focus on students’ data literacy; the analysis of other stakeholders’ data literacy for student success is still few. This study aims to explore how stakeholders’ data literacy contributes to student success in a holistic view. The salience model is used to identify core stakeholders. The goal-modeling language iStar is used to present how stakeholders contribute to student success. A competencies matrix of data literacy is used to discuss the specific data literacy competencies that stakeholders should focus on promoting student success. A survey is conducted to validate the goal-oriented analysis and the discussions on specific competencies of data literacy for stakeholders. The goal-oriented analysis presents the complexity of interactions and dependencies among stakeholders for student success. This study helps to raise stakeholders to be aware of the importance of their data literacy and the necessity of collaboration on exploiting vast available data for student success.

Salient Ship Detection via Background Prior and Foreground Constraint in Remote Sensing Images

Automatic ship detection in complicated maritime background is a challenging task in the field of optical remote sensing image interpretation and analysis. In this paper, we propose a novel and reliable ship detection framework based on a visual saliency model, which can efficiently detect multiple targets of different scales in complex scenes with sea clutter, clouds, wake and islands interferences. Firstly, we present a reliable background prior extraction method adaptive for the random locations of targets by computing boundary probability and then generate a saliency map based on the background prior. Secondly, we compute the prior probability of salient foreground regions and propose a weighting function to constrain false foreground clutter, gaining the foreground-based prediction map. Thirdly, we integrate the two prediction maps and improve the details of the integrated map by a guided filter function and a wake adjustment function, obtaining the fine selection of candidate regions. Afterwards, a classification is further performed to reduce false alarms and produce the final ship detection results. Qualitative and quantitative evaluations on two public available datasets demonstrate the robustness and efficiency of the proposed method against four advanced baseline methods.

Multi-Visual Feature Saliency Detection for Sea-Surface Targets through Improved Sea-Sky-Line Detection

To visually detect sea-surface targets, the objects of interest must be effectively and rapidly isolated from the background of sea-surface images. In contrast to traditional image detection methods, which employ a single visual feature, this paper proposes a significance detection algorithm based on the fusion of multi-visual features after detecting the sea-sky-lines. The gradient edges of the sea-surface images are enhanced using a Gaussian low-pass filter to eliminate the effect of the image gradients pertaining to the clouds, wave points, and illumination. The potential region and points of the sea-sky-line are identified. The sea-sky-line is fitted through polynomial iterations to obtain a sea-surface image containing the target object. The saliency subgraphs of the high and low frequency, gradient texture, luminance, and color antagonism features are fused to obtain an integrated saliency map of the sea-surface image. The saliency target area of the sea surface is segmented. The effectiveness of the proposed method was verified. The average detection rate and time for the sea-sky-line detection were 96.3% and 1.05 fps, respectively. The proposed method outperformed the existing saliency models on the marine obstacle detection dataset and Singapore maritime dataset, with mean absolute errors of 0.075 and 0.051, respectively.