Definition Of Saliency Research Articles

Reward prediction-errors weighted by cue salience produces addictive behaviours in simulations, with asymmetrical learning and steeper delay discounting

Dysfunction in learning and motivational systems are thought to contribute to addictive behaviours. Previous models have suggested that dopaminergic roles in learning and motivation could produce addictive behaviours through pharmacological manipulations that provide excess dopaminergic signalling towards these learning and motivational systems. Redish (2004) suggested a role based on dopaminergic signals of value prediction error, while (Zhang et al., 2009) suggested a role based on dopaminergic signals of motivation. However, both models present significant limitations. They do not explain the reduced sensitivity to drug-related costs/negative consequences, the increased impulsivity generally found in people with a substance use disorder, craving behaviours, and non-pharmacological dependence, all of which are key hallmarks of addictive behaviours. Here, we propose a novel mathematical definition of salience, that combines aspects of dopamine’s role in both learning and motivation within the reinforcement learning framework. Using a single parameter regime, we simulated addictive behaviours that the (Zhang et al., 2009; Redish, 2004) models also produce but we went further in simulating the downweighting of drug-related negative prediction-errors, steeper delay discounting of drug rewards, craving behaviours and aspects of behavioural/non-pharmacological addictions. The current salience model builds on our recently proposed conceptual theory that salience modulates internal representation updating and may contribute to addictive behaviours by producing misaligned internal representations (Kalhan et al., 2021). Critically, our current mathematical model of salience argues that the seemingly disparate learning and motivational aspects of dopaminergic functioning may interact through a salience mechanism that modulates internal representation updating.

Saliency map in image visual quality assessment and processing

Images are mainly viewed and analyzed by humans. Because of this, in the characterization of image quality and effectiveness of image processing, it is necessary to take into account the peculiarities of the human vision system and cognition that are very complex. Saliency maps as well as priority and meaning maps introduced recently are the attempts to incorporate specific features of human vision into image analysis and processing fields. Many authors that consider the aforementioned maps consider them from different viewpoints. Thus, the basic subject of this paper is the factors that influence and determine these maps. Among such factors, there are low-level features as well as social and psychological ones such as emotions, age, and life values. The main goal of this paper is to give a brief survey of these factors and to consider how maps are already used in image quality assessment and processing as well as how they can be employed in the future. The tasks of the paper are to provide a definition of saliency, priority, and meaning maps, to analyze the factors that influence these maps, and to evaluate what improvement can be obtained due to taking maps into account in the assessment of image visual quality and such image processing operations as quality assessment, denoising, and lossy compression. The main result is that, by taking saliency maps into account, image quality assessment and processing efficiency can be sufficiently improved, especially for applications oriented on image viewing and analysis by observers or customers. This can be done by the simple weighting of local estimates of a given metric with further aggregation as well as by approaches based on neural networks. Using different quantitative criteria, we show what positive results can be got due to incorporating maps into quality assessment and image processing. As conclusion, we present possible directions of future research that are mainly related to an adaptation of denoising and lossy compression parameters to peculiarities of human attention.

Early Visual Saliency Based on Isolated Optimal Features.

Under fast viewing conditions, the visual system extracts salient and simplified representations of complex visual scenes. Saccadic eye movements optimize such visual analysis through the dynamic sampling of the most informative and salient regions in the scene. However, a general definition of saliency, as well as its role for natural active vision, is still a matter for discussion. Following the general idea that visual saliency may be based on the amount of local information, a recent constrained maximum-entropy model of early vision, applied to natural images, extracts a set of local optimal information-carriers, as candidate salient features. These optimal features proved to be more informative than others in fast vision, when embedded in simplified sketches of natural images. In the present study, for the first time, these features were presented in isolation, to investigate whether they can be visually more salient than other non-optimal features, even in the absence of any meaningful global arrangement (contour, line, etc.). In four psychophysics experiments, fast discriminability of a compound of optimal features (target) in comparison with a similar compound of non-optimal features (distractor) was measured as a function of their number and contrast. Results showed that the saliency predictions from the constrained maximum-entropy model are well verified in the data, even when the optimal features are presented in smaller numbers or at lower contrast. In the eye movements experiment, the target and the distractor compounds were presented in the periphery at different angles. Participants were asked to perform a simple choice-saccade task. Results showed that saccades can select informative optimal features spatially interleaved with non-optimal features even at the shortest latencies. Saccades’ choice accuracy and landing position precision improved with SNR. In conclusion, the optimal features predicted by the reference model, turn out to be more salient than others, despite the lack of any clues coming from a global meaningful structure, suggesting that they get preferential treatment during fast image analysis. Also, peripheral fast visual processing of these informative local features is able to guide gaze orientation. We speculate that active vision is efficiently adapted to maximize information in natural visual scenes.

Compositorial ‘Weight’ & ‘Luminance’

Compositorial weight might be understood as an operational definition of salience. It is not a psychophysical entity, but holds a key position between psychophysics and aesthetics. Several factors ranging over raw photometric/colorimetric parameters, various kinds of psychophysical contrast, image geometry, even semantic properties are readily shown to influence weight. A down-to-earth proposition is that luminance might play a dominant role. We investigate this notion and show that luminance per se is hardly important, except in certain paradigms like the ones considered here. We find that observers indeed readily judge weight based on luminance in such paradigms, although there are strong idiosyncratic differences. Our results have some generic implications for graphical design.

Scene analysis for effective visual search in rough three-dimensional-modeling scenes

Visual search is a fundamental technology in the computer vision community. It is difficult to find an object in complex scenes when there exist similar distracters in the background. We propose a target search method in rough three-dimensional-modeling scenes based on a vision salience theory and camera imaging model. We give the definition of salience of objects (or features) and explain the way that salience measurements of objects are calculated. Also, we present one type of search path that guides to the target through salience objects. Along the search path, when the previous objects are localized, the search region of each subsequent object decreases, which is calculated through imaging model and an optimization method. The experimental results indicate that the proposed method is capable of resolving the ambiguities resulting from distracters containing similar visual features with the target, leading to an improvement of search speed by over 50%.

What the Heck Is Salience? How Predictive Language Processing Contributes to Sociolinguistic Perception.

Some sociolinguistic variables are prone to hypercorrection, stigmatization and style shifting, while other variables are not. The status of the former type—sometimes called stereotypes and markers (Labov, 1972)—has been attributed to the increased meta-linguistic awareness language users seem to have of these variables. This awareness in turn is attributed to the salience of these variables, such that greater salience is assumed to cause greater meta-linguistic awareness (e.g., Trudgill, 1986). Salience has similarly been invoked when aiming to explain implicit social inferences about, or attitudes toward, speakers who exhibit certain variables in their speech (Babel, 2016; Drager and Kirtley, 2016; Squires, 2016). However, salience is a hard to define concept (for review, see Auer et al., 1998; Kerswill and Williams, 2002) and, partly as a consequence, “notoriously difficult to quantify” (Hickey, 2000). For a concept that plays such a central and ubiquitous role in sociolinguistic explanations, this is arguably a dangerous state of affairs. This motivates the present commentary. We believe that advances in computational psycholinguistics offer definitions of sociolinguistic salience that are more concrete, both empirically and formally grounded, and quantifiable (and thus falsifiable). We propose that it is important to distinguish between the initial salience a listener experiences when first encountering a novel variant (e.g., because of exposure to a previously unfamiliar dialect, sociolect, or idiolect—henceforth lects; Schirmunski, 1930; Preston, 1996), and salience at later stages. Salience after the initial encounter is the cumulative product of an individual's experience related to the lectal variant, including direct experience, as well as discourse about the variant (e.g., explicit stereotyping or enregisterment, Agha, 2003). Here we focus on the causes for initial salience, which we think can be defined in a principled and quantifiable way. Specifically, we propose that salience in the first moments when a novel lect is encountered cannot be understood without reference to prior expectations based on listeners' past language experience and the ensuing expectation violation that a listener experiences relative to those prior expectations—an idea explored in more depth by Racz (2012, 2013). Here we contribute to these efforts. We draw on basic concepts from probability and information theory to define initial salience as a function of (top-down) prior expectations. This has several advantages. First, the proposed definition of salience is quantifiable (see also Racz, 2013). Second, computational psycholinguistics has linked the very same quantities to language processing and learning. Recognizing this link offers the opportunity to ground sociolinguistic salience in human information processing—both empirically and theoretically—offering a parsimonious account of initial salience. After we have outlined our proposal, we briefly turn to an apparent puzzle that was raised during the workshop leading to this special issue: several presenters pointed out that salience sometimes seems to be inversely related to the frequency of a variant and other times positively related. This puzzle readily dissolves once the view proposed here is taken into account.

Robust Search-Free Car Number Plate Localization Incorporating Hierarchical Saliency

There are two major shortcomings associated with presently implemented automatic license plate recognition (ALPR) systems: first, processing images with complex background is time-consuming and second, the results are not sufficiently accurate. To overcome these problems and also to achieve a robust recognition of multiple car number plates, saliency detection based on the ALPR system is used in this paper and also an improved and more effective definition of saliency is presented. In this new approach, the notion of the directionality of the edges using Gabor filtering and the detection of the patterns of numbers using L1-norm have been added to the traditional saliency detection method. The proposed algorithm was tested on 660 images; some consisting of two or more cars. A detection accuracy of 94.77% and an average execution time of 40 ms for 600 × 800 images are the marked outcomes. The proposed SB-ALPR method outperforms most of the state of the art techniques in terms of execution time and accuracy, and can be used in real-time applications. Also, unlike some recently introduced saliency-based ALPR methods, our two-stage saliency detection approach exploits smaller numbers of sample sizes to reduce the computation cost.

Stereo matching-based definition of saliency via sample-based Kullback–Leibler divergence estimation

The saliency of a given image region for stereo matching is measured by the Kullback---Leibler divergence from the conditional probability density function (pdf) for the matching region to a pdf for the background. If the region has a high saliency, then there is a high probability that the correct matching region can be found. The Kullback---Leibler divergence is directly estimated from the samples without an intermediate estimation of the pdfs. Experiments with the stereo images in the Middlebury database show that the proposed estimation of the Kullback---Leibler divergence is faster than the estimation based on parametric models for the pdfs, with no loss of accuracy.

An Improved Saliency Detection Approach for Flying Apsaras in the Dunhuang Grotto Murals, China

Saliency can be described as the ability of an item to be detected from its background in any particular scene, and it aims to estimate the probable location of the salient objects. Due to the salient map that computed by local contrast features can extract and highlight the edge parts including painting lines of Flying Apsaras, in this paper, we proposed an improved approach based on a frequency-tuned method for visual saliency detection of Flying Apsaras in the Dunhuang Grotto Murals, China. This improved saliency detection approach comprises three important steps: (1) image color and gray channel decomposition; (2) gray feature value computation and color channel convolution; (3) visual saliency definition based on normalization of previous visual saliency and spatial attention function. Unlike existing approaches that rely on many complex image features, this proposed approach only used local contrast and spatial attention information to simulate human’s visual attention stimuli. This improved approach resulted in a much more efficient salient map in the aspect of computing performance. Furthermore, experimental results on the dataset of Flying Apsaras in the Dunhuang Grotto Murals showed that the proposed visual saliency detection approach is very effective when compared with five other state-of-the-art approaches.

"in dem moment wo ich es dann erkenne dann ist es auch gleich wieder weg" – Salienzeffekte in der Sprachperzeption

In context of the first study on folk linguistic concepts in the German language area carried out by the Kiel DFG research project "perceptual dialectology", this article looks at how salient features could be surveyed and categorized by a stimulus-response-test. After a definition of salience, the study design including the stimulus-response-test is presented. The test was created and modified during the project as a guessing game by the Institute for German Language (IDS Mannheim). The central question in this article is which linguistic features stimulate the informants to locate a speech sample on a map with predetermined cities and hence which salient features trigger the regional identification. In a second step, the speech samples are analyzed by the variables 'pleasantness' and 'correctness' defined by Dennis R. Preston. The central question here is: Are speech samples with a high pleasure value also automatically considered correct? Finally, an interpretation of metalinguistic comments in the speech examples will give more insight into folk linguistic concepts and the role of salient features in this regard.

Saliency detection with color contrast based on boundary information and neighbors

Object-level saliency detection is significant in many computer vision tasks. In this paper, we propose a novel saliency detection model based on color contrast and image boundaries. The saliency of an image is defined as the contrast between the image elements (regions) and image boundaries elements (regions). We consider the saliency in two-stage procedure rather than in one stage. First of all, according to the definition of saliency, we take four boundaries of image into consideration respectively to obtain a combination coarse saliency map. Furthermore, a new energy function based on the coarse saliency map is proposed, which takes the coarse saliency map as input to yield the final full resolution saliency map. Experimental results on two public datasets demonstrate that the proposed model performs better than the state-of-the-art methods.

Timbre saliency vs. timbre dissimilarity – What is the relationship?

We have proposed the notion of timbre saliency as the attention-capturing quality of timbre. The definition of saliency requires an object to stand out with respect to its surroundings, implying dissimilarity between the object and its neighbors. What then might be the relationship between timbre saliency and timbre dissimilarity? A classic timbre dissimilarity experiment and a timbre saliency experiment were carried out with 20 participants on the same set of stimuli. Multidimensional scaling revealed a two-dimensional dissimilarity space. Using the features obtained from the Timbre Toolbox [Peeters et al., J. Acoust. Soc. Am. 130, 2902–2916 (2011)], the first dimension shows a high correlation with spectral centroid [r(13) = 0.845, p < 0.0001] and spectral spread [r(13) = 0.855, p < 0.0001], both based on the ERB-FFT model spectrum, and the second with the attack time [r(13) = -0.692, p = 0.004] and power spectral crest [r(13) = 0.732, p < .005]. This confirms spectral centroid and attack time as two major acoustic correlates of timbre dissimilarity. The saliency dimension shows a moderate correlation with the second dimension [r(13) = 0.578, p = 0.0241] but not with the first dimension [r(13) = 0.182, p = 0.517], suggesting that the saliency might be more related to the temporal characteristics of timbre.

A probabilistic definition of salient regions for image matching

A probabilistic definition of saliency is given in a form suitable for applications to image matching. In order to make this definition, the values of the pixels in pairs of matching regions are modeled using an elliptically symmetric distribution (ESD). The values of the pixels in background pairs of regions are also modeled using an ESD. If a region is given in one image, then the conditional probability density function for the pixel values in a matching region can be calculated. The saliency of the given region is defined to be the Kullback–Leibler divergence between this conditional pdf and a background conditional pdf. Experiments carried out using images in the Middlebury stereo database show that if the salience of a given image region is high, then there are relatively few background regions that have a better match to the given region than the true matching region.

Context-Aware Saliency Detection

We propose a new type of saliency—context-aware saliency—which aims at detecting the image regions that represent the scene. This definition differs from previous definitions whose goal is to either identify fixation points or detect the dominant object. In accordance with our saliency definition, we present a detection algorithm which is based on four principles observed in the psychological literature. The benefits of the proposed approach are evaluated in two applications where the context of the dominant objects is just as essential as the objects themselves. In image retargeting, we demonstrate that using our saliency prevents distortions in the important regions. In summarization, we show that our saliency helps to produce compact, appealing, and informative summaries.

Multi-spectral saliency detection

Visual saliency detection has been applied in many tasks in the fields of pattern recognition and computer vision, such as image segmentation, object recognition, and image retargeting. However, the accurate detection of saliency remains a challenge. The reasons behind this are that: (1) well-defined mechanism for saliency definition is rarely established; and (2) supporting information for detecting saliency is limited in general. In this paper, a multi-spectrum based saliency detection algorithm is proposed. Instead of only using the conventional RGB information as what existing algorithms do, this work incorporates near-infrared clues into the detection framework. Features of color and texture from both types of image modes are explored simultaneously. When calculating the color contrast, an effective color component analysis method is employed to produce more precise results. With respect to the texture analysis, texton representation is adopted for fast processing. Experiments are done to compare the proposed algorithm with other 11 state-of-the-art algorithms and the results indicate that our algorithm outperforms the others.

The Improbability of Harris Interest Points

An elementary characterization of the map underlying Harris corners, also known as Harris interest points or key points, is provided. Two principal and basic assumptions made are: 1) Local image structure is captured in an uncommitted way, simply using weighted raw image values around every image location to describe the local image information, and 2) the lower the probability of observing the image structure present in a particular point, the more salient, or interesting, this position is, i.e., saliency is related to how uncommon it is to see a certain image structure, how surprising it is. Through the latter assumption, the axiomatization proposed makes a sound link between image saliency in computer vision on the one hand and, on the other, computational models of preattentive human visual perception, where exactly the same definition of saliency has been proposed. Because of this link, the characterization provides a compelling case in favor of Harris interest points over other approaches.

SUN: A Bayesian framework for saliency using natural statistics

We propose a definition of saliency by considering what the visual system is trying to optimize when directing attention. The resulting model is a Bayesian framework from which bottom-up saliency emerges naturally as the self-information of visual features, and overall saliency (incorporating top-down information with bottom-up saliency) emerges as the pointwise mutual information between the features and the target when searching for a target. An implementation of our framework demonstrates that our model's bottom-up saliency maps perform as well as or better than existing algorithms in predicting people's fixations in free viewing. Unlike existing saliency measures, which depend on the statistics of the particular image being viewed, our measure of saliency is derived from natural image statistics, obtained in advance from a collection of natural images. For this reason, we call our model SUN (Saliency Using Natural statistics). A measure of saliency based on natural image statistics, rather than based on a single test image, provides a straightforward explanation for many search asymmetries observed in humans; the statistics of a single test image lead to predictions that are not consistent with these asymmetries. In our model, saliency is computed locally, which is consistent with the neuroanatomy of the early visual system and results in an efficient algorithm with few free parameters.

Components of bottom-up gaze allocation in natural images

Recent research [Parkhurst, D., Law, K., & Niebur, E., 2002. Modeling the role of salience in the allocation of overt visual attention. Vision Research 42 (1) (2002) 107–123] showed that a model of bottom-up visual attention can account in part for the spatial locations fixated by humans while free-viewing complex natural and artificial scenes. That study used a definition of salience based on local detectors with coarse global surround inhibition. Here, we use a similar framework to investigate the roles of several types of non-linear interactions known to exist in visual cortex, and of eccentricity-dependent processing. For each of these, we added a component to the salience model, including richer interactions among orientation-tuned units, both at spatial short range (for clutter reduction) and long range (for contour facilitation), and a detailed model of eccentricity-dependent changes in visual processing. Subjects free-viewed naturalistic and artificial images while their eye movements were recorded, and the resulting fixation locations were compared with the models’ predicted salience maps. We found that the proposed interactions indeed play a significant role in the spatiotemporal deployment of attention in natural scenes; about half of the observed inter-subject variance can be explained by these different models. This suggests that attentional guidance does not depend solely on local visual features, but must also include the effects of interactions among features. As models of these interactions become more accurate in predicting behaviorally-relevant salient locations, they become useful to a range of applications in computer vision and human-machine interface design.

Relevance feedback based saliency adaptation in CBIR

Content-based image retrieval (CBIR) has been under investigation for a long time, with many systems built to meet different application demands. However, in all systems there is still a gap between user expectations and system retrieval capabilities. Therefore, user interaction is an essential component of any CBIR system. Interaction up to now has mostly focused on changing global image features or similarities between images. We consider the interaction with salient details in an image, i.e., points, lines, and regions. Interactive salient detail definition goes further than summarizing the image into a set of salient details. We aim to dynamically update the user- and context-dependent definition of saliency based on relevance feedback. To that end, we propose an interaction framework for salient details from the perspective of the user. A number of instantiations of the framework are presented. Finally, we apply our approach for query refinement in a detail-based image retrieval system with salient points and regions. Experimental results prove the effectiveness of adapting the saliency from user feedback in the retrieval process.

L'apprenant, le chercheur et les discours: quelques réflexions sur la notion de saillance

The learner is exposed to second language through discourses. Due to their relative impenetrableness, the learner must choose selection fragments in order to initiate some sort of interpretation, or even participation, as limited as it may be, in a communication activity. This selection unfolds as a focus on the most salient fragments of the speech, which hence become inputs. The definition of saliency is a highly complex process based on objective and subjective criteria, which we are trying to detail in our article. To what extent do these inputs coincide with the data investigators use in order to understand the learning process? A discussion on this matter is used as the framework for this study on saliency.