Salience Signals Research Articles

How we learn to ignore singleton distractors: Suppressing saliency signals or specific features?

ABSTRACT Salient visual information can sometimes capture attention despite our goals. However, there are several ways we can minimize or eliminate such distraction. One such way is learned distractor rejection, in which we increasingly ignore salient, irrelevant distractors across repeated exposures. Here, we probe the mechanism underlying this learned rejection. What must be learned about the distractor to promote effective ignoring? Specifically, is feature rejection, alone, sufficient to ignore salient distractors, or do the items' saliency signals need to also be rejected? We used a modified version of the learned distractor rejection paradigm (Vatterott & Vecera, 2012) in which participants viewed training blocks containing either a salient singleton distractor or three non-salient “tripleton” distractors, followed by test blocks in which the distractor shared a feature (color) with the corresponding training blocks, but was always salient. Results revealed unexpected difficulties in replicating learned distractor rejection, suggesting the true effect size may be smaller than initially reported. With respect to our main objective, we found similar attenuation of singleton presence costs in the test blocks regardless of whether they followed singleton or tripleton training blocks. These results show that experience in rejecting saliency signals is not a requirement of learned distractor rejection.

Separate and overlapping mechanisms of statistical regularities and salience processing in the occipital cortex and dorsal attention network.

Attention selects behaviorally relevant inputs for in-depth processing. Beside the role of traditional signals related to goal-directed and stimulus-driven control, a debate exists regarding the mechanisms governing the effect of statistical regularities on attentional selection, and how these are integrated with other control signals. Using a visuo-spatial search task under fMRI, we tested the joint effects of statistical regularities and stimulus-driven salience. We found that both types of signals modulated occipital activity in a spatially specific manner. Salience acted primarily by reducing the attention bias towards the target location when associated with irrelevant distractors, while statistical regularities reduced this attention bias when the target was presented at a low probability location, particularly at the lower levels of the visual hierarchy. In addition, we found that both statistical regularities and salience activated the dorsal frontoparietal network. Additional exploratory analyses of functional connectivity revealed that only statistical regularities modulated the inter-regional coupling between the posterior parietal cortex and the occipital cortex. These results show that statistical regularities and salience signals are both spatially represented at the occipital level, but that their integration into attentional processing priorities relies on dissociable brain mechanisms.

How we learn to ignore singleton distractors: Suppressing saliency signals or specific features?

How we learn to ignore singleton distractors: Suppressing saliency signals or specific features?

A Critique of the Attentional Window Account of Capture Failures.

There has been a lengthy debate about whether salient stimuli have the power to automatically capture attention, even when entirely task irrelevant. Theeuwes (2022) has suggested that an attentional window account could explain why capture is observed in some studies, but not others. According to this account, when search is difficult, participants narrow their attentional window, and this prevents the salient distractor from generating a saliency signal. In turn, this causes the salient distractor to fail to capture attention. In the present commentary, we describe two major problems with this account. First, the attentional window account proposes that attention must be focused so narrowly that featural information from the salient distractor will be filtered prior to saliency computations. However, many previous studies observing no capture provided evidence that featural processing was sufficiently detailed to guide attention toward the target shape. This indicates that the attentional window was sufficiently broad to allow featural processing. Second, the attentional window account proposes that capture should occur more readily in easy search tasks than difficult search tasks. We review previous studies that violate this basic prediction of the attentional window account. A more parsimonious account of the data is that control over feature processing can be exerted proactively to prevent capture, at least under certain conditions.

Clinical response to treatment with a partial dopamine agonist is related to changes in reward processing

Aberrant neuronal coding of reward processing has been linked to psychosis. It remains unresolved how treatment with a partial dopamine agonist affects reward processing, and whether treatment affects reward processing differently in patients responding and not responding to treatment. Here, 33 antipsychotic-naïve psychosis patients and 33 matched healthy controls underwent functional magnetic resonance imaging before and after patients received aripiprazole monotherapy for six weeks. Processing of motivational salient events and negative outcome evaluation (NOE) was examined using a monetary incentive delay task. Psychopathology was assessed with the Positive and Negative Syndrome Scale, and responders were identified by having ≥30% reduction in positive symptoms (N=21). At baseline, patients displayed an increased NOE signal in the caudate and dorsolateral prefrontal cortex compared to healthy controls. In the caudate, the NOE signal was normalized at follow-up, and normalization was driven by responders. In responders only, there was a significant improvement in the motivational salience signal in the caudate at follow-up. Motivational salience and NOE signals in the caudate may be associated with a dopaminergic mechanism in patients characterized as responders which may not be the case in non-responders. Likewise, non-dopaminergic mechanism may underly abnormal NOE processing in dorsolateral prefrontal cortex.

Long-term (statistically learnt) and short-term (inter-trial) distractor-location effects arise at different pre- and post-selective processing stages.

A salient distractor interferes less with visual search if it appears at a location where it is likely to occur, referred to as distractor-location probability cueing. Conversely, if the current target appears at the same location as a distractor on the preceding trial, search is impeded. While these two location-specific "suppression" effects reflect long-term, statistically learnt and short-term, inter-trial adaptations of the system to distractors, it is unclear at what stage(s) of processing they arise. Here, we adopted the additional-singleton paradigm and examined lateralized event-related potentials (L-ERPs) and lateralized alpha (8-12 Hz) power to track the temporal dynamics of these effects. Behaviorally, we confirmed both effects: reaction times (RTs) interference was reduced for distractors at frequent versus rare (distractor) locations, and RTs were delayed for targets that appeared at previous distractor versus non-distractor locations. Electrophysiologically, the statistical-learning effect was not associated with lateralized alpha power during the pre-stimulus period. Rather, it was seen in an early N1pc referenced to the frequent distractor location (whether or not a distractor or a target occurred there), indicative of a learnt top-down prioritization of this location. This early top-down influence was systematically modulated by (competing) target- and distractor-generated bottom-up saliency signals in the display. In contrast, the inter-trial effect was reflected in an enhanced SPCN when the target was preceded by a distractor at its location. This suggests that establishing that an attentionally selected item is a task-relevant target, rather than an irrelevant distractor, is more demanding at a previously "rejected" distractor location.

The role of salience in the suppression of distracting stimuli.

Researchers have long debated whether salient distractors have the power to automatically capture attention. Recent research has suggested a potential resolution, called the signal suppression hypothesis, whereby salient distractors produce a bottom-up salience signal, but can be suppressed to prevent visual distraction. This account, however, has been criticized on the grounds that previous studies may have used distractors that were only weakly salient. This claim has been difficult to empirically test because there are currently no well-established measures of salience. The current study addresses this by introducing a psychophysical technique to measure salience. First, we generated displays that aimed to manipulate the salience of two color singletons via color contrast. We then verified that this manipulation was successful using a psychophysical technique to determine the minimum exposure duration required to detect each color singleton. The key finding was that high-contrast singletons were detected at briefer exposure thresholds than low-contrast singletons, suggesting that high-contrast singletons were more salient. Next, we evaluated the participants' ability to ignore these singletons in a task in which they were task irrelevant. The results showed that, if anything, high-salience singletons were more strongly suppressed than low-salience singletons. These results generally support the signal suppression hypothesis and refute claims that highly salient singletons cannot be ignored.

Distinct cortico-striatal compartments drive competition between adaptive and automatized behavior.

Cortical and basal ganglia circuits play a crucial role in the formation of goal-directed and habitual behaviors. In this study, we investigate the cortico-striatal circuitry involved in learning and the role of this circuitry in the emergence of inflexible behaviors such as those observed in addiction. Specifically, we develop a computational model of cortico-striatal interactions that performs concurrent goal-directed and habit learning. The model accomplishes this by distinguishing learning processes in the dorsomedial striatum (DMS) that rely on reward prediction error signals as distinct from the dorsolateral striatum (DLS) where learning is supported by salience signals. These striatal subregions each operate on unique cortical input: the DMS receives input from the prefrontal cortex (PFC) which represents outcomes, and the DLS receives input from the premotor cortex which determines action selection. Following an initial learning of a two-alternative forced choice task, we subjected the model to reversal learning, reward devaluation, and learning a punished outcome. Behavior driven by stimulus-response associations in the DLS resisted goal-directed learning of new reward feedback rules despite devaluation or punishment, indicating the expression of habit. We repeated these simulations after the impairment of executive control, which was implemented as poor outcome representation in the PFC. The degraded executive control reduced the efficacy of goal-directed learning, and stimulus-response associations in the DLS were even more resistant to the learning of new reward feedback rules. In summary, this model describes how circuits of the dorsal striatum are dynamically engaged to control behavior and how the impairment of executive control by the PFC enhances inflexible behavior.

Target templates and the time course of distractor location learning

When searching for a shape target, colour distractors typically capture our attention. Capture is smaller when observers search for a fixed target that allows for a feature-specific target template compared to a varying shape singleton target. Capture is also reduced when observers learn to predict the likely distractor location. We investigated how the precision of the target template modulates distractor location learning in an additional singleton search task. As observers are less prone to capture with a feature-specific target, we assumed that distractor location learning is less beneficial and therefore less pronounced than with a mixed-feature target. Hierarchical Bayesian parameter estimation was used to fit fine-grained distractor location learning curves. A model-based analysis of the time course of distractor location learning revealed an effect on the asymptotic performance level: when searching for a fixed-feature target, the asymptotic distractor cost indicated smaller distractor interference than with a mixed-feature target. Although interference was reduced for distractors at the high-probability location in both tasks, asymptotic distractor suppression was less pronounced with fixed-feature compared to mixed-feature targets. We conclude that with a more precise target template less distractor location learning is required, likely because the distractor dimension is down-weighted and its salience signal reduced.

Master-CAM: Multi-scale fusion guided by Master map for high-quality class activation maps

Class Activation Map (CAM) is one of the most popular approaches to visually explain the convolutional neural networks (CNNs). To obtain fine-grained saliency maps, some works fuse saliency signals of the same image at larger scales. However, existing methods based on multi-scale fusion cannot effectively remove the noise from larger-scale images. In this paper, we propose Master-CAM, which uses Master map to guide multi-scale fusion process to obtain a high-quality class activation map. Master-CAM utilizes the general localization ability of the Master map to reduce the noise of the maps. We call the one with the general localization ability among the saliency maps from the same image as Master map, which is the saliency map of the original-scale input in the multi-scale scenario. In addition, we also present a simple yet effective fusion strategy, Master-Fusion, which is derived from the fusion operation in Master-CAM. Master-Fusion strategy can be easily attached to some saliency methods to improve the performance of these methods. We show through qualitative and quantitative experiments that the proposed Master-CAM outperforms the state-of-the-art methods in different CNN frameworks and datasets.

Neuropeptide S Encodes Stimulus Salience in the Paraventricular Thalamus

Evaluation of stimulus salience is critical for any higher organism, as it allows for prioritizing of vital information, preparation of responses, and formation of valuable memory. The paraventricular nucleus of the thalamus (PVT) has recently been identified as an integrator of stimulus salience but the neurochemical basis and afferent input regarding salience signaling have remained elusive. Here we report that neuropeptide S (NPS) signaling in the PVT is necessary for stimulus salience encoding, including aversive, neutral and reinforcing sensory input. Taking advantage of a striking deficit of both NPS receptor (NPSR1) and NPS precursor knockout mice in fear extinction or novel object memory formation, we demonstrate that intra-PVT injections of NPS can rescue the phenotype in NPS precursor knockout mice by increasing the salience of otherwise low-intensity stimuli, while intra-PVT injections of NPSR1 antagonist in wild type mice partially replicates the knockout phenotype. The PVT appears to provide stimulus salience encoding in a dose- and NPS-dependent manner. PVT NPSR1 neurons recruit the nucleus accumbens shell and structures in the prefrontal cortex and amygdala, which were previously linked to the brain salience network. Overall, these results demonstrate that stimulus salience encoding is critically associated with NPS activity in the PVT.

Linking Salience Signaling With Early Adversity and Affective Distress in Individuals at Clinical High Risk for Psychosis: Results From an Event-Related fMRI Study.

Evidence suggests dysregulation of the salience network in individuals with psychosis, but few studies have examined the intersection of stress exposure and affective distress with prediction error (PE) signals among youth at clinical high-risk (CHR). Here, 26 individuals at CHR and 19 healthy volunteers (HVs) completed a monetary incentive delay task in conjunction with fMRI. We compared these groups on the amplitudes of neural responses to surprising outcomes—PEs without respect to their valence—across the whole brain and in two regions of interest, the anterior insula and amygdala. We then examined relations of these signals to the severity of depression, anxiety, and trauma histories in the CHR group. Relative to HV, youth at CHR presented with aberrant PE-evoked activation of the temporoparietal junction and weaker deactivation of the precentral gyrus, posterior insula, and associative striatum. No between-group differences were observed in the amygdala or anterior insula. Among youth at CHR, greater trauma histories were correlated with stronger PE-evoked amygdala activation. No associations were found between affective symptoms and the neural responses to PE. Our results suggest that unvalenced PE signals may provide unique information about the neurobiology of CHR syndromes and that early adversity exposure may contribute to neurobiological heterogeneity in this group. Longitudinal studies of young people with a range of risk syndromes are needed to further disentangle the contributions of distinct aspects of salience signaling to the development of psychopathology.

Schizophrenia Patients Show Largely Similar Salience Signaling Compared to Healthy Controls in an Observational Task Environment.

Recent evidence suggests that the aberrant signaling of salience is associated with psychotic illness. Salience, however, can take many forms in task environments. For example, salience may refer to any of the following: (1) the valence of an outcome, (2) outcomes that are unexpected, called reward prediction errors (PEs), or (3) cues associated with uncertain outcomes. Here, we measure brain responses to different forms of salience in the context of a passive PE-signaling task, testing whether patients with schizophrenia (SZ) showed aberrant signaling of particular types of salience. We acquired event-related MRI data from 29 SZ patients and 23 controls during the performance of a passive outcome prediction task. Across groups, we found that the anterior insula and posterior parietal cortices were activated to multiple different types of salience, including PE magnitude and heightened levels of uncertainty. However, BOLD activation to salient events was not significantly different between patients and controls in many regions, including the insula, posterior parietal cortices, and default mode network nodes. Such results suggest that deficiencies in salience processing in SZ may not result from an impaired ability to signal salience per se, but instead the ability to use such signals to guide future actions. Notably, no between-group differences were observed in BOLD signal changes associated with PE-signaling in the striatum. However, positive symptom severity was found to significantly correlate with the magnitudes of salience contrasts in default mode network nodes. Our results suggest that, in an observational environment, SZ patients may show an intact ability to activate striatal and cortical regions to rewarding and non-rewarding salient events. Furthermore, reduced deactivation of a hypothesized default mode network node for SZ participants with high levels of positive symptoms, following salient events, point to abnormalities in interactions of the salience network with other brain networks, and their potential importance to positive symptoms.

Individual differences in error-related frontal midline theta activity during visuomotor adaptation

Post-feedback frontal midline EEG activity has been found to correlate with error magnitude during motor adaptation. However, the role of this neuronal activity remains to be elucidated. It has been hypothesized that post-feedback frontal midline activity may represent a prediction error, which in turn may be directly related to the adaptation process or to an unspecific orienting response. To address these hypotheses, we replicated a previous visuomotor adaptation experiment with very small perturbations, likely to invoke implicit adaptation, in a new group of 60 participants and combined it with EEG recordings. We found error-related peaks in the frontal midline electrodes in the time domain. However, these were best understood as modulations of frontal midline theta activity (FMT, 4–8 Hz). Trial-level differences in FMT correlated with error magnitude. This correlation was robust even for very small errors as well as in the absence of imposed perturbations, indicating that FMT does not depend on explicit or strategic re-aiming. Within participants, trial-level differences in FMT were not related to between-trial error corrections. Between participants, individual differences in FMT-error-sensitivity did not predict differences in adaptation rate. Taken together, these results imply that FMT does not drive implicit motor adaptation. Finally, individual differences in FMT-error-sensitivity negatively correlate to motor execution noise. This suggests that FMT reflects saliency: larger execution noise means a larger standard deviation of errors so that a fixed error magnitude is less salient. In conclusion, this study suggests that frontal midline theta activity represents a saliency signal and does not directly drive motor adaptation.

Relative salience signaling within a thalamo-orbitofrontal circuit governs learning rate

Learning to predict rewards is essential for the sustained fitness of animals. Contemporary views suggest that such learning is driven by a reward prediction error (RPE)-the difference between received and predicted rewards. The magnitude of learning induced by an RPE is proportional to the product of the RPE and a learning rate. Here we demonstrate using two-photon calcium imaging and optogenetics in mice that certain functionally distinct subpopulations of ventral/medial orbitofrontal cortex (vmOFC) neurons signal learning rate control. Consistent with learning rate control, trial-by-trial fluctuations in vmOFC activity positively correlate with behavioral updating when the RPE is positive, and negatively correlates with behavioral updating when the RPE is negative. Learning rate is affected by many variables including the salience of a reward. We found that the average reward response of these neurons signals the relative salience of a reward, because it decreases after reward prediction learning or the introduction of another highly salient aversive stimulus. The relative salience signaling in vmOFC is sculpted by medial thalamic inputs. These results support emerging theoretical views that prefrontal cortex encodes and controls learning parameters.

Separable actions of acetylcholine and noradrenaline on neuronal ensemble formation in hippocampal CA3 circuits.

In the hippocampus, episodic memories are thought to be encoded by the formation of ensembles of synaptically coupled CA3 pyramidal cells driven by sparse but powerful mossy fiber inputs from dentate gyrus granule cells. The neuromodulators acetylcholine and noradrenaline are separately proposed as saliency signals that dictate memory encoding but it is not known if they represent distinct signals with separate mechanisms. Here, we show experimentally that acetylcholine, and to a lesser extent noradrenaline, suppress feed-forward inhibition and enhance Excitatory-Inhibitory ratio in the mossy fiber pathway but CA3 recurrent network properties are only altered by acetylcholine. We explore the implications of these findings on CA3 ensemble formation using a hierarchy of models. In reconstructions of CA3 pyramidal cells, mossy fiber pathway disinhibition facilitates postsynaptic dendritic depolarization known to be required for synaptic plasticity at CA3-CA3 recurrent synapses. We further show in a spiking neural network model of CA3 how acetylcholine-specific network alterations can drive rapid overlapping ensemble formation. Thus, through these distinct sets of mechanisms, acetylcholine and noradrenaline facilitate the formation of neuronal ensembles in CA3 that encode salient episodic memories in the hippocampus but acetylcholine selectively enhances the density of memory storage.

Relations Among Anhedonia, Reinforcement Learning, and Global Functioning in Help-seeking Youth.

Dysfunction in the neural circuits underlying salience signaling is implicated in symptoms of psychosis and may predict conversion to a psychotic disorder in youth at clinical high risk (CHR) for psychosis. Additionally, negative symptom severity, including consummatory and anticipatory aspects of anhedonia, may predict functional outcome in individuals with schizophrenia-spectrum disorders. However, it is unclear whether anhedonia is related to the ability to attribute incentive salience to stimuli (through reinforcement learning [RL]) and whether measures of anhedonia and RL predict functional outcome in a younger, help-seeking population. We administered the Salience Attribution Test (SAT) to 33 participants who met criteria for either CHR or a recent-onset psychotic disorder and 29 help-seeking youth with nonpsychotic disorders. In the SAT, participants must identify relevant and irrelevant stimulus dimensions and be sensitive to different reinforcement probabilities for the 2 levels of the relevant dimension ("adaptive salience"). Adaptive salience attribution was positively related to both consummatory pleasure and functioning in the full sample. Analyses also revealed an indirect effect of adaptive salience on the relation between consummatory pleasure and both role (αβ = .22, 95% CI = 0.02, 0.48) and social functioning (αβ = .14, 95% CI = 0.02, 0.30). These findings suggest a distinct pathway to poor global functioning in help-seeking youth, via impaired reward sensitivity and RL.

Voluntary choice tasks increase control settings and reduce capture

ABSTRACT In their article, Luck et al. ([2021]. Progress toward resolving the attentional capture debate. Visual Cognition, 29(1), 1–21. https://doi.org/10.1080/13506285.2020.1848949) outline alternative hypotheses regarding the cognitive mechanisms that modulate attention capture. This commentary addresses how control signals can be used to modulate feature gain control prior to saliency computations, in addition to the implicitly learned effects of alternate selection histories. We suggest that voluntary choice paradigms provide much stronger task-induced attentional control than pre-trial cueing, and allow a comparison of the same search stimuli under different control states. We argue that voluntary task choice results in an attenuation of salience signals, eliminating attention capture, although response time distractor costs remain.

Attentional capture: An ameliorable side-effect of searching for salient targets

ABSTRACT This commentary highlights that some of the remaining discrepancies in the attentional-capture debate can be resolved by a simple assumption: observers do not use the priority map when this map is useless to solve the task. Rather, whenever search targets are known to be non-salient, observers resort to a previously postulated alternative search strategy for which (distractor) saliency signals are irrelevant. Equipped with this assumption, we trace thus-far unaccounted-for discrepancies between empirical studies on attentional capture back to specific design choices that affect relative target saliency (display density and non-target heterogeneity).

Standing out in a small crowd: The role of display size in attracting attention

ABSTRACT Strong evidence supporting the top-down modulation of attention has come from studies in which participants learned to suppress a singleton in a heterogeneous four-item display. These studies have been criticized on the grounds that the displays are so sparse that the singleton is not actually salient. We argue that similar evidence of suppression has been found with substantially larger displays where salience is not in question. Additionally, we examine the results of applying salience models to four-item displays, and find prominent markers of salience at the location of the singleton. We conclude that small heterogeneous displays do not preclude strong salience signals. Beyond that, we reflect on how further basic research on salience may speed resolution of the attentional capture debate.