Bottom-up Attention Mechanism Research Articles

Task-Based Visual Attention for Continually Improving the Performance of Autonomous Game Agents

Deep Reinforcement Learning (DRL) has been effectively performed in various complex environments, such as playing video games. In many game environments, DeepMind’s baseline Deep Q-Network (DQN) game agents performed at a level comparable to that of humans. However, these DRL models require many experience samples to learn and lack the adaptability to changes in the environment and handling complexity. In this study, we propose Attention-Augmented Deep Q-Network (AADQN) by incorporating a combined top-down and bottom-up attention mechanism into the DQN game agent to highlight task-relevant features of input. Our AADQN model uses a particle-filter -based top-down attention that dynamically teaches an agent how to play a game by focusing on the most task-related information. In the evaluation of our agent’s performance across eight games in the Atari 2600 domain, which vary in complexity, we demonstrate that our model surpasses the baseline DQN agent. Notably, our model can achieve greater flexibility and higher scores at a reduced number of time steps.Across eight game environments, AADQN achieved an average relative improvement of 134.93%. Pong and Breakout games both experienced improvements of 9.32% and 56.06%, respectively. Meanwhile, SpaceInvaders and Seaquest, which are more intricate games, demonstrated even higher percentage improvements, with 130.84% and 149.95%, respectively. This study reveals that AADQN is productive for complex environments and produces slightly better results in elementary contexts.

Driver's visual fixation attention prediction in dynamic scenes using hybrid neural networks

The traffic driving scenario is dynamically varying and experienced drivers can allocate visual attention effectively in brief time intervals, focusing on prominent targets and areas in advance, thus ensuring safe driving. Therefore, it is significant to research the behavior of driver's visual attention allocation in the development of driver assistance systems and autonomous vehicles. Recently, inspired by top-down and bottom-up attention mechanisms, most visual attention models based on neural networks have been proposed. However, these models tend to yield highly distributed predictions for the specific driving scenario, and have relatively low prediction accuracy, making them unfeasible for being used in practice. Hence, to overcome these challenges, we propose a driver's visual attention model (DVAM) built on the encoder-decoder architecture with the utilization of the convolutional neural network (CNN) and the recurrent neural networks (RNN). Specifically, the model leverages the temporal and spatial dimension information to better mimic realistic dynamic driving, and extract abundant feature information, thus ensuring that the final predictions are authentic and effective. Then, extensive experiments demonstrate that our suggested DVAM achieves better robustness and precision in predicting driver attention to areas or targets compared to state-of-the-art methods on the DR(eye)VE dataset. Finally, our model is deployed on the TDV dataset to validate its generalization capabilities, which prove the potential for field adaptation.

Towards Parallel Selective Attention Using Psychophysiological States as the Basis for Functional Cognition.

Attention is a complex cognitive process with innate resource management and information selection capabilities for maintaining a certain level of functional awareness in socio-cognitive service agents. The human-machine society depends on creating illusionary believable behaviors. These behaviors include processing sensory information based on contextual adaptation and focusing on specific aspects. The cognitive processes based on selective attention help the agent to efficiently utilize its computational resources by scheduling its intellectual tasks, which are not limited to decision-making, goal planning, action selection, and execution of actions. This study reports ongoing work on developing a cognitive architectural framework, a Nature-inspired Humanoid Cognitive Computing Platform for Self-aware and Conscious Agents (NiHA). The NiHA comprises cognitive theories, frameworks, and applications within machine consciousness (MC) and artificial general intelligence (AGI). The paper is focused on top-down and bottom-up attention mechanisms for service agents as a step towards machine consciousness. This study evaluates the behavioral impact of psychophysical states on attention. The proposed agent attains almost 90% accuracy in attention generation. In social interaction, contextual-based working is important, and the agent attains 89% accuracy in its attention by adding and checking the effect of psychophysical states on parallel selective attention. The addition of the emotions to attention process produced more contextual-based responses.

"Emotional distractor images disrupt target processing in a graded manner": Correction.

Reports an error in "Emotional distractor images disrupt target processing in a graded manner" by Jonathan M. Keefe and David H. Zald (Emotion, Advanced Online Publication, Aug 27, 2020, np). In the article "Emotional Distractor Images Disrupt Target Processing in a Graded Manner" by Jonathan M. Keefe and David H. Zald (Emotion, advance online publication, August 27, 2020, https://doi.org/10.1037/emo0000893), there were errors in the reporting of nonresponse rate and accuracy data. Nonresponse rate was underreported for the data in the Lag 2 condition, resulting in incorrect analysis of variance values for this portion of the Results section as well as incorrect Lag 2 t-test values in Table 1 and incorrect Lag 2 values in Figure 2A. Additionally, error bars for the accuracy data in Figure 2B were mistakenly calculated with data including excluded trials, resulting in larger estimates of standard error of the mean. These corrections do not affect the interpretation of any inferential statistics and in fact increased the effect of lag and distractor valence upon both of these measures. Therefore, no conclusions of the study are altered. All versions of this article have been corrected. (The following abstract of the original article appeared in record 2020-63434-001). The emotional attentional blink (EAB), also referred to as emotion-induced blindness, refers to a transient impairment in the ability to discriminate a single target when it is presented closely in time to an emotional distractor. Although the EAB has typically been characterized as representing a complete loss of target information due to attentional capture by the emotional distractors, it is unclear whether the impact of the emotional distractor is in fact discrete or graded. Here, we tested whether the emotional distractor of the EAB interfered with target processing in a continuous or all-or-none manner by measuring changes in both reaction time (RT) and target-vividness ratings in addition to target-discrimination accuracy. Rapid sequences of landscape images were presented centrally, and participants reported the orientation of a ± 90° rotated target as quickly and accurately as possible. Replicating the classic EAB phenomenon, we found a strong impairment in target discrimination when an emotional distractor shortly preceded the target, and we also found a moderate impairment when the target preceded an emotional distractor. This decrement in accuracy at short lags was accompanied by increases in RT to the target as well as lower ratings of subjective target vividness even when the target was detected, indicating that emotional distractors impacted target processing in a lag-dependent, graded manner. We argue that these results are consistent with an interactive race model of the competition between stimulus representations in the conflict between top-down and bottom-up attentional mechanisms. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Emotional distractor images disrupt target processing in a graded manner.

[Correction Notice: An Erratum for this article was reported online in Emotion on May 16 2022 (see record 2022-64181-001). In the article "Emotional Distractor Images Disrupt Target Processing in a Graded Manner" by Jonathan M. Keefe and David H. Zald (Emotion, advance online publication, August 27, 2020, https://doi.org/10.1037/emo0000893), there were errors in the reporting of nonresponse rate and accuracy data. Nonresponse rate was underreported for the data in the Lag 2 condition, resulting in incorrect analysis of variance values for this portion of the Results section as well as incorrect Lag 2 t-test values in Table 1 and incorrect Lag 2 values in Figure 2A. Additionally, error bars for the accuracy data in Figure 2B were mistakenly calculated with data including excluded trials, resulting in larger estimates of standard error of the mean. These corrections do not affect the interpretation of any inferential statistics and in fact increased the effect of lag and distractor valence upon both of these measures. Therefore, no conclusions of the study are altered. All versions of this article have been corrected.] The emotional attentional blink (EAB), also referred to as emotion-induced blindness, refers to a transient impairment in the ability to discriminate a single target when it is presented closely in time to an emotional distractor. Although the EAB has typically been characterized as representing a complete loss of target information due to attentional capture by the emotional distractors, it is unclear whether the impact of the emotional distractor is in fact discrete or graded. Here, we tested whether the emotional distractor of the EAB interfered with target processing in a continuous or all-or-none manner by measuring changes in both reaction time (RT) and target-vividness ratings in addition to target-discrimination accuracy. Rapid sequences of landscape images were presented centrally, and participants reported the orientation of a ± 90° rotated target as quickly and accurately as possible. Replicating the classic EAB phenomenon, we found a strong impairment in target discrimination when an emotional distractor shortly preceded the target, and we also found a moderate impairment when the target preceded an emotional distractor. This decrement in accuracy at short lags was accompanied by increases in RT to the target as well as lower ratings of subjective target vividness even when the target was detected, indicating that emotional distractors impacted target processing in a lag-dependent, graded manner. We argue that these results are consistent with an interactive race model of the competition between stimulus representations in the conflict between top-down and bottom-up attentional mechanisms. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Anxiety and Attentional Processes: The Role of Resting Heart Rate Variability

Individuals with high anxiety preferentially focus attention on emotional information. The autonomic nervous system (ANS) plays an important role in modulating both anxiety and attentional processes. Despite many studies having evaluated attentional bias in anxious people, few of them have investigated the change blindness phenomenon associated with the attentional response toward salient stimuli, considering the role of the ANS. This study aimed to examine the role of heart rate variability (HRV) in trait anxiety and top-down and bottom-up attentional processes toward emotional stimuli. Seventy-five healthy university students were divided into high (N = 39) and low (N = 36) trait anxiety groups and completed a change detection flicker task with neutral, positive, and negative stimuli. The results evidenced a different attentional pattern between people with high and low anxiety considering both the two attentional processes and the valence of the stimuli. Specifically, individuals with high anxiety showed a bias in elaborating emotional stimuli related to their salience (i.e., negative stimuli were faster elaborated than neutral and positive stimuli when top-down attentional mechanisms were involved, while slower performances were highlighted considering bottom-up attentional mechanisms in response to emotional stimuli compared to neutral stimuli). Moreover, an association between HRV, trait anxiety levels, and change blindness phenomenon was confirmed. These results underline the role of HRV as a possible predictor of the alteration of attentional mechanism in anxiety.

Bottom-up sensory processing can induce negative BOLD responses and reduce functional connectivity in nodes of the default mode-like network in rats

The default mode network is a large-scale brain network that is active during rest and internally focused states and deactivates as well as desynchronizes during externally oriented (top-down) attention demanding cognitive tasks. However, it is not sufficiently understood if salient stimuli, able to trigger bottom-up attentional processes, could also result in similar reduction of activity and functional connectivity in the DMN. In this study, we investigated whether bottom-up sensory processing could influence the default mode-like network (DMLN) in rats. DMLN activity was examined using block-design visual functional magnetic resonance imaging (fMRI) while its synchronization was investigated by comparing functional connectivity during a resting versus a continuously stimulated brain state by unpredicted light flashes. We demonstrated that the BOLD response in DMLN regions was decreased during visual stimulus blocks and increased during blanks. Furthermore, decreased inter-network functional connectivity between the DMLN and visual networks as well as decreased intra-network functional connectivity within the DMLN was observed during the continuous visual stimulation. These results suggest that triggering of bottom-up attention mechanisms in sedated rats can lead to a cascade similar to top-down orienting of attention in humans and is able to deactivate and desynchronize the DMLN.

Asymmetric development of dorsal and ventral attention networks in the human brain

Two neural systems for goal-directed and stimulus-driven attention have been described in the adult human brain; the dorsal attention network (DAN) centered in the frontal eye fields (FEF) and intraparietal sulcus (IPS), and the ventral attention network (VAN) anchored in the temporoparietal junction (TPJ) and ventral frontal cortex (VFC). Little is known regarding the processes governing typical development of these attention networks in the brain. Here we use resting state functional MRI data collected from thirty 7 to 12 year-old children and thirty 18 to 31 year-old adults to examine two key regions of interest from the dorsal and ventral attention networks. We found that for the DAN nodes (IPS and FEF), children showed greater functional connectivity with regions within the network compared with adults, whereas adults showed greater functional connectivity between the FEF and extra-network regions including the posterior cingulate cortex. For the VAN nodes (TPJ and VFC), adults showed greater functional connectivity with regions within the network compared with children. Children showed greater functional connectivity between VFC and nodes of the salience network. This asymmetric pattern of development of attention networks may be a neural signature of the shift from over-representation of bottom-up attention mechanisms to greater top-down attentional capacities with development.

Effects of Bottom-Up Versus Top-Down Cueing on Conjunction Search in 3-Month-Old Infants

Previous research with infants has indicated that they are fully capable of performing a feature search in a manner nearly identical to adults (Adler & Orprecio, 2006; Fuda & Adler, 2012), but are developmentally immature in localizing a target in a conjunction search (Fuda & Adler, 2012). This might be accounted for by feature searches relying mainly on bottom-up attentional resources to localize a target, whereas conjunction searches would require top-down attentional resources in addition to bottom-up resources (Wolfe, 1994). Since infants have been shown to perform a feature search but not a conjunction search in a similar manner to that performed by adults, the current study attempted to investigate whether this might be due to bottom-up attentional mechanisms developing before top-down mechanisms. To this end, 3-month-old infants were presented with two types of cues prior to a conjunction search array that provided them with either bottom-up or top-down information that might facilitate their conjunction search. The bottom-up cue consisted of four rectangular frames indicating where the possible location of the target would be, whereas the top-down cue consisted of the brief presentation in the visual center of what the target would be. Infants’ saccade latencies were recorded for three different set sizes of (5, 8, & 10) when the target was either present or absent. Results revealed that the top-down cue, but not the bottom-up cue, facilitated infants’ exhibition of a more adult-like conjunction search function where latencies increased with increasing set sizes. These findings suggest young infants’ top-down processing mechanisms are developmentally immature relative to their bottom-up mechanisms, at least in visual search tasks. Furthermore, this study represents the first time these attentional mechanisms have been assessed in infancy and may serve as a window into understanding not only typical attentional development but also atypical development with attentional deficits. Meeting abstract presented at VSS 2013

A computational coding model for saliency detection in primary visual cortex

This study researches the coding model adaptive for information processing of the bottom-up attention mechanism. We constructed a coding model satisfying the neurobiological constraints of the primary visual cortex. By quantitatively changing the coding constraints, we carried out experiments on images used in cognitive psychology and natural image sets to compare the effects on the saliency detection performance. The experimental results statistically demonstrated that the encoding of invariant features and representation of overcomplete bases is advantageous to the bottom-up attention mechanism.

Unified framework for recognition, localization and mapping using wearable cameras

Monocular approaches to simultaneous localization and mapping (SLAM) have recently addressed with success the challenging problem of the fast computation of dense reconstructions from a single, moving camera. Thus, if these approaches initially relied on the detection of a reduced set of interest points to estimate the camera position and the map, they are currently able to reconstruct dense maps from a handheld camera while the camera coordinates are simultaneously computed. However, these maps of 3-dimensional points usually remain meaningless, that is, with no memorable items and without providing a way of encoding spatial relationships between objects and paths. In humans and mobile robotics, landmarks play a key role in the internalization of a spatial representation of an environment. They are memorable cues that can serve to define a region of the space or the location of other objects. In a topological representation of the space, landmarks can be identified and located according to its structural, perceptive or semantic significance and distinctiveness. But on the other hand, landmarks may be difficult to be located in a metric representation of the space. Restricted to the domain of visual landmarks, this work describes an approach where the map resulting from a point-based, monocular SLAM is annotated with the semantic information provided by a set of distinguished landmarks. Both features are obtained from the image. Hence, they can be linked by associating to each landmark all those point-based features that are superimposed to the landmark in a given image (key-frame). Visual landmarks will be obtained by means of an object-based, bottom-up attention mechanism, which will extract from the image a set of proto-objects. These proto-objects could not be always associated with natural objects, but they will typically constitute significant parts of these scene objects and can be appropriately annotated with semantic information. Moreover, they will be affine covariant regions, that is, they will be invariant to affine transformation, being detected under different viewing conditions (view-point angle, rotation, scale, etc.). Monocular SLAM will be solved using the accurate parallel tracking and mapping (PTAM) framework by Klein and Murray in Proceedings of IEEE/ACM international symposium on mixed and augmented reality, 2007.

Esaliency (Extended Saliency): Meaningful Attention Using Stochastic Image Modeling

Computer vision attention processes assign variable-hypothesized importance to different parts of the visual input and direct the allocation of computational resources. This nonuniform allocation might help accelerate the image analysis process. This paper proposes a new bottom-up attention mechanism. Rather than taking the traditional approach, which tries to model human attention, we propose a validated stochastic model to estimate the probability that an image part is of interest. We refer to this probability as saliency and thus specify saliency in a mathematically well-defined sense. The model quantifies several intuitive observations, such as the greater likelihood of correspondence between visually similar image regions and the likelihood that only a few of interesting objects will be present in the scene. The latter observation, which implies that such objects are (relaxed) global exceptions, replaces the traditional preference for local contrast. The algorithm starts with a rough preattentive segmentation and then uses a graphical model approximation to efficiently reveal which segments are more likely to be of interest. Experiments on natural scenes containing a variety of objects demonstrate the proposed method and show its advantages over previous approaches.