Bottom-up Attention Research Articles

Top-down and bottom-up attentional multiple instance learning for still image action recognition

We propose an end-to-end, top-down and bottom-up attentional deep multiple instance learning approach for still image action recognition. Our approach does not rely on neither any attribute nor object labels and/or explicitly trained object detectors. Instead, our approach simultaneously proposes image regions, chooses the action related regions with a top-down attention mechanism, weights each image region with a bottom-up attention layer to mask unrelated pixels to yield a fine-grained pixel-level action related map. Then, it embeds the selected regions into a single image-level feature descriptor and assigns an action label. We evaluate our approach on four benchmark still image action datasets: Stanford40 (Yao et al., 2011), Pascal VOC 2012 (Everingham et al., 2012) MPII (Andriluka et al., 2014) and VCOCO (Gupta and Malik, 2015). Our results demonstrate that, while increasing the overall recognition performance, our framework successfully selects action related image regions and creates pixel grade action masks.

Value-driven effects on perceptual averaging

Perceptual averaging refers to a strategy of encoding the statistical properties of entire sets of objects rather than encoding individual object properties, potentially circumventing the visual system’s strict capacity limitations. Prior work has shown that such average representations of set properties, such as its mean size, can be modulated by top-down and bottom-up attention. However, it is unclear to what extent attentional biases through selection history, in the form of value-driven attentional capture, influences this type of summary statistical representation. To investigate, we conducted two experiments in which participants estimated the mean size of a set of heterogeneously sized circles while a previously rewarded color singleton was part of the set. In Experiment 1, all circles were gray, except either the smallest or the largest circle, which was presented in a color previously associated with a reward. When the largest circle in the set was associated with the highest value (as a proxy of selection history), we observed the largest biases, such that perceived mean size scaled linearly with the increasing value of the attended color singleton. In Experiment 2, we introduced a dual-task component in the form of an attentional search task to ensure that the observed bias of reward on perceptual averaging was not fully explained by focusing attention solely on the reward-signaling color singleton. Collectively, findings support the proposal that selection history, like bottom-up and top-down attention, influences perceptual averaging, and that this happens in a flexible manner proportional to the extent to which attention is captured.

Weak task synchronization of default mode network in task based paradigms

The brain's Default mode network (DMN) is generally characterized by brain areas that gets deactivated upon the presentation of a wide variety of externally focused, attention demanding tasks. These areas also exhibit significant intra-DMN functional connectivity and significant negative functional connectivity with other brain areas, especially with attention networks, in both resting state and task conditions. Therefore, the DMN has been hypothesized to be involved in internally directed cognitive activities such as autobiographical recall of the past, future planning and mind wandering. Recent research has discovered the role of bottom-up attention in modulating the behaviour of DMN. We hypothesize that the de-engagement of the DMN regions upon the presentation of an externally-focused attention-demanding stimulus may not be strictly stimulus locked and may exhibit significant trial-to-trial as well as subject-to-subject variability. Due to the involvement of frontoparietal control network in modulating the anticorrelations between DMN and dorsal attention network (DAN), we expect the DMN regions to have lower inter-trial and inter-subject synchronization in their fMRI BOLD responses as compared to the bottom-up early-sensory task-positive regions. To test this hypothesis, we designed new statistical methods called Inter Trial Temporal Synchronization Analysis (IT-TSA) and Inter Subject TSA (IS-TSA) to analyse variability across trials and subjects respectively. We analysed four publicly available datasets (total 223 subjects) across seven tasks related to different cognitive modalities and found out that there is significantly low stimulus-locked synchronization across trials and subjects in the DMN regions as compared to early sensory task positive regions. Our study challenges the understanding of DMN as a strictly task-negative region and supports the recent findings that DMN acts as an active component associated with intrinsic processing which deactivates differentially and non-linearly across trials and subjects in the presence of extrinsic processes.

Gender moderates the association between chronic academic stress with top-down and bottom-up attention

Research on the relationship between chronic stress and cognition is limited by a lack of concurrent measurement of state-anxiety, physiological arousal, and gender. For the first time, we assessed the impact of these factors on top-down/conscious (simple and choice reaction time) and bottom-up/reflexive (saccadic reaction time) measures of attention using CONVIRT virtual-reality cognitive tests. Participants (N = 163) completed measures of academic stress (effort-reward imbalance; ERI) and state-anxiety while heart-rate variability was recorded continuously throughout the experiment. Gender moderated the association between academic stress with the top-down measures (b = -0.002, t = -2.023, p = .045; b = -0.063, t = -3.080, p = .002) and higher academic stress was associated with poorer/slower reaction times only for male participants. For bottom-up attention, heart rate variability moderated the relationship between academic stress and saccadic reaction time (b = 0.092, t = 1.991, p = .048), and only female participants who were more stressed (i.e., ERI ≥ 1) and displayed stronger sympathetic dominance had slower reaction times. Our findings align with emerging evidence that chronic stress is related to hyperarousal in women and cognitive decrements in men. Our findings suggest that higher ERI and sympathetic dominance during cognitive testing was associated with poorer bottom-up attention in women, whereas for men, academic stress was related with poorer top-down attention irrespective of sympathovagal balance.

Sound gaps boost emotional audiovisual integration independent of attention: Evidence from an ERP study

The emotion discrimination paradigm was adopted to study the effect of interrupted sound on visual emotional processing under different attentional states. There were two experiments: Experiment 1: judging facial expressions (explicit task), Experiment 2: judging the position of a bar (implicit task). In Experiment 1, ERP results showed that there was a sound gap accelerating the effect of P1 present only under neutral faces. In Experiment 2, the accelerating effect (P1) existed regardless of the emotional condition. Combining two experiments, P1 findings suggest that sound gap enhances bottom-up attention. The N170 and late positive component (LPC) were found to be regulated by emotion face in both experiments, with fear over the neutral. Comparing the two experiments, the explicit task induced a larger LPC than the implicit task. Overall, sound gaps boosted the audiovisual integration by bottom-up attention in early integration, while cognitive expectations led to top-down attention in late stages.

Neural attention for image captioning: review of outstanding methods

Image captioning is the task of automatically generating sentences that describe an input image in the best way possible. The most successful techniques for automatically generating image captions have recently used attentive deep learning models. There are variations in the way deep learning models with attention are designed. In this survey, we provide a review of literature related to attentive deep learning models for image captioning. Instead of offering a comprehensive review of all prior work on deep image captioning models, we explain various types of attention mechanisms used for the task of image captioning in deep learning models. The most successful deep learning models used for image captioning follow the encoder-decoder architecture, although there are differences in the way these models employ attention mechanisms. Via analysis on performance results from different attentive deep models for image captioning, we aim at finding the most successful types of attention mechanisms in deep models for image captioning. Soft attention, bottom-up attention, and multi-head attention are the types of attention mechanism widely used in state-of-the-art attentive deep learning models for image captioning. At the current time, the best results are achieved from variants of multi-head attention with bottom-up attention.

Attention and behaviour on fashion retail websites: an eye-tracking study

PurposeThe purpose of this paper is to identify attention, cognitive and affective responses towards a fashion retailer's website and the behavioural outcomes when shopping online.Design/methodology/approach52 eye-tracking tests and 52 qualitative semi-structured interviews were conducted.FindingsConsumer attention and behaviour differ across web pages throughout the shopping journey depending on its content, function and consumers' goal. Top-down attention is more dominant than bottom-up attention when consumers are shopping online for fashion items. The product listings page was the most frequented and had the most time spent on it. Consumers enjoy browsing for products and adding them to their basket to evaluate them together later. Customisation and personalisation features are the most valued due to their ability to make the experience more convenient and enjoyable.Originality/valueThis article contributes novel findings that the content and design of the website affects attention in different ways. It demonstrates that research cannot simplify viewing patterns for fashion shopping online. The study extends the SOR framework, showing that top-down attention, when provided with personalisation and customisation features, results in approach behaviour. A lack of personalisation or customisation features results in avoidance behaviour. The complex nature of consumer attention and behaviour during their holistic shopping journey advocates the need for eye-tracking research to be conducted on a live website for ecological validity, providing a methodological contribution which can be used for future research.

Awareness-Dependent Normalization Framework of Visual Bottom-up Attention.

Although bottom-up attention can improve visual performance with and without awareness to the exogenous cue, whether they are governed by a common neural computation remains unclear. Using a modified Posner paradigm with backward masking, we found that the cueing effect displayed a monotonic gradient profile (Gaussian-like), both with and without awareness, whose scope, however, was significantly wider with than without awareness. This awareness-dependent scope offered us a unique opportunity to change the relative size of the attention field to the stimulus, differentially modulating the gain of attentional selection, as proposed by the normalization model of attention. Therefore, for each human subject (male and female), the stimulus size was manipulated as their respective mean attention fields with and without awareness while stimulus contrast was varied in a spatial cueing task. By measuring the gain pattern of contrast-response functions on the spatial cueing effect derived by visible or invisible cues, we observed changes in the cueing effect consonant with changes in contrast gain for visible cues and response gain for invisible cues. Importantly, a complementary analysis confirmed that subjects' awareness-dependent attention fields can be simulated by using the normalization model of attention. Together, our findings indicate an awareness-dependent normalization framework of visual bottom-up attention, placing a necessary constraint, namely, awareness, on our understanding of the neural computations underlying visual attention.SIGNIFICANCE STATEMENT Bottom-up attention is known to improve visual performance with and without awareness. We discovered that manipulating subjects' awareness can modulate their attention fields of visual bottom-up attention, which offers a unique opportunity to regulate its normalization processes. On the one hand, by measuring the gain pattern of contrast-response functions on the spatial cueing effect derived by visible or invisible cues, we observed changes in the cueing effect consonant with changes in contrast gain for visible cues and response gain for invisible cues. On the other hand, by using the normalization model of attention, subjects' awareness-dependent attention fields can be simulated successfully. Our study supports important predictions of the normalization model of visual bottom-up attention and further reveals its dependence on awareness.

Integrating top-down and bottom-up attention control factors: an EEG study

Integrating top-down and bottom-up attention control factors: an EEG study

How top-down and bottom-up attention modulate risky choice

We examine how bottom-up (or stimulus-driven) and top-down (or goal-driven) processes govern the distribution of attention in risky choice. In three experiments, participants chose between a certain payoff and the chance of receiving a payoff drawn randomly from an array of eight numbers. We tested the hypothesis that initial attention is driven by perceptual properties of the stimulus (e.g., font size of the numbers), but subsequent choice is goal-driven (e.g., win the best outcome). Two experiments in which task framing (goal driven) and font size (stimulus driven) were manipulated demonstrated that payoffs with the highest values and the largest font sizes had the greatest impact on choice. The third experiment added a number in large font to the array, which could not be an outcome of the gamble (i.e., a distractor). Eye movement and choice data indicated that although the distractor attracted attention, it had no influence on option selection. Together with computational modeling analyses, the results suggest that perceptual salience can induce bottom-up effects of overt selection but that the perceived value of information is the crucial arbiter of intentional control over risky choice.

Functional Differentiation of Dorsal and Ventral Posterior Parietal Cortex of the Rat: Implications for Controlled and Stimulus-Driven Attention.

The posterior parietal cortex (PPC) is important for visuospatial attention. The primate PPC shows functional differentiation such that dorsal areas are implicated in top-down, controlled attention, and ventral areas are implicated in bottom-up, stimulus-driven attention. Whether the rat PPC also shows such functional differentiation is unknown. Here, we address this open question using functional neuroanatomy and in vivo electrophysiology. Using conventional tract-tracing methods, we examined connectivity with other structures implicated in visuospatial attention including the lateral posterior nucleus of the thalamus (LPn) and the postrhinal cortex (POR). We showed that the LPn projects to the entire PPC, preferentially targeting more ventral areas. All parts of the PPC and POR are reciprocally connected with the strongest connections evident between ventral PPC and caudal POR. Next, we simultaneously recorded neuronal activity in dorsal and ventral PPC as rats performed a visuospatial attention (VSA ) task that engages in both bottom-up and top-down attention. Previously, we provided evidence that the dorsal PPC is engaged in multiple cognitive process including controlled attention (Yang etal. 2017). Here, we further showed that ventral PPC cells respond to stimulus onset more rapidly than dorsal PPC cells, providing evidence for a role in stimulus-driven, bottom-up attention.

Behavioral evidence for attention selection as entrained synchronization without awareness.

Animal physiological and human neuroimaging studies have established a link between attention and γ-band (30-90 Hz) oscillations and synchronizations. However, a behavioral link between entrained γ-band oscillations and attention has been fraught with technical challenges. In particular, while entrainment at mid-γ band (40-70 Hz) has been claimed to be privileged in evoking attentional modulations without awareness, the effect may be attributed to display artifacts. Here, by exploiting isoluminant chromatic flicker without luminance modulation and not subject to these artifacts, we tested attentional attraction by chromatic flicker too fast to perceive. Awareness of flicker was subjectively and objectively tested with a high-powered design and evaluated with traditional and Bayesian statistics. Across 2 experiments in human participants, we observed-and also replicated-that 30-Hz chromatic flicker outside mid-γ band attracted attention, resulting in a facilitation effect at a 50 ms interstimulus interval (ISI) and an inhibition effect at a 500 ms ISI. The attention test was confirmed to be more sensitive to the cue than the direct cue-localization task was. We further showed that these attention effects were absent for 50-Hz chromatic flicker. These results provide strong direct evidence against a privileged role of mid-γ band in unconscious attention, but are consistent with known cortical responses to chromatic flicker in early visual cortex. Taken together, our findings provide behavioral evidence that entrained synchronization may serve as a mechanism for bottom-up attention selection and that chromatic flicker may offer a fruitful avenue for investigating unconscious processing. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Distractibility When Suffering from Mental Fatigue after a Mild Traumatic Brain Injury

Most Traumatic Brain Injuries are mild (mTBI) yet many people suffer from long-term mental fatigue and cognitive impairment. Despite comments from patients, cognitive difficulties can go undetected. Distractibility is commonly reported but is seldom included in standard neuropsychological assessment. This study was designed to investigate the effect distraction may induce in topdown and bottom-up attention among people who suffer from mental fatigue after mTBI. Thirty mTBI patients suffering from mental fatigue and 30 healthy controls performed a computerized test, including Simple Reaction Time, Choice Reaction Time and Attentional Capture tasks with a salient distractor. A slower processing speed was found in all subtests for the mTBI group and was particularly noticeable for the decision-making task. The distraction stimulus reduced processing speed for both groups, while the mTBI group made more omissions when a distractor emerged, indicating increased distractibility. However, no effect in top-down and bottom-up attention was found. Response time in the presence of a distractor was a predictor for mental fatigue, while depression and anxiety were not, showing the importance to carefully distinguish between emotional distress and mental fatigue. In conclusion, it is suggested that people suffering from mental fatigue after mTBI are slower at processing information, and this is more pronounced when a cognitive demand is added to the task. Distractibility was indicated with more omissions during distraction, but a distinction between top-down and bottom-up systems was not found. Further research is needed to better understand the link between distractibility and mental fatigue after a brain injury.

Isoflurane effects on the N1 and other long-latency auditory evoked potentials in Wistar rats

Long-latency auditory evoked potentials (LLAEPs) may help further advances in understanding consciousness under general anesthesia and promote more objective means of assessing sedation depth than conventional clinical signs. Among the LLAEP components, the auditory N1 shows promise as a measure of sedation depth and a marker of consciousness, but findings are so far inconclusive. Research with animals can help elucidate the effects of various anesthetics on the N1 and other LLAEPs, but investigations of LLAEPs under anesthesia in animals is lacking. To address this deficit, we examined the P1, N1, P2, and N2, along with their corresponding peak-to-peak complexes, in 10 Wistar rats anesthetized with 1.5–2 % isoflurane in pure oxygen and again after recovery. While under anesthesia, subdermal needle electrodes were inserted and secured for electroencephalographic (EEG) recordings. LLAEPs were assessed during a 20-min, passive, two-tone (500 ms inter-tone interval) paradigm with randomized short (1 s) and long (5 s) inter-pair intervals (IPIs). Overall, while the LLAEP peaks under isoflurane were less defined, they were not eliminated. The peak-to-peak amplitudes, particularly the P1-N1, were significantly smaller under isoflurane than during post-recovery. Our preliminary findings indicate that isoflurane produces global suppression across LLAEP components, presumably reflecting impaired integration of top-down and bottom-up attention and sensory systems under profound sedation with isoflurane.

Neuroimaging Techniques in Advertising Research: Main Applications, Development, and Brain Regions and Processes

Despite the advancement in neuroimaging tools, studies about using neuroimaging tools to study the impact of advertising on brain regions and processes are scant and remain unclear in academic literature. In this article, we have followed a literature review methodology and a bibliometric analysis to select empirical and review papers that employed neuroimaging tools in advertising campaigns and to understand the global research trends in the neuromarketing domain. We extracted and analyzed sixty-three articles from the Web of Science database to answer our study questions. We found four common neuroimaging techniques employed in advertising research. We also found that the orbitofrontal cortex (OFC), the ventromedial prefrontal cortex, and the dorsolateral prefrontal cortex play a vital role in decision-making processes. The OFC is linked to positive valence, and the lateral OFC and left dorsal anterior insula related in negative valence. In addition, the thalamus and primary visual area associated with the bottom-up attention system, whereas the top-down attention system connected to the dorsolateral prefrontal cortex, parietal cortex, and primary visual areas. For memory, the hippocampus is responsible for generating and processing memories. We hope that this study provides valuable insights about the main brain regions and processes of interest for advertising.

COCO-Search18 fixation dataset for predicting goal-directed attention control

Attention control is a basic behavioral process that has been studied for decades. The currently best models of attention control are deep networks trained on free-viewing behavior to predict bottom-up attention control – saliency. We introduce COCO-Search18, the first dataset of laboratory-quality goal-directed behavior large enough to train deep-network models. We collected eye-movement behavior from 10 people searching for each of 18 target-object categories in 6202 natural-scene images, yielding sim 300,000 search fixations. We thoroughly characterize COCO-Search18, and benchmark it using three machine-learning methods: a ResNet50 object detector, a ResNet50 trained on fixation-density maps, and an inverse-reinforcement-learning model trained on behavioral search scanpaths. Models were also trained/tested on images transformed to approximate a foveated retina, a fundamental biological constraint. These models, each having a different reliance on behavioral training, collectively comprise the new state-of-the-art in predicting goal-directed search fixations. Our expectation is that future work using COCO-Search18 will far surpass these initial efforts, finding applications in domains ranging from human-computer interactive systems that can anticipate a person’s intent and render assistance to the potentially early identification of attention-related clinical disorders (ADHD, PTSD, phobia) based on deviation from neurotypical fixation behavior.

The Modulating Effect of Top-down Attention on the Optimal Pre-target Onset Oscillatory States of Bottom-up Attention

Research in both bottom-up and top-down attention has shown that behavioural performance is related to brain oscillations at the time of stimulus presentation: the angle of the theta phase in bottom-up attention and the inhibition of alpha oscillations in top-down attention. However, whether the conditions most favourable for bottom-up attention change with the addition of top-down cues is unclear. To explore the characteristics of favourable oscillations during bottom-up processing, in experiment 1, 36 participants completed a selective attention task (visual search) without cues. Then, in experiment 2, we examined whether favourable oscillatory characteristics were changed by top-down attentional cues; in this experiment, 62 subjects were asked to perform an attention network task. We found that without anticipation, oscillatory states that were associated with better performance were characterized by lower theta power in the frontal area, higher alpha power in the occipital area, higher beta power in the frontal area, and weaker gamma-theta amplitude-envelope coupling in the parietal area. However, some characteristics that were associated with better performance, including theta power and low beta power, were changed after the addition of different cues. In addition, there were some new characteristics related to improved performance under temporal and spatial anticipation. These results suggest that top-down attention implements a more energy-efficient strategy to process information, optimizing the process of bottom-up attention.

A bottom-up and top-down human visual attention approach for hyperspectral anomaly detection

Hyperspectral anomaly detection (HAD) is a branch of target detection which tries to locate pixels that are spectrally or spatially different from their background. In this paper, a visual attention approach is developed to leverage HAD. Traditional HAD methods often try to locate anomalous pixels based on spectral information. However, the spatial features of hyperspectral datasets provide valuable information. Here, we aim to fuse spatial and spectral anomaly features based on bottom-up (BU) and top-down (TD) visual attention mechanisms. Owe to the BU attention, spatial features are extracted by mimicking the primary visual cortex neurons functionality. Also, spectral information is obtained throughout a deep neural network that imitating the TD visual attention. The BU and TD approaches’ results are then integrated to provide both spectral and spatial information. The key findings of our results demonstrate the proposed method outperforms the six state-of-the-art AD methods based on different evaluation metrics.

Dual Attention on Pyramid Feature Maps for Image Captioning

Generating natural sentences from images is a fundamental learning task for visual-semantic understanding in multimedia. In this paper, we propose to apply dual attention on pyramid image feature maps to fully explore the visual-semantic correlations and improve the quality of generated sentences. Specifically, with the full consideration of the contextual information provided by the hidden state of the RNN controller, the pyramid attention can better localize the visually indicative and semantically consistent regions in images. On the other hand, the contextual information can help re-calibrate the importance of feature components by learning the channel-wise dependencies, to improve the discriminative power of visual features for better content description. We conducted comprehensive experiments on three well-known datasets: Flickr8K, Flickr30 K and MS COCO, which achieved impressive results in generating descriptive and smooth natural sentences from images. Using either convolution visual features or more informative bottom-up attention features, the composite model can boost the performance of image-to-sentence translation, with a limited computational resource overhead. The proposed pyramid attention and dual attention methods are highly modular, which can be inserted into various image captioning modules to further improve the performance.

Nontarget emotional stimuli must be highly conspicuous to modulate the attentional blink.

The attentional blink (AB) is often considered a top-down phenomenon because it is triggered by matching an initial target (T1) in a rapid serial visual presentation (RSVP) stream to a search template. However, the AB is modulated when targets are emotional, and is evoked when a task-irrelevant, emotional critical distractor (CDI) replaces T1. Neither manipulation fully captures the interplay between bottom-up and top-down attention in the AB: Valenced targets intrinsically conflate top-down and bottom-up attention. The CDI approach cannot manipulate second target (T2) valence, which is critical because valenced T2s can “break through” the AB (in the target-manipulation approach). The present research resolves this methodological challenge by indirectly measuring whether a purely bottom-up CDI can modulate report of a subsequent T2. This novel approach adds a valenced CDI to the “classic,” two-target AB. Participants viewed RSVP streams containing a T1–CDI pair preceding a variable lag to T2. If the CDI’s valence is sufficient to survive the AB, it should modulate T2 performance, indirectly signaling bottom-up capture by an emotional stimulus. Contrary to this prediction, CDI valence only affected the AB when CDIs were also extremely visually conspicuous. Thus, emotional valence alone is insufficient to modulate the AB.