Attentional Guidance Research Articles

Multiscale Residual and Attention Guidance for Low-Light Image Enhancement in Visual SLAM

Multiscale Residual and Attention Guidance for Low-Light Image Enhancement in Visual SLAM

Spatiotemporal predictions guide attention throughout the adult lifespan

Older adults struggle with tasks requiring selective attention amidst distractions. Experimental observations about age-related decline have relied on visual search designs using static displays. However, natural environments often embed dynamic structures that afford proactive anticipation of task-relevant information. We investigate the capacity to benefit from spatiotemporal predictions across the adult lifespan. Participants (N = 300, aged 20–80) searched for multiple targets that faded in and out of displays among distractors. Half of the targets appeared at a fixed time and approximate location, whereas others appeared unpredictably. Overall search performance was reduced with age. Nevertheless, prediction-led behaviour, reflected in a higher detection of predictable targets, remained resistant to aging. Predictions were most pronounced when targets appeared in quick succession. When evaluating response speed, predictions were also significant but reduced with progressing age. While our findings confirm an age-related decline, we identified clear indications for proactive attentional guidance throughout adulthood.

Eye Movement Modelling Examples as Cues to Guide Attention and Improve Learning in Short or Long Animations

ABSTRACTBackgroundBoth eye movement modelling examples (EMMEs) and cues guide attention and improve learning in multimedia learning environments. EMME can act as a special form of cue. However, no studies have directly examined whether EMME would be superior to visual cues.ObjectiveThe study was to investigate whether there were advantages of EMME over cues on visual attention, learning outcomes and subjective perception in a multimedia environment.MethodIn Experiment 1, 67 college students were randomly assigned to the EMME group, cue group and control group and learned from animations explaining the synaptic transmission. In Experiment 2, we replicated Experiment 1 using longer learning material. Seventy‐one college students were randomly assigned to three groups (EMME, cue and control) and learned from animations explaining the process of cell division.Results and ConclusionsIn Experiment 1, results indicated both EMMEs and cues guide attention and facilitate learning. Regarding learning outcomes and subjective perception, learners did not benefit more from EMMEs than cues. Eye movement indicators showed that EMMEs were better than cues. In Experiment 2, results were consistent with Experiment 1.ImplicationsThis study is the first to compare EMMEs and cues. We found that EMMEs and cues were equivalent in producing effects on learning outcomes and subjective perception. EMMEs were superior to cues in guiding attention and conveying strategies, as indicated by eye movements. Both theoretical and practical implications could be drawn.

ScSEETV‐Net: Spatial and Channel Squeeze‐Excitation and Edge Attention Guidance V‐Shaped Network for Skin Lesion Segmentation

Early detection of skin cancer ensures the survival of many cases. There are still challenges in segmenting dermoscopic skin lesion images. Artifacts in the lesion images, such as various dirt, hairs, low contrast, and unclear boundaries, are challenges that affect segmentation accuracy. Convolutional neural networks have brought success in skin lesion segmentation. U‐shaped and V‐shaped deep learning‐based segmentation architectures learn boundary information in the first layers. However, this information becomes weaker in the following layers. Herein, the Edge‐aTtention module is added to the V‐Net architecture to move edge information to the last layer, and the spatial and channel squeeze‐excitation module is added to emphasize high‐level features by recalibrating the channel information to learn lesion boundaries better. The scSEETV‐Net is supported by fusing the binary cross‐entropy, which calculates the loss on a pixel‐based, and the focal Twersky loss function, which has significant success in class imbalances. The proposed architecture achieves 0.9212 Jaccard and 0.9552 Dice in the ISIC2016 dataset, 0.8273 Jaccard and 0.8949 Dice in the ISIC2017 dataset, and 0.8070 Jaccard and 0.8831 Dice in the ISIC2018 dataset. Comparative analyses show that the proposed methodology outperforms the state‐of‐the‐art techniques in the literature.

The capacity limitations of multiple-template visual search during task preparation and target selection.

Visual search is guided by mental representations of target-defining features (attentional templates) that are activated in a preparatory fashion. It remains unknown how many templates can be maintained concurrently, and what kind of costs are associated with multiple-template versus single-template search. Here, we compared the operation of attentional templates during three-color and single-color search tasks. Preparatory template activation processes were tracked by measuring N2pc components to task-irrelevant singleton color probes that appeared in rapid succession during the interval between search displays. These probes attract attention (as indexed by an N2pc) if the corresponding color template is active at the time when the probe appears. In a three-color search task where target identity was fully predictable (Experiment 1), only probes that matched the upcoming target color triggered N2pcs, demonstrating that only a single target template was activated. When three possible color targets appeared randomly and unpredictably (Experiment 2), probes that matched any of these colors triggered N2pcs, demonstrating that all three templates were activated concurrently. However, relative to a single-color search task, clear costs emerged in this three-color task for attentional guidance toward search targets and for search performance. These costs appear to be linked to inhibitory interactions between simultaneously active search templates. These findings show that while at least three target templates can be maintained in parallel, multiple-template search is still subject to capacity limitations which affect both template-guided attentional guidance and the subsequent selective processing of search targets.

Supporting Driver Attention Toward Potential Hazards During Takeover: A Preliminary Result

This study investigates the impact of a supportive system on takeover transitions in conditionally automated driving (SAE Level 3). The supportive system is designed to direct drivers’ attention toward potential hazards in the environment when a takeover request occurs. The study comprises two components: (a) identifying various types of potential hazards using naturalistic driving data, and (b) conducting a driving simulator study to develop and assess a gaze guidance system based on the N-SEEV model of visual attention. Results indicate that drivers using a highly salient attention guidance system were less likely to collide with a secondary hazard during takeover transitions. This suggests that gaze guidance support is an effective approach for assisting drivers during takeover transitions.

Ship detection method based on attention guidance and multi-sample decision making

Single-stage target detection methods have the characteristics of fast training speed and short detection time. However, its feature pyramid network is difficult to suppress the background and noise information of SAR ship images, and the detection head has prediction errors. To address this problem, this paper proposes a detection model based on attention guidance and multi-sample decision for synthetic aperture radar ship detection. Firstly, an attention guidance network is proposed and added to the highest level of the feature pyramid to suppress background and noise interference, thereby improving the representation ability of features. Secondly, a multi-sample decision network is proposed to participate in the prediction of target position. This network alleviates the impact of prediction errors on detection results by increasing the number of samples output in the regression branch. Finally, a novel maximum likelihood loss function is designed. This loss function constructs a maximum likelihood function using the samples output from the multi-sample decision network, which is used to standardize the training of the decision network and further improve the accuracy of target positioning. Taking the RetinaNet network model as the baseline method, compared with the baseline method and the current advanced target detection methods, this method shows the highest detection accuracy on the ship detection dataset SSDD, with AP reaching 52.8%. Compared with the baseline method, the proposed method improves the AP evaluation index 3.4% ∼ 5.7%, and the training parameter Params only increases by 2.03 M, and the frame rate FPS only decreases 0.5Iter/s.

Attentional guidance through object associations in visual cortex

Efficient behavior requires the rapid attentional selection of task-relevant objects. Preparatory activity of target-selective neurons in visual cortex is thought to support attentional selection, guiding spatial attention and favoring processing of target-matching input. However, naturalistic searches are often guided by nontargets, including target-associated “anchor” objects. For instance, when looking for a pen, we may direct our attention to the office desk on which we expect to find it. Here, using fMRI and eye tracking in a context-guided search task, we tested whether preparatory activity in visual cortex reflected the target, the guiding anchor object, or both. Participants learned associations between targets and anchors, reversing across two scene contexts, before searching for these targets. Participants’ first fixations were reliably guided by the associated anchor. Preparatory activity in lateral occipital cortex (LOC) and right intraparietal sulcus (IPS) represented the target-associated anchor rather than the target. These results shed light on the neural basis of context-guided search in structured environments.

Contextual cuing survives an interruption from an endogenous cue for attention.

Three experiments explored how the repetition of a visual search display guides search during contextual cuing under conditions in which the search process is interrupted by an instructional (endogenous) cue for attention. In Experiment 1, participants readily learned about repeated configurations of visual search, before being presented with an endogenous cue for attention towards the target on every trial. Participants used this cue to improve search times, but the repeated contexts continued to guide attention. Experiment 2 demonstrated that the presence of the endogenous cue did not impede the acquisition of contextual cuing. Experiment 3 confirmed the hypothesis that the contextual cuing effect relies largely on localized distractor contexts, following the guidance of attention. Together, the experiments point towards an interplay between two drivers of attention: after the initial guidance of attention, memory representations of the context continue to guide attention towards the target. This suggests that the early part of visual search is inconsequential for the development and maintenance of the contextual cuing effect, and that memory representations are flexibly deployed when the search procedure is dramatically interrupted.

ECG classification based on guided attention mechanism

Background and ObjectiveIntegrating domain knowledge into deep learning models can improve their effectiveness and increase explainability. This study aims to enhance the classification performance of electrocardiograms (ECGs) by customizing specific guided mechanisms based on the characteristics of different cardiac abnormalities. MethodsTwo novel guided attention mechanisms, Guided Spatial Attention (GSA) and CAM-based spatial guided attention mechanism (CGAM), were introduced. Different attention guidance labels were created based on clinical knowledge for four ECG abnormality classification tasks: ST change detection, premature contraction identification, Wolf-Parkinson-White syndrome (WPW) classification, and atrial fibrillation (AF) detection. The models were trained and evaluated separately for each classification task. Model explainability was quantified using Shapley values. ResultsGSA improved the F1 score of the model by 5.74%, 5%, 8.96%, and 3.91% for ST change detection, premature contraction identification, WPW classification, and AF detection, respectively. Similarly, CGAM exhibited improvements of 3.89%, 5.40%, 8.21%, and 1.80% for the respective tasks. The combined use of GSA and CGAM resulted in even higher improvements of 6.26%, 5.58%, 8.85%, and 4.03%, respectively. Moreover, when all four tasks were conducted simultaneously, a notable overall performance boost was achieved, demonstrating the broad adaptability of the proposed model. The quantified Shapley values demonstrated the effectiveness of the guided attention mechanisms in enhancing the model's explainability. ConclusionsThe guided attention mechanisms, utilizing domain knowledge, effectively directed the model's attention, leading to improved classification performance and explainability. These findings have significant implications in facilitating accurate automated ECG classification.

On-site burn severity assessment using smartphone-captured color burn wound images

Accurate assessment of burn severity is crucial for the management of burn injuries. Currently, clinicians mainly rely on visual inspection to assess burns, characterized by notable inter-observer discrepancies. In this study, we introduce an innovative analysis platform using color burn wound images for automatic burn severity assessment. To do this, we propose a novel joint-task deep learning model, which is capable of simultaneously segmenting both burn regions and body parts, the two crucial components in calculating the percentage of total body surface area (%TBSA). Asymmetric attention mechanism is introduced, allowing attention guidance from the body part segmentation task to the burn region segmentation task. A user-friendly mobile application is developed to facilitate a fast assessment of burn severity at clinical settings. The proposed framework was evaluated on a dataset comprising 1340 color burn wound images captured on-site at clinical settings. The average Dice coefficients for burn depth segmentation and body part segmentation are 85.12 % and 85.36 %, respectively. The R2 for %TBSA assessment is 0.9136. The source codes for the joint-task framework and the application are released on Github (https://github.com/xjtu-mia/BurnAnalysis). The proposed platform holds the potential to be widely used at clinical settings to facilitate a fast and precise burn assessment.

Referring Image Segmentation With Fine-Grained Semantic Funneling Infusion.

Recently, referring image segmentation has attracted wide attention given its huge potential in human-robot interaction. Networks to identify the referred region must have a deep understanding of both the image and language semantics. To do so, existing works tend to design various mechanisms to achieve cross-modality fusion, for example, tile and concatenation and vanilla nonlocal manipulation. However, the plain fusion usually is either coarse or constrained by the exorbitant computation overhead, finally causing not enough understanding of the referent. In this work, we propose a fine-grained semantic funneling infusion (FSFI) mechanism to solve the problem. The FSFI introduces a constant spatial constraint on the querying entities from different encoding stages and dynamically infuses the gleaned language semantic into the vision branch. Moreover, it decomposes the features from different modalities into more delicate components, allowing the fusion to happen in multiple low-dimensional spaces. The fusion is more effective than the one only happening in one high-dimensional space, given its ability to sink more representative information along the channel dimension. Another problem haunting the task is that the instilling of high-abstract semantic will blur the details of the referent. Targetedly, we propose a multiscale attention-enhanced decoder (MAED) to alleviate the problem. We design a detail enhancement operator (DeEh) and apply it in a multiscale and progressive way. Features from the higher level are used to generate attention guidance to enlighten the lower-level features to more attend to the detail regions. Extensive results on the challenging benchmarks show that our network performs favorably against the state-of-the-arts (SOTAs).

Spatial Predictive Context Speeds Up Visual Search by Biasing Local Attentional Competition.

The human visual system is equipped to rapidly and implicitly learn and exploit the statistical regularities in our environment. Within visual search, contextual cueing demonstrates how implicit knowledge of scenes can improve search performance. This is commonly interpreted as spatial context in the scenes becoming predictive of the target location, which leads to a more efficient guidance of attention during search. However, what drives this enhanced guidance is unknown. First, it is under debate whether the entire scene (global context) or more local context drives this phenomenon. Second, it is unclear how exactly improved attentional guidance is enabled by target enhancement and distractor suppression. In the present magnetoencephalography experiment, we leveraged rapid invisible frequency tagging to answer these two outstanding questions. We found that the improved performance when searching implicitly familiar scenes was accompanied by a stronger neural representation of the target stimulus, at the cost specifically of those distractors directly surrounding the target. Crucially, this biasing of local attentional competition was behaviorally relevant when searching familiar scenes. Taken together, we conclude that implicitly learned spatial predictive context improves how we search our environment by sharpening the attentional field.

Chunking in Visual Working Memory changes the Guidance of Attention in a Visual Search

Chunking in Visual Working Memory changes the Guidance of Attention in a Visual Search

Looking for the red shirt: Meaningful objects strengthen memory and attentional guidance

Looking for the red shirt: Meaningful objects strengthen memory and attentional guidance

The role of object co-occurrence in attentional guidance: evidence from eye-movements

The role of object co-occurrence in attentional guidance: evidence from eye-movements

Strategic Attention Guidance: The Impact of Dual and Single Cues in Wide Field of View Searches

Strategic Attention Guidance: The Impact of Dual and Single Cues in Wide Field of View Searches

A multimodal dual-branch fusion network for fetal hypoxia detection

Labor is the most severe test of a fetus’s ability to survive hypoxia. Even low-risk full-term fetuses may endure variable degrees of hypoxia as a result of intrapartum uterine contractions. It is imperative for clinicians to expeditiously identify fetal hypoxia and implement timely interventions. However, due to the intricacy and intra-class variability of fetal heart rate (FHR) data, distinguishing normal babies from acidotic fetuses is extremely challenging. This research proposes a novel multimodal dual-branch fusion network to improve the accuracy of fetal hypoxia identification. The dual-branch network is constructed through the signal slicing method. Additionally, we present a novel attention guidance module that leverages spatial attention to capture hypoxia-related information from two branches of signal. Ultimately, the network integrates maternal electronic medical records, FHR features, and FHR signals to achieve multimodal fusion of the input features. Furthermore, label smoothing can yield better calibrated networks, thus improving classification performance even further. On the public dataset, this method obtained a sensitivity of 72.58 %, a specificity of 71.08 %, a quality index of 71.59 %, and an area under the curve of 74.70 %. This method combines maternal-fetal medicine and artificial intelligence, represents a new strategy to recognize fetal acidosis.

Polarized reflection removal with dual-stream attention guidance

Addressing the challenge of removing reflections from images captured through glass surfaces holds significant importance across various practical applications. Existing methods often fall short due to reliance on intermediate predictions or special constraints, leading to undesirable artifacts. This paper addresses this issue by recognizing the inherent correlation between background and reflection. Leveraging the distinct optical properties of polarized images, we propose a novel dual-stream network with feature attention guidance for reflection removal. The model, taking multi-channel polarized images as input, exploits the optical characteristics of transmitted and reflected light to tackle the ill-posed problem. Our dual-stream structure predicts both reflectance and transmission images simultaneously, capitalizing on the unique relationship between these two elements. Additionally, we introduce the innovative differential feature fusion (DFF) block to enhance communication between the two streams. Our model shows superior performance to the state-of-the-art methods when tested on real-world polarization datasets.

A mixed Mamba U-net for prostate segmentation in MR images

The diagnosis of early prostate cancer depends on the accurate segmentation of prostate regions in magnetic resonance imaging (MRI). However, this segmentation task is challenging due to the particularities of prostate MR images themselves and the limitations of existing methods. To address these issues, we propose a U-shaped encoder-decoder network MM-UNet based on Mamba and CNN for prostate segmentation in MR images. Specifically, we first proposed an adaptive feature fusion module based on channel attention guidance to achieve effective fusion between adjacent hierarchical features and suppress the interference of background noise. Secondly, we propose a global context-aware module based on Mamba, which has strong long-range modeling capabilities and linear complexity, to capture global context information in images. Finally, we propose a multi-scale anisotropic convolution module based on the principle of parallel multi-scale anisotropic convolution blocks and 3D convolution decomposition. Experimental results on two public prostate MR image segmentation datasets demonstrate that the proposed method outperforms competing models in terms of prostate segmentation performance and achieves state-of-the-art performance. In future work, we intend to enhance the model's robustness and extend its applicability to additional medical image segmentation tasks.