Attentional Guidance Research Articles

Supporting Driver Attention Toward Potential Hazards During Takeover: A Preliminary Result

This study investigates the impact of a supportive system on takeover transitions in conditionally automated driving (SAE Level 3). The supportive system is designed to direct drivers’ attention toward potential hazards in the environment when a takeover request occurs. The study comprises two components: (a) identifying various types of potential hazards using naturalistic driving data, and (b) conducting a driving simulator study to develop and assess a gaze guidance system based on the N-SEEV model of visual attention. Results indicate that drivers using a highly salient attention guidance system were less likely to collide with a secondary hazard during takeover transitions. This suggests that gaze guidance support is an effective approach for assisting drivers during takeover transitions.

Ship detection method based on attention guidance and multi-sample decision making

Single-stage target detection methods have the characteristics of fast training speed and short detection time. However, its feature pyramid network is difficult to suppress the background and noise information of SAR ship images, and the detection head has prediction errors. To address this problem, this paper proposes a detection model based on attention guidance and multi-sample decision for synthetic aperture radar ship detection. Firstly, an attention guidance network is proposed and added to the highest level of the feature pyramid to suppress background and noise interference, thereby improving the representation ability of features. Secondly, a multi-sample decision network is proposed to participate in the prediction of target position. This network alleviates the impact of prediction errors on detection results by increasing the number of samples output in the regression branch. Finally, a novel maximum likelihood loss function is designed. This loss function constructs a maximum likelihood function using the samples output from the multi-sample decision network, which is used to standardize the training of the decision network and further improve the accuracy of target positioning. Taking the RetinaNet network model as the baseline method, compared with the baseline method and the current advanced target detection methods, this method shows the highest detection accuracy on the ship detection dataset SSDD, with AP reaching 52.8%. Compared with the baseline method, the proposed method improves the AP evaluation index 3.4% ∼ 5.7%, and the training parameter Params only increases by 2.03 M, and the frame rate FPS only decreases 0.5Iter/s.

Attentional guidance through object associations in visual cortex

Efficient behavior requires the rapid attentional selection of task-relevant objects. Preparatory activity of target-selective neurons in visual cortex is thought to support attentional selection, guiding spatial attention and favoring processing of target-matching input. However, naturalistic searches are often guided by nontargets, including target-associated “anchor” objects. For instance, when looking for a pen, we may direct our attention to the office desk on which we expect to find it. Here, using fMRI and eye tracking in a context-guided search task, we tested whether preparatory activity in visual cortex reflected the target, the guiding anchor object, or both. Participants learned associations between targets and anchors, reversing across two scene contexts, before searching for these targets. Participants’ first fixations were reliably guided by the associated anchor. Preparatory activity in lateral occipital cortex (LOC) and right intraparietal sulcus (IPS) represented the target-associated anchor rather than the target. These results shed light on the neural basis of context-guided search in structured environments.

Contextual cuing survives an interruption from an endogenous cue for attention.

Three experiments explored how the repetition of a visual search display guides search during contextual cuing under conditions in which the search process is interrupted by an instructional (endogenous) cue for attention. In Experiment 1, participants readily learned about repeated configurations of visual search, before being presented with an endogenous cue for attention towards the target on every trial. Participants used this cue to improve search times, but the repeated contexts continued to guide attention. Experiment 2 demonstrated that the presence of the endogenous cue did not impede the acquisition of contextual cuing. Experiment 3 confirmed the hypothesis that the contextual cuing effect relies largely on localized distractor contexts, following the guidance of attention. Together, the experiments point towards an interplay between two drivers of attention: after the initial guidance of attention, memory representations of the context continue to guide attention towards the target. This suggests that the early part of visual search is inconsequential for the development and maintenance of the contextual cuing effect, and that memory representations are flexibly deployed when the search procedure is dramatically interrupted.

ECG classification based on guided attention mechanism

Background and ObjectiveIntegrating domain knowledge into deep learning models can improve their effectiveness and increase explainability. This study aims to enhance the classification performance of electrocardiograms (ECGs) by customizing specific guided mechanisms based on the characteristics of different cardiac abnormalities. MethodsTwo novel guided attention mechanisms, Guided Spatial Attention (GSA) and CAM-based spatial guided attention mechanism (CGAM), were introduced. Different attention guidance labels were created based on clinical knowledge for four ECG abnormality classification tasks: ST change detection, premature contraction identification, Wolf-Parkinson-White syndrome (WPW) classification, and atrial fibrillation (AF) detection. The models were trained and evaluated separately for each classification task. Model explainability was quantified using Shapley values. ResultsGSA improved the F1 score of the model by 5.74%, 5%, 8.96%, and 3.91% for ST change detection, premature contraction identification, WPW classification, and AF detection, respectively. Similarly, CGAM exhibited improvements of 3.89%, 5.40%, 8.21%, and 1.80% for the respective tasks. The combined use of GSA and CGAM resulted in even higher improvements of 6.26%, 5.58%, 8.85%, and 4.03%, respectively. Moreover, when all four tasks were conducted simultaneously, a notable overall performance boost was achieved, demonstrating the broad adaptability of the proposed model. The quantified Shapley values demonstrated the effectiveness of the guided attention mechanisms in enhancing the model's explainability. ConclusionsThe guided attention mechanisms, utilizing domain knowledge, effectively directed the model's attention, leading to improved classification performance and explainability. These findings have significant implications in facilitating accurate automated ECG classification.

On-site burn severity assessment using smartphone-captured color burn wound images

Accurate assessment of burn severity is crucial for the management of burn injuries. Currently, clinicians mainly rely on visual inspection to assess burns, characterized by notable inter-observer discrepancies. In this study, we introduce an innovative analysis platform using color burn wound images for automatic burn severity assessment. To do this, we propose a novel joint-task deep learning model, which is capable of simultaneously segmenting both burn regions and body parts, the two crucial components in calculating the percentage of total body surface area (%TBSA). Asymmetric attention mechanism is introduced, allowing attention guidance from the body part segmentation task to the burn region segmentation task. A user-friendly mobile application is developed to facilitate a fast assessment of burn severity at clinical settings. The proposed framework was evaluated on a dataset comprising 1340 color burn wound images captured on-site at clinical settings. The average Dice coefficients for burn depth segmentation and body part segmentation are 85.12 % and 85.36 %, respectively. The R2 for %TBSA assessment is 0.9136. The source codes for the joint-task framework and the application are released on Github (https://github.com/xjtu-mia/BurnAnalysis). The proposed platform holds the potential to be widely used at clinical settings to facilitate a fast and precise burn assessment.

Referring Image Segmentation With Fine-Grained Semantic Funneling Infusion.

Recently, referring image segmentation has attracted wide attention given its huge potential in human-robot interaction. Networks to identify the referred region must have a deep understanding of both the image and language semantics. To do so, existing works tend to design various mechanisms to achieve cross-modality fusion, for example, tile and concatenation and vanilla nonlocal manipulation. However, the plain fusion usually is either coarse or constrained by the exorbitant computation overhead, finally causing not enough understanding of the referent. In this work, we propose a fine-grained semantic funneling infusion (FSFI) mechanism to solve the problem. The FSFI introduces a constant spatial constraint on the querying entities from different encoding stages and dynamically infuses the gleaned language semantic into the vision branch. Moreover, it decomposes the features from different modalities into more delicate components, allowing the fusion to happen in multiple low-dimensional spaces. The fusion is more effective than the one only happening in one high-dimensional space, given its ability to sink more representative information along the channel dimension. Another problem haunting the task is that the instilling of high-abstract semantic will blur the details of the referent. Targetedly, we propose a multiscale attention-enhanced decoder (MAED) to alleviate the problem. We design a detail enhancement operator (DeEh) and apply it in a multiscale and progressive way. Features from the higher level are used to generate attention guidance to enlighten the lower-level features to more attend to the detail regions. Extensive results on the challenging benchmarks show that our network performs favorably against the state-of-the-arts (SOTAs).

Spatial Predictive Context Speeds Up Visual Search by Biasing Local Attentional Competition.

The human visual system is equipped to rapidly and implicitly learn and exploit the statistical regularities in our environment. Within visual search, contextual cueing demonstrates how implicit knowledge of scenes can improve search performance. This is commonly interpreted as spatial context in the scenes becoming predictive of the target location, which leads to a more efficient guidance of attention during search. However, what drives this enhanced guidance is unknown. First, it is under debate whether the entire scene (global context) or more local context drives this phenomenon. Second, it is unclear how exactly improved attentional guidance is enabled by target enhancement and distractor suppression. In the present magnetoencephalography experiment, we leveraged rapid invisible frequency tagging to answer these two outstanding questions. We found that the improved performance when searching implicitly familiar scenes was accompanied by a stronger neural representation of the target stimulus, at the cost specifically of those distractors directly surrounding the target. Crucially, this biasing of local attentional competition was behaviorally relevant when searching familiar scenes. Taken together, we conclude that implicitly learned spatial predictive context improves how we search our environment by sharpening the attentional field.

Chunking in Visual Working Memory changes the Guidance of Attention in a Visual Search

Chunking in Visual Working Memory changes the Guidance of Attention in a Visual Search

Looking for the red shirt: Meaningful objects strengthen memory and attentional guidance

Looking for the red shirt: Meaningful objects strengthen memory and attentional guidance

The role of object co-occurrence in attentional guidance: evidence from eye-movements

The role of object co-occurrence in attentional guidance: evidence from eye-movements

Strategic Attention Guidance: The Impact of Dual and Single Cues in Wide Field of View Searches

Strategic Attention Guidance: The Impact of Dual and Single Cues in Wide Field of View Searches

A multimodal dual-branch fusion network for fetal hypoxia detection

Labor is the most severe test of a fetus’s ability to survive hypoxia. Even low-risk full-term fetuses may endure variable degrees of hypoxia as a result of intrapartum uterine contractions. It is imperative for clinicians to expeditiously identify fetal hypoxia and implement timely interventions. However, due to the intricacy and intra-class variability of fetal heart rate (FHR) data, distinguishing normal babies from acidotic fetuses is extremely challenging. This research proposes a novel multimodal dual-branch fusion network to improve the accuracy of fetal hypoxia identification. The dual-branch network is constructed through the signal slicing method. Additionally, we present a novel attention guidance module that leverages spatial attention to capture hypoxia-related information from two branches of signal. Ultimately, the network integrates maternal electronic medical records, FHR features, and FHR signals to achieve multimodal fusion of the input features. Furthermore, label smoothing can yield better calibrated networks, thus improving classification performance even further. On the public dataset, this method obtained a sensitivity of 72.58 %, a specificity of 71.08 %, a quality index of 71.59 %, and an area under the curve of 74.70 %. This method combines maternal-fetal medicine and artificial intelligence, represents a new strategy to recognize fetal acidosis.

Polarized reflection removal with dual-stream attention guidance

Addressing the challenge of removing reflections from images captured through glass surfaces holds significant importance across various practical applications. Existing methods often fall short due to reliance on intermediate predictions or special constraints, leading to undesirable artifacts. This paper addresses this issue by recognizing the inherent correlation between background and reflection. Leveraging the distinct optical properties of polarized images, we propose a novel dual-stream network with feature attention guidance for reflection removal. The model, taking multi-channel polarized images as input, exploits the optical characteristics of transmitted and reflected light to tackle the ill-posed problem. Our dual-stream structure predicts both reflectance and transmission images simultaneously, capitalizing on the unique relationship between these two elements. Additionally, we introduce the innovative differential feature fusion (DFF) block to enhance communication between the two streams. Our model shows superior performance to the state-of-the-art methods when tested on real-world polarization datasets.

A mixed Mamba U-net for prostate segmentation in MR images

The diagnosis of early prostate cancer depends on the accurate segmentation of prostate regions in magnetic resonance imaging (MRI). However, this segmentation task is challenging due to the particularities of prostate MR images themselves and the limitations of existing methods. To address these issues, we propose a U-shaped encoder-decoder network MM-UNet based on Mamba and CNN for prostate segmentation in MR images. Specifically, we first proposed an adaptive feature fusion module based on channel attention guidance to achieve effective fusion between adjacent hierarchical features and suppress the interference of background noise. Secondly, we propose a global context-aware module based on Mamba, which has strong long-range modeling capabilities and linear complexity, to capture global context information in images. Finally, we propose a multi-scale anisotropic convolution module based on the principle of parallel multi-scale anisotropic convolution blocks and 3D convolution decomposition. Experimental results on two public prostate MR image segmentation datasets demonstrate that the proposed method outperforms competing models in terms of prostate segmentation performance and achieves state-of-the-art performance. In future work, we intend to enhance the model's robustness and extend its applicability to additional medical image segmentation tasks.

StarAN: A star attention network utilizing inter-view and intra-view correlations for sparse-view cone-beam computed tomography reconstruction

Sparse sampling can reduce the radiation dose and accelerate the scanning speed of cone-beam computed tomography (CBCT) by increasing the angular interval between projections. However, this approach can compromise the completeness of the projection data, leading to severe artifacts in the reconstructed images. CBCT projection sequences exhibit correlations, and repairing disrupted correlations in sparse-view CBCT projection sequences contributes to improving the quality of the reconstructed images. In this paper, a star attention network (StarAN) based on star attention and a sinogram projection-based cascaded restoration strategy (SPCRS) are proposed. First, missing projection data in the sparse-view projection sequences are filled using interpolation. Second, based on Res-U-Net, the StarAN network is designed with star attention guidance, which integrates channel attention and spatial attention to form a “star-shaped” three-dimensional receptive field, fully capturing the correlations between any pixel in the feature maps and all pixels in the same row or column within the same channel, as well as in corresponding spatial positions across different channels, ensuring the accuracy of the restored projection data. Additionally, an SPCRS is presented that employs two cascaded StarAN networks to sequentially restore projection sequences, further repairing the correlations in the sinogram and projection domains. Finally, the filtered back-projection method is used to reconstruct the restored projection sequences. Experiments conducted on a public walnut dataset and a private dental dataset demonstrate that, compared with state-of-the-art methods, our method achieves superior results, stability, and generalizability.

Measuring User Response to Attention Guidance Using the Integrated Cognitive User Assistance System

Humans are limited in their attentional capacity and miss salient cues when multitasking. We built a platform, ICUa (Integrated Cognitive User assistance) that simulates parts of MATBII, a computer-based multitasking testbed, adding gaze-based attention guidance in an attempt to improve multitasking performance. ICUa uses software agents for monitoring the interface, human behavior and eye gaze in order to guide attention (displaying highlights and arrows). We varied the difficulty of our testbed and found effects on human performance (based on error times) and characterized the types of errors that occurred with higher difficulty and how agent response changed. There was no clear effect of ICUa guidance on gaze patterns or human performance. The visual guidance we deployed may not be sufficient to measurably influence behavior. Future work will develop the detection eye gaze patterns and effective cues to redirect attention using this human-agent interface for experiments on attention guidance.

Gaze cueing improves pattern recognition of histology learners.

Experts perceive and evaluate domain-specific visual information with high accuracy. In doing so, they exhibit eye movements referred to as "expert gaze" to rapidly focus on task-relevant areas. Using eye tracking, it is possible to record these implicit gaze patterns and present them to histology novice learners during training. This article presents a comprehensive evaluation of such expert gaze cueing on pattern recognition of medical students in histology. For this purpose, 53 students were randomized into two groups over eight histology sessions. The control group was presented with an instructional histology video featuring voice commentary. The gaze cueing group was presented the same video, but with an additional overlay of a live recording of the expert's eye movements. Afterward, students' pattern recognition was assessed through 20 image-based tasks (5 retention, 15 transfer) and their cognitive load with the Paas scale. Results showed that gaze cueing significantly outperformed the control group (p = 0.007; d = 0.40). This effect was evident for both, retention (p = 0.003) and transfer tasks (p = 0.046), and generalized across different histological contexts. The cognitive load was similar in both groups. In conclusion, gaze cueing helps histology novice learners to develop their pattern recognition skills, offering a promising method for histology education. Histology educators could benefit from this instructional strategy to provide new forms of attentional guidance to learners in visually complex learning environments.

Employing a Computational Model to Reveal Stimulus-Driven Influences in Interfaces

It is challenging to design complex interfaces that support efficient visual searches. It requires understanding how numerous sources of information contribute to attentional selection, along with an understanding of the tools and tasks used to study visual search. This work is intended to examine visual salience, a pre-attentive process often overlooked in traditional design approaches. Although the importance of visual salience in task-driven searches has been debated, recent evidence suggests this stimulus-driven property contributes to the guidance of attention. Computational modeling has proven effective in predicting the influence of salience (e.g., N-SEEV and Attentional Priority models). With advances in this area of research, continuing examination of the applications and limitations of computational salience models is warranted. Therefore, the present work also examines the predictive power of computational models in displays like web pages and mobile interfaces. Emphasis is placed on the role salience should play in optimizing interface search efficiency.

Reverse Pyramid Attention Guidance Network for Person Re-Identification

Person re-identification aims to retrieve pedestrians with the same identity across different cameras. However, current methods increase attention to interfering regions when dealing with complex backgrounds and occlusion, especially in the presence of similar interfering features. To enhance the robustness of the model, we propose the Reverse Pyramid Attention Guidance (RPAG) network, using a reverse pyramid structure to learn features at multiple granularities. To mitigate the impact of occlusion, we introduce the Similar Feature Filtering (SFF) attention module at the pixel level, using graph convolution to adaptively select occluded regions, thereby enhancing retrieval accuracy by filtering out irrelevant parts. Combining the reverse pyramid structure with the pixel-level attention module strengthens adaptability to complex scenes, guides multi-granularity feature learning, and effectively handles various occlusion scenarios. RPAG achieved Rank-1 accuracies of 96.2%, 93.2%, 88.7%, and 73.2% on the Market1501, DukeMTMC-ReID, MSMT17, and Occluded-Duke datasets, respectively.