Spatial Attention Mechanism Research Articles

An improved efficientnet B7 underwater marine organism image classification model based on the spatial group enhancement attention mechanisms

In this paper, we adopt the Efficientnet B7 model, which is an excellent performer in deep learning algorithms, and improve it by combining with the Spatial Group Enhancement attention mechanism to provide an innovative method for underwater biological image classification. After a reasonable division of the dataset, we introduced the improved Efficientnet B7 model for training. By observing the change of the loss curve on the training set, it can be seen that the loss value of the model gradually decreases from 0.6 to 0.05 and tends to be stable, which indicates that the classification prediction of the model is gradually close to the real result, and it can classify the underwater marine organisms images effectively. The accuracy on the training set gradually increases from the initial 80.12% to 98.31%, and the recall also grows from 75.82% to 98.93%, showing that the model predicts well. The performance on the test set shows an accuracy of 84.86%, a recall of 82.40%, and an F1 value of 80.2%, indicating that the model has good generalisation ability. This means that our proposed improved methods and ideas have achieved significant results in the task of underwater biological image classification, providing a useful reference for research in related fields. By comparing the experimental results, it can be seen that the optimisation of the Efficientnet B7 model by introducing the Spatial Group Enhancement attention mechanism achieves better performance in the underwater marine organisms image classification task, which lays the foundation for the future application of deep learning in underwater environments. This study is of positive significance for the promotion of underwater biology research, marine protection and other fields, and provides new ideas and methods for carrying out more in-depth related research.

Volleyball training video classification description using the BiLSTM fusion attention mechanism

This study aims to explore methods for classifying and describing volleyball training videos using deep learning techniques. By developing an innovative model that integrates Bi-directional Long Short-Term Memory (BiLSTM) and attention mechanisms, referred to BiLSTM-Multimodal Attention Fusion Temporal Classification (BiLSTM-MAFTC), the study enhances the accuracy and efficiency of volleyball video content analysis. Initially, the model encodes features from various modalities into feature vectors, capturing different types of information such as positional and modal data. The BiLSTM network is then used to model multi-modal temporal information, while spatial and channel attention mechanisms are incorporated to form a dual-attention module. This module establishes correlations between different modality features, extracting valuable information from each modality and uncovering complementary information across modalities. Extensive experiments validate the method's effectiveness and state-of-the-art performance. Compared to conventional recurrent neural network algorithms, the model achieves recognition accuracies exceeding 95 % under Top-1 and Top-5 metrics for action recognition, with a recognition speed of 0.04 s per video. The study demonstrates that the model can effectively process and analyze multimodal temporal information, including athlete movements, positional relationships on the court, and ball trajectories. Consequently, precise classification and description of volleyball training videos are achieved. This advancement significantly enhances the efficiency of coaches and athletes in volleyball training and provides valuable insights for broader sports video analysis research.

Single channel speech enhancement using time-frequency attention mechanism based nested U-net model

Deep-learning models have used attention mechanisms to improve quality and intelligibility of noisy speech, demonstrating the effectiveness of attention mechanisms. We rely on either spatial or temporal-based attention mechanisms, resulting in severe information loss. In this paper, a time-frequency attention mechanism with a nested U-network (TFANUNet) is proposed for single-channel speech enhancement. By using TFA, learns which channel, frequency and time information is more significant for speech enhancement. Basically, the proposed model is an encoder-decoder model, where each layer in the encoder and decoder is followed by a nested dense residual dilated DensNet (NDRD) based multi-scale context aggression block. NDRD involves multiple dilated convolutions with different dilatation factors to explore the large receptive area at different scales simultaneously. NDRD avoids the aliasing problem in DenseNet. We integrated the TFA and NDRD blocks into the proposed model to enable refined feature set extraction without information loss and utterance-level context aggregation, respectively. Under seen and unseen noise conditions, the proposed TFAD3MNet model produces an average of 87.02% and 85.04% of STOI values, and 3.19 and 3.01 averaged PESQ values. The trainable parameters of proposed model are 2.09 million,which is very less compared to baselines. TFANUNet model results outperform baselines in terms of STOI and PESQ.

FSNet: A dual-domain network for few-shot image classification

Few-shot learning is a challenging task, that aims to learn and identify novel classes from a limited number of unseen labeled samples. Previous work has focused primarily on extracting features solely in the spatial domain of images. However, the compressed representation in the frequency domain which contains rich pattern information is a powerful tool in the field of signal processing. Combining the frequency and spatial domains to obtain richer information can effectively alleviate the overfitting problem. In this paper, we propose a dual-domain combined model called Frequency Space Net (FSNet), which preprocesses input images simultaneously in both the spatial and frequency domains, extracts spatial and frequency information through two feature extractors, and fuses them to a composite feature for image classification tasks. We start from a different view of frequency analysis, linking conventional average pooling to Discrete Cosine Transformation (DCT). We generalize the compression of the attention mechanism in the frequency domain. Consequently, we propose a novel Frequency Channel Spatial (FCS) attention mechanism. Extensive experiments demonstrate that frequency and spatial information are complementary in few-shot image classification, improving the performance of the model. Our method outperforms state-of-the-art approaches on miniImageNet and CUB.

3D Reconstruction of flame temperature field based on lightweight residual network with spatial attention mechanism

3D Reconstruction of flame temperature field based on lightweight residual network with spatial attention mechanism

Enhancing Classification of Alzheimer’s Disease using Spatial Attention Mechanism

Aim This study aims to enhance the precision of Alzheimer's disease (AD) detection by integrating Spatial Attention Mechanism into a Convolutional Neural Network (CNN) architecture. Background Alzheimer's disease is a progressive neurodegenerative disorder characterized by abnormal protein deposits in the brain, leading to nerve cell loss and posing a significant global health challenge. Early and accurate detection is crucial for disease management and treatment due to the lack of a cure and the disease's severe progression. Objective The objective of this research is to improve the accuracy of Alzheimer's disease classification using MRI data by implementing a Spatial Attention Mechanism in a CNN architecture. Methods The study utilized T1-weighted MRI data from the OASIS 1 and OASIS 2 datasets. The key innovation is the Spatial Attention layer incorporated within a CNN model, which computes the average of each channel in the input feature map. This layer guides subsequent layers to focus on critical brain regions, enhancing the model's accuracy in differentiating between Alzheimer's disease stages. Results The model achieved a validation accuracy of 99.69% with a sensitivity and specificity of 1.0000, demonstrating its reliability in distinguishing between different stages of Alzheimer's disease. The adaptability of the Spatial Attention layer allows the model to assign higher weights to crucial brain regions, improving its discriminative power. Conclusion The integration of the Spatial Attention Mechanism into the CNN architecture significantly contributes to the early detection of Alzheimer's disease, enabling timely interventions. This innovative approach has the potential to revolutionize Alzheimer's diagnosis by enhancing accuracy and offering a robust solution for classification.

Lung segment anything model (LuSAM): a decoupled prompt-integrated framework for automated lung segmentation on chest x-Ray images

Accurate lung segmentation in chest x-ray images plays a pivotal role in early disease detection and clinical decision-making. In this study, we introduce an innovative approach to enhance the precision of lung segmentation using the Segment Anything Model (SAM). Despite its versatility, SAM faces the challenge of prompt decoupling, often resulting in misclassifications, especially with intricate structures like the clavicle. Our research focuses on the integration of spatial attention mechanisms within SAM. This approach enables the model to concentrate specifically on the lung region, fostering adaptability to image variations and reducing the likelihood of false positives. This work has the potential to significantly advance lung segmentation, improving the identification and quantification of lung anomalies across diverse clinical contexts.

Gasoline Engine Misfire Fault Diagnosis Method Based on Improved YOLOv8

In order to realize the online diagnosis and prediction of gasoline engine fire faults, this paper proposes an improved misfire fault detection algorithm model based on YOLOv8 for sound signals of gasoline engines. The improvement involves substituting a C2f module in the YOLOv8 backbone network by a BiFormer attention module and another C2f module substituted by a CBAM module that combines channel and spatial attention mechanisms which enhance the neural network’s capacity to extract the complex features. The normal and misfire sound signals of a gasoline engine are processed by wavelet transformation and converted to time–frequency images for the training, verification, and testing of convolutional neural network. The experimental results show that the precision of the improved YOLOv8 algorithm model is 99.71% for gasoline engine fire fault tests, which is 2 percentage points higher than for the YOLOv8 network model. The diagnosis time of each sound is less than 100 ms, making it suitable for developing IoT devices for gasoline engine misfire fault diagnosis and driverless vehicles.

Implementation of surface crack detection method for nuclear fuel pellets by weakly supervised learning

ABSTRACT Surface cracks are one of the primary defects in nuclear fuel pellets, posing a significant hazard to nuclear safety production. Deep learning-based methods recently developed for crack detection are typically trained using a supervised learning, with detection performance dependent on pixel-level annotations, which suffer from high labeling costs and low efficiency. To address this issue, we propose a Weakly Supervised Crack Detection (WSCD) network for surface crack detection of nuclear fuel pellets. The method adopts bounding-box annotations instead of pixel-level annotations, and rapidly generates pixel-level pseudo-labels using the designed Local Fusion Segmentation (LFS) module. Leveraging the Mask-RCNN network as the backbone, the network introduces spatial attention mechanisms to optimize feature extraction networks, enhancing the extraction capability of multi-scale crack features. Lastly, a novel loss function is optimized to address sample imbalance issues and expedite network convergence. Experimental results on the established crack dataset demonstrate that the proposed method improves labeling efficiency by approximately 20 times, achieving a segmentation accuracy (IoU) of 81.9%, which reaches 92.2% of the segmentation accuracy achieved by supervised learning and outperforms that the IoU (80.3%) of other supervised crack segmentation network for nuclear fuel pellets, and meeting the precision requirements of nuclear fuel pellets production lines.

Contraband detection method in X-ray image based on improved CenterNet network

Security inspection has been played a crucial role in ensuring the safety of railway passenger transportation. The detection of contraband in X-ray images is an important part of safety inspection work. Since the X-ray image of security inspection is a pseudo color image with a unique imaging style, and the restricted objects are mostly small targets, the accuracy of the object detection algorithm using visible light images directly is not high. This research will propose an improved CenterNet method, and design a 16 times down sampling rate backbone network, which is more suitable for detecting contraband in X-ray images. This article will introduce attention mechanisms to increase attention to effective and key areas in X-ray images, and reveal the mechanisms of spatial and channel attention in X-ray images. Furthermore, by designing an adaptive feature fusion mechanism, the drawback of the original CenterNet by using only the last layer of output features is compensated. Experiments have shown that on the SIXray dataset, mAP is improved by 3.3% compared to the original CenterNet method, and 5.5%, 2.4%, 3.6%, 10.1% compared to SSD, Yolo V3, FPN, DERT algorithms. The detection accuracy of restricted items in X-ray images is effectively improved.

LESM-YOLO: An Improved Aircraft Ducts Defect Detection Model.

Aircraft ducts play an indispensable role in various systems of an aircraft. The regular inspection and maintenance of aircraft ducts are of great significance for preventing potential failures and ensuring the normal operation of the aircraft. Traditional manual inspection methods are costly and inefficient, especially under low-light conditions. To address these issues, we propose a new defect detection model called LESM-YOLO. In this study, we integrate a lighting enhancement module to improve the accuracy and recognition of the model under low-light conditions. Additionally, to reduce the model's parameter count, we employ space-to-depth convolution, making the model more lightweight and suitable for deployment on edge detection devices. Furthermore, we introduce Mixed Local Channel Attention (MLCA), which balances complexity and accuracy by combining local channel and spatial attention mechanisms, enhancing the overall performance of the model and improving the accuracy and robustness of defect detection. Finally, we compare the proposed model with other existing models to validate the effectiveness of LESM-YOLO. The test results show that our proposed model achieves an mAP of 96.3%, a 5.4% improvement over the original model, while maintaining a detection speed of 138.7, meeting real-time monitoring requirements. The model proposed in this paper provides valuable technical support for the detection of dark defects in aircraft ducts.

CVTNet: A Fusion of Convolutional Neural Networks and Vision Transformer for Wetland Mapping Using Sentinel-1 and Sentinel-2 Satellite Data

Wetland mapping is a critical component of environmental monitoring, requiring advanced techniques to accurately represent the complex land cover patterns and subtle class differences innate in these ecosystems. This study aims to address these challenges by proposing CVTNet, a novel deep learning (DL) model that integrates convolutional neural networks (CNNs) and vision transformer (ViT) architectures. CVTNet uses channel attention (CA) and spatial attention (SA) mechanisms to enhance feature extraction from Sentinel-1 (S1) and Sentinel-2 (S2) satellite data. The primary goal of this model is to achieve a balanced trade-off between Precision and Recall, which is essential for accurate wetland mapping. The class-specific analysis demonstrated CVTNet’s proficiency across diverse classes, including pasture, shrubland, urban, bog, fen, and water. Comparative analysis showed that CVTNet outperforms contemporary algorithms such as Random Forest (RF), ViT, multi-layer perceptron mixer (MLP-mixer), and hybrid spectral net (HybridSN) classifiers. Additionally, the attention mechanism (AM) analysis and sensitivity analysis highlighted the crucial role of CA, SA, and ViT in focusing the model’s attention on critical regions, thereby improving the mapping of wetland regions. Despite challenges at class boundaries, particularly between bog and fen, and misclassifications of swamp pixels, CVTNet presents a solution for wetland mapping.

Automated Left Ventricle Segmentation in Echocardiography Using YOLO: A Deep Learning Approach for Enhanced Cardiac Function Assessment

Accurate segmentation of the left ventricle (LV) using echocardiogram (Echo) images is essential for cardiovascular analysis. Conventional techniques are labor-intensive and exhibit inter-observer variability. Deep learning has emerged as a powerful tool for automated medical image segmentation, offering advantages in speed and potentially superior accuracy. This study explores the efficacy of employing a YOLO (You Only Look Once) segmentation model for automated LV segmentation in Echo images. YOLO, a cutting-edge object detection model, achieves exceptional speed–accuracy balance through its well-designed architecture. It utilizes efficient dilated convolutional layers and bottleneck blocks for feature extraction while incorporating innovations like path aggregation and spatial attention mechanisms. These attributes make YOLO a compelling candidate for adaptation to LV segmentation in Echo images. We posit that by fine-tuning a pre-trained YOLO-based model on a well-annotated Echo image dataset, we can leverage the model’s strengths in real-time processing and precise object localization to achieve robust LV segmentation. The proposed approach entails fine-tuning a pre-trained YOLO model on a rigorously labeled Echo image dataset. Model performance has been evaluated using established metrics such as mean Average Precision (mAP) at an Intersection over Union (IoU) threshold of 50% (mAP50) with 98.31% and across a range of IoU thresholds from 50% to 95% (mAP50:95) with 75.27%. Successful implementation of YOLO for LV segmentation has the potential to significantly expedite and standardize Echo image analysis. This advancement could translate to improved clinical decision-making and enhanced patient care.

Super-resolution of digital rock images with hybrid attention multi-branch neural network

High-resolution digital rock images are vital in studying microstructure and fluid flow characterization of oil and gas reservoirs. Computed tomography (CT) is currently the primary tool to acquire digital rock images because of its non-destructiveness and three-dimensional imaging capability. However, the restriction of imaging capability of CT sets a compromise between image resolution and field of view (FOV). The Hybrid Attention Multi-branch Super-Resolution Network (HAMSR) proposed in this study aims to remedy this deficiency. HAMSR employs a limited discrete distribution to capture all possible high-order information and generate reliable high-resolution images. The network is built to prioritize the recovery of pores, cracks, and textures in digital rock images by introducing channel and spatial attention mechanisms. The structure together with modules of HAMSR have been designed and optimized to yield superior performance with fewer parameters and less training time. HAMSR and several contemporary advanced models are trained on the DeepRock-SR 2D dataset which contains lots of sandstone and carbonate images. It is evident that HAMSR-generated images have minimal noise, clearer edges, and more micropores. Quantitative calculations show that compared to traditional bicubic interpolation, HAMSR acquires a 2.669 dB and 1.376 dB increase in peak signal-to-noise ratio (PSNR) respectively on the test sets (27%–38% reduction in relative error). Furthermore, the high-resolution (HR) images reconstructed by HAMSR are generally consistent with real HR images concerning statistical characteristics such as porosity, pore size distribution, shape factor distribution, and two-point correlation function. This work provides a new super-resolution model to generate high-quality digital rock images for further analysis.

O2SAT: Object-Oriented-Segmentation-Guided Spatial-Attention Network for 3D Object Detection in Autonomous Vehicles

Autonomous vehicles (AVs) strive to adapt to the specific characteristics of sustainable urban environments. Accurate 3D object detection with LiDAR is paramount for autonomous driving. However, existing research predominantly relies on the 3D object-based assumption, which overlooks the complexity of real-world road environments. Consequently, current methods experience performance degradation when targeting only local features and overlooking the intersection of objects and road features, especially in uneven road conditions. This study proposes a 3D Object-Oriented-Segmentation Spatial-Attention (O2SAT) approach to distinguish object points from road points and enhance the keypoint feature learning by a channel-wise spatial attention mechanism. O2SAT consists of three modules: Object-Oriented Segmentation (OOS), Spatial-Attention Feature Reweighting (SFR), and Road-Aware 3D Detection Head (R3D). OOS distinguishes object and road points and performs object-aware downsampling to augment data by learning to identify the hidden connection between landscape and object; SFR performs weight augmentation to learn crucial neighboring relationships and dynamically adjust feature weights through spatial attention mechanisms, which enhances the long-range interactions and contextual feature discrimination for noise suppression, improving overall detection performance; and R3D utilizes refined object segmentation and optimized feature representations. Our system forecasts prediction confidence into existing point-backbones. Our method’s effectiveness and robustness across diverse datasets (KITTI) has been demonstrated through vast experiments. The proposed modules seamlessly integrate into existing point-based frameworks, following a plug-and-play approach.

Physics-informed interactive network for hyperspectral image classification

In hyperspectral image (HSI) classification, convolutional neural networks (CNNs) have made great progress. In recent CNN-based HSI classification methods, spatial attention mechanism has been widely used to emphasize more salient features and suppress less useful ones. However, these methods tend to utilize plug-and-play attention modules which ignore the physical meaning of HSI pixels. To reflect the physical construction of HSI pixels in the attention mask, we propose a physics-informed interactive network (PI2Net) integrating a spatial guided spectral component stream (SGSC-stream) and a spectral constrained spatial attention stream (SCSA-stream). The SGSC-stream can obtain more expressive abundance features of the input HSI cubes through a component analysis block. The SCSA-stream completes the prediction of hyperspectral data in three-stage features extraction cells and a classification cell. To connect and interact between these two streams, a component-aware attention module (CAAM) and a class-wise endmember activation gate (CWEAG) are designed. The CAAM measures the similarity of abundance features to generate a component-aware attention mask, so as to learn the component-based relationships between pixels. Then, the multiscale attention masks are embedded into different stages of the SCSA-stream. The CWEAG is proposed to iteratively generate more accurate abundance features with the classification result of SCSA-stream. Experimental results based on three common hyperspectral data sets demonstrate that PI2Net outperforms several state-of-the-art methods in terms of qualitative and quantitative analysis.

Multi-scale medical image segmentation based on pixel encoding and spatial attention mechanism

In response to the issues of single-scale information loss and large model parameter size during the sampling process in U-Net and its variants for medical image segmentation, this paper proposes a multi-scale medical image segmentation method based on pixel encoding and spatial attention. Firstly, by redesigning the input strategy of the Transformer structure, a pixel encoding module is introduced to enable the model to extract global semantic information from multi-scale image features, obtaining richer feature information. Additionally, deformable convolutions are incorporated into the Transformer module to accelerate convergence speed and improve module performance. Secondly, a spatial attention module with residual connections is introduced to allow the model to focus on the foreground information of the fused feature maps. Finally, through ablation experiments, the network is lightweighted to enhance segmentation accuracy and accelerate model convergence. The proposed algorithm achieves satisfactory results on the Synapse dataset, an official public dataset for multi-organ segmentation provided by the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), with Dice similarity coefficient (DSC) and 95% Hausdorff distance (HD95) scores of 77.65 and 18.34, respectively. The experimental results demonstrate that the proposed algorithm can enhance multi-organ segmentation performance, potentially filling the gap in multi-scale medical image segmentation algorithms, and providing assistance for professional physicians in diagnosis.

Reading direction interacts with spatial processes of temporal order verbal working memory: evidence from Iranian right-to-left readers

ABSTRACT Reading direction affects cognitive processes such as spatial processing and attention. Testing Farsi right-to-left readers, we investigated the effect of reading direction on the involved spatial and attentional processes in a verbal working memory task. Previous research in left-to-right readers has shown that serial order in verbal WM is encoded spatially from left to right and that mechanisms of spatial attention operate on these mental representations. First, we confirmed that the spatial representation of serial information in WM follows the right-to-left reading direction of Farsi. Second, we demonstrated the influence of reading direction on the distribution of spatial attention over the reading-direction-contingent mental representation of serial information in WM. Behavioural RTs and neural markers of spatial attention shifts (EDAN and ADAN ERPs and lateralised posterior alpha activity) confirm the right-to-left representation of serial order in WM and the involvement of spatial attention in retrieving items from this mnemonic space.

HEAL: High-Frequency Enhanced and Attention-Guided Learning Network for Sparse-View CT Reconstruction.

X-ray computed tomography (CT) imaging technology has become an indispensable diagnostic tool in clinical examination. However, it poses a risk of ionizing radiation, making the reduction of radiation dose one of the current research hotspots in CT imaging. Sparse-view imaging, as one of the main methods for reducing radiation dose, has made significant progress in recent years. In particular, sparse-view reconstruction methods based on deep learning have shown promising results. Nevertheless, efficiently recovering image details under ultra-sparse conditions remains a challenge. To address this challenge, this paper proposes a high-frequency enhanced and attention-guided learning Network (HEAL). HEAL includes three optimization strategies to achieve detail enhancement: Firstly, we introduce a dual-domain progressive enhancement module, which leverages fidelity constraints within each domain and consistency constraints across domains to effectively narrow the solution space. Secondly, we incorporate both channel and spatial attention mechanisms to improve the network's feature-scaling process. Finally, we propose a high-frequency component enhancement regularization term that integrates residual learning with direction-weighted total variation, utilizing directional cues to effectively distinguish between noise and textures. The HEAL network is trained, validated and tested under different ultra-sparse configurations of 60 views and 30 views, demonstrating its advantages in reconstruction accuracy and detail enhancement.

A multi-scale semantic feature fusion method for remote sensing crop classification

The efficient and accurate acquisition of crop spatial planting structure information is of great significance to ensure national food security. Current crop classification studies have still not achieved optimal results in accurate crop classification identification due to the complex heterogeneous details of clear images and insufficient analysis and utilization of information. This study takes the Yuncheng-Linfen Basin in Shanxi, China as the study area and uses the ReliefF-RFE feature selection algorithm to achieve data dimensionality reduction. Contextual semantic feature aggregation, spatial channel attention mechanism, and hierarchical refinement strategy are used to fuse the multi-scale feature information in the extracted remote sensing images from the top down and bottom up. This study shows that 1) the ReliefF-RFE algorithm can effectively reduce the dimensionality of 224-dimensional multisource features and generate a 31-dimensional optimal feature space subset with strong interpretation to improve the computational efficiency of the model; 2) the multi-scale feature fusion model constructed in this study is better than UNet, ResNet, DeepLabv3+, HRNet and Swin Transformer in classification recognition of winter wheat and corn, with an overall accuracy of 95.76% and 93.44%; 3) A multiple feature fusion strategy designed through ablation experiments was evaluated to evaluate the reasonableness of each module and validate the irreplaceability and superiority of the model for improving crop accuracy in complex and heterogeneous terrain; 4) the multi-scale feature fusion model converts deep abstract features into local detail information by cross-scale contextual aggregation of deep and shallow multi-scale features, it obtains good extraction effect on fragmented small plots, and the reduction of local fine details of features is significantly better than other models. The multi-scale feature fusion classification model constructed in this study fully considers the interpretability of the input features and fuses the semantic features at different scales to eliminate the “semantic gap”, which shows that the model has strong stability and robustness and provides a reference for fast and accurate mapping of Sentinel-2 images of crops in large areas.