Spatial Attention Module Research Articles

YOLOLayout: Multi-Scale Cross Fusion Former for Document Layout Analysis

Document layout analysis (DLA) is a technique used to locate and classify layout elements in a document, such as Table, Figure, List, and Text. While deep-learning-based methods in computer vision have shown excellent performance in detecting Text and Figures, they are still unsatisfactory in accurately recognizing the blocks of List, Title, and Table categories with limited data. To address this issue, we propose a single-stage DLA model that incorporates a Multi-Scale Shallow Visual Feature Enhancement Module (MS-SVFEM) and a Multi-Scale Cross-Feature Fusion Module (MS-CFF). The MS-SVFEM extracts multi-scale spatial information through the channel attention module, spatial attention module, and multi-branch convolution. The MS-CFF fuses different level features through an attention mechanism. The experiments showed that the mAP accuracy of YOLOLayout compared to the baseline model is 2.2% and 1.5% higher on the PubLayNet Dataset and the ISCAS-CLAD dataset.

A feature‐enhanced hybrid attention network for traffic sign recognition in real scenes

AbstractCurrently, traffic sign recognition techniques have been brought into the assistive driving of automobiles. However, small traffic sign recognition in real scenes is still a challenging task due to the class imbalance issue and the size limit of the traffic signs. To address the above issues, a feature‐enhanced hybrid attention network is proposed based on YOLOv5s for a small, fast, and accurate traffic sign detector. First, a series of online data augmentation strategies are designed in the preprocessing module for the model training. Second, the hybrid channel and spatial attention module CSAM are integrated into the backbone for a better feature extraction ability. Third, the channel attention module CAM is used in the detection head for a more efficient feature fusion ability. To validate the approach, extensive experiments are conducted based on the Tsinghua‐Tencent 100K dataset. It is found that the novel method achieves state‐of‐the‐art performance with only negligible increases in the model parameter and computational overhead. Specifically, the , parameters, and FLOPs are 85.8%, 7.13 M, and 16.1 G, respectively.

Assessment of Power System Stability During Transients Using Deep Residual Shrinkage Network and CBAM Integration

Aiming at the noise that may exist in the data collection and transmission of power system synchronous phasor measurement unit (PMU), as well as the imbalance between stable and unstable samples, making the model likely to tilt to most kinds of samples, thus affecting the evaluation results of power system transient stability model, a transient stability assessment method integrating Convolutional Block Attention Module (CBAM) and deep residual shrinkage network is proposed. First, the mapping relationship between the input values and the labels of the stability results is established, and the electrical quantities are transformed into the form of convolutional feature maps to be input into the model. Second, channel attention module and spatial attention module are introduced, and soft threshold function is used to automatically learn the noise threshold and reduce the interference of the noise. As the model tends to favor the majority of the samples during training, gradient harmonized mechanism (GHM) loss function is introduced to improve the attention of the minority of samples as well as the training effect and evaluation performance of the model. Finally, the model is tested by simulation on the IEEE39 node system.

BUA-Net: Boundary and uncertainty-aware attention network for lumbar multi-region magnetic resonance imaging segmentation

Background and Objective:Multi-region segmentation by axial lumbar magnetic resonance imaging (MRI) is important for the diagnosis of lumbar spinal stenosis (LSS) and lumbar disc herniation (LDH) disease. Deep learning-based methods have been widely used in this field, but the segmentation task is still challenging due to intra-class grayscale unevenness, inter-class low contrast and unclear boundaries. Methods:To solve the above problems, we propose a novel boundary and uncertainty-aware attention network (BUA-Net). It is structured around a single encoder and two decoders, each dedicated to either multi-region or boundary segmentation. We use an uncertainty-aware attention (UA) module to extract regions with large segmentation uncertainty and generate an attention mask. The learning and supervisory capabilities of the model are enhanced by a fine seg (FS) module and an attention supervision loss (ASL) in the boundaries and blurred regions. Moreover, a multi-scale spatial channel attention (MSCA) module is proposed that refines the high-level features without increasing any layers. Results:The results of the BUA-Net experiments were tested against seven networks and showed a 0.26% improvement in Dice scores, a 0.86% improvement in precision, and a 0.2% improvement in recall. In metrics related to boundary segmentation effectiveness, our method nearly matches the state-of-the-art in the ASD metric, surpasses it by 2.32 in the 95HD metric, and leads by 2.04, 2.68, and 1.22 in the IVD, TS, and PE categories, respectively. Conclusions:The experimental results demonstrate that our proposed method outperforms other state-of-the-art methods and shows excellent segmentation ability in the boundaries and blurred regions.

MatchDet: A Collaborative Framework for Image Matching and Object Detection

Image matching and object detection are two fundamental and challenging tasks, while many related applications consider them two individual tasks (i.e. task-individual). In this paper, a collaborative framework called MatchDet (i.e. task-collaborative) is proposed for image matching and object detection to obtain mutual improvements. To achieve the collaborative learning of the two tasks, we propose three novel modules, including a Weighted Spatial Attention Module (WSAM) for Detector, and Weighted Attention Module (WAM) and Box Filter for Matcher. Specifically, the WSAM highlights the foreground regions of target image to benefit the subsequent detector, the WAM enhances the connection between the foreground regions of pair images to ensure high-quality matches, and Box Filter mitigates the impact of false matches. We evaluate the approaches on a new benchmark with two datasets called Warp-COCO and miniScanNet. Experimental results show our approaches are effective and achieve competitive improvements.

Unified Convolutional Sparse Transformer for Disease Diagnosis, Monitoring, Drug Development, and Therapeutic Effect Prediction from EEG Raw Data.

Electroencephalogram (EEG) analysis plays an indispensable role across contemporary medical applications, which encompasses diagnosis, monitoring, drug discovery, and therapeutic assessment. This work puts forth an end-to-end deep learning framework that is uniquely tailored for versatile EEG analysis tasks by directly operating on raw waveform inputs. It aims to address the challenges of manual feature engineering and the neglect of spatial interrelationships in existing methodologies. Specifically, a spatial channel attention module is introduced to emphasize the critical inter-channel dependencies in EEG signals through channel statistics aggregation and multi-layer perceptron operations. Furthermore, a sparse transformer encoder is used to leverage selective sparse attention in order to efficiently process long EEG sequences while reducing computational complexity. Distilling convolutional layers further concatenates the temporal features and retains only the salient patterns. As it was rigorously evaluated on key EEG datasets, our model consistently accomplished a superior performance over the current approaches in detection and classification assignments. By accounting for both spatial and temporal relationships in an end-to-end paradigm, this work facilitates a versatile, automated EEG understanding across diseases, subjects, and objectives through a singular yet customizable architecture. Extensive empirical validation and further architectural refinement may promote broader clinical adoption prospects.

Rice disease segmentation method based on CBAM-CARAFE-DeepLabv3+

Rice is an important food crop, but it is susceptible to diseases during its growth process. Rapid, accurate, and effective identification of rice diseases is important for targeted measures to control disease spread. It is crucial for improving rice yield and quality. Therefore, this study proposes a CBAM-CARAFE-DeepLabv3+ rice disease segmentation method that combines attention mechanisms and feature recombination. This method focuses on three common diseases in rice growth: bacterial blight, blast, and brown spot disease. To enhance the extraction of favorable features, the algorithm adopts CBAM-RepViT as the backbone network. That is, the Squeeze-and-Excitation (SE) attention mechanism embedded in the efficient and lightweight RepViT network is replaced by the lightweight Convolutional Block Attention Module (CBAM). Compared to the SE attention mechanism, CBAM introduces a spatial attention module that focuses on important spatial positions in the feature map. It allows the model to extract more rich and detailed feature information by attending to both the channel and spatial dimensions of the feature map. Additionally, to further improve the feature extraction ability and image edge segmentation accuracy during upsampling, the lightweight Content-Aware ReAssembly of FEatures (CARAFE) operator is introduced into the decoding module for upsampling. Finally, to address the issue of imbalance between foreground and background pixel ratios in rice disease, a hybrid loss function composed of cross-entropy (CE) loss and Dice loss is proposed. Experimental results show that, compared to other networks such as DeepLabv3+, the proposed CBAM-CARAFE-DeepLabv3+ method achieves further improvement in segmentation accuracy, providing a new method for the development of rice disease segmentation technology.

Research on bronze wine vessel classification using improved SSA-CBAM-GNNs.

This article proposes an advanced classification algorithm for bronze drinking utensils, taking into account the complexity of their cultural characteristics and the challenges of dynasty classification. The SSA-CBAM-GNNs algorithm integrates the Sparrow Search Algorithm (SSA), Spatial and Spectral Attention (CBAM) modules, and Graph Neural Networks (GNNs). The CBAM module is essential for optimizing feature extraction weights in graph neural networks, while SSA enhances the weighted network and expedites the convergence process. Experimental results, validated through various performance evaluation indicators, illustrate the outstanding performance of the improved SSA-CBAM-GNNs algorithm in accurately identifying and classifying cultural features of bronze drinking utensils. Comparative experiments confirm the algorithm's superiority over other methods. Overall, this study proposes a highly efficient identification and classification algorithm, and its effectiveness and excellence in extracting and identifying cultural features of bronze drinking utensils are experimentally demonstrated.

Celiac disease diagnosis from endoscopic images based on multi-scale adaptive hybrid architecture model

Objective. Celiac disease (CD) has emerged as a significant global public health concern, exhibiting an estimated worldwide prevalence of approximately 1%. However, existing research pertaining to domestic occurrences of CD is confined mainly to case reports and limited case analyses. Furthermore, there is a substantial population of undiagnosed patients in the Xinjiang region. This study endeavors to create a novel, high-performance, lightweight deep learning model utilizing endoscopic images from CD patients in Xinjiang as a dataset, with the intention of enhancing the accuracy of CD diagnosis. Approach. In this study, we propose a novel CNN-Transformer hybrid architecture for deep learning, tailored to the diagnosis of CD using endoscopic images. Within this architecture, a multi-scale spatial adaptive selective kernel convolution feature attention module demonstrates remarkable efficacy in diagnosing CD. Within this module, we dynamically capture salient features within the local channel feature map that correspond to distinct manifestations of endoscopic image lesions in the CD-affected areas such as the duodenal bulb, duodenal descending segment, and terminal ileum. This process serves to extract and fortify the spatial information specific to different lesions. This strategic approach facilitates not only the extraction of diverse lesion characteristics but also the attentive consideration of their spatial distribution. Additionally, we integrate the global representation of the feature map obtained from the Transformer with the locally extracted information via convolutional layers. This integration achieves a harmonious synergy that optimizes the diagnostic prowess of the model. Main results. Overall, the accuracy, specificity, F1-Score, and precision in the experimental results were 98.38%, 99.04%, 98.66% and 99.38%, respectively. Significance. This study introduces a deep learning network equipped with both global feature response and local feature extraction capabilities. This innovative architecture holds significant promise for the accurate diagnosis of CD by leveraging endoscopic images captured from diverse anatomical sites.

Multi-task learning for segmentation and classification of breast tumors from ultrasound images

Segmentation and classification of breast tumors are critical components of breast ultrasound (BUS) computer-aided diagnosis (CAD), which significantly improves the diagnostic accuracy of breast cancer. However, the characteristics of tumor regions in BUS images, such as non-uniform intensity distributions, ambiguous or missing boundaries, and varying tumor shapes and sizes, pose significant challenges to automated segmentation and classification solutions. Many previous studies have proposed multi-task learning methods to jointly tackle tumor segmentation and classification by sharing the features extracted by the encoder. Unfortunately, this often introduces redundant or misleading information, which hinders effective feature exploitation and adversely affects performance. To address this issue, we present ACSNet, a novel multi-task learning network designed to optimize tumor segmentation and classification in BUS images. The segmentation network incorporates a novel gate unit to allow optimal transfer of valuable contextual information from the encoder to the decoder. In addition, we develop the Deformable Spatial Attention Module (DSAModule) to improve segmentation accuracy by overcoming the limitations of conventional convolution in dealing with morphological variations of tumors. In the classification branch, multi-scale feature extraction and channel attention mechanisms are integrated to discriminate between benign and malignant breast tumors. Experiments on two publicly available BUS datasets demonstrate that ACSNet not only outperforms mainstream multi-task learning methods for both breast tumor segmentation and classification tasks, but also achieves state-of-the-art results for BUS tumor segmentation. Code and models are available at https://github.com/qqhe-frank/BUS-segmentation-and-classification.git.

Feature disparity learning for weakly supervised object localization

Weakly supervised object localization (WSOL) aims to localize objects with only image-level labels. As a common WSOL method, adversarial erasing always masks the most discriminative region in the feature space to compel the network to localize more regions of the object. However, with the discriminative region vanishing, the localizer is confused when distinguishing the regions of object from the background. In this paper, we propose a new feature disparity learning (FDL), which encourages the network to learn more distinctive features from the object region with similarity measurement after feature enhancement. Specifically, we first introduce a Spatial Vector Cross Attention (SVCA) module. This module enhances responses in less discriminative region of erased feature maps by reintegrating the spatial distribution of features through the capture of interdependencies among spatial vectors on each channel. Furthermore, we propose a feature complementarity loss to measure the similarity between unerased features and erased features, guiding the network to learn feature disparities caused by adversarial erasing for improved localization and classification. Several experimental studies demonstrate a significant increase in localization performance over the existing state-of-the-art erasing methods on the CUB 200–2011 and ILSVRC 2016 datasets.

SSTrack: An Object Tracking Algorithm Based on Spatial Scale Attention

The traditional Siamese object tracking algorithm uses a convolutional neural network as the backbone and has achieved good results in improving tracking precision. However, due to the lack of global information and the use of spatial and scale information, the accuracy and speed of such tracking algorithms still need to be improved in complex environments such as rapid motion and illumination variation. In response to the above problems, we propose SSTrack, an object tracking algorithm based on spatial scale attention. We use dilated convolution branch and covariance pooling to build a spatial scale attention module, which can extract the spatial and scale information of the target object. By embedding the spatial scale attention module into Swin Transformer as the backbone, the ability to extract local detailed information has been enhanced, and the success rate and precision of tracking have been improved. At the same time, to reduce the computational complexity of self-attention, Exemplar Transformer is applied to the encoder structure. SSTrack achieved 71.5% average overlap (AO), 86.7% normalized precision (NP), and 68.4% area under curve (AUC) scores on the GOT-10k, TrackingNet, and LaSOT. The tracking speed reached 28fps, which can meet the need for real-time object tracking.

Plug-and-play multi-dimensional attention module for accurate Human Activity Recognition

Unlike image data, Human Activity Recognition (HAR) data is collected through sensors placed on different parts of the body. Each sensor may have varying importance for different human actions. Attention mechanisms can dynamically identify the significance of signals from different sensors. However, existing attention modules often operate along channel or spatial dimensions, producing one-dimensional or two-dimensional weights, and considering neurons within every channel or spatial location as equivalent. This might constrain their capacity to acquire more distinctive cues. In this paper, we present a novel Three-Dimensional Weight Attention Module (WAM) that considers both spatial and weight information of each channel. Specifically, we commence by introducing an energy function and subsequently assess the significance of each neuron through optimization. Unlike conventional channel or spatial attention modules, this module can compute 3D attention weights for feature maps without additional parameters. Experiments conducted on four diverse HAR datasets showcase that the Three-Dimensional Weight Attention Module can seamlessly integrate with convolutional models, significantly enhancing model performance without introducing extra computational burden. To further highlight the utility of WAM, we measure the actual inference time on an embedded platform.

Non-destructive detection of defective maize kernels using hyperspectral imaging and convolutional neural network with attention module

Rapid, effective and non-destructive detection of the defective maize kernels is crucial for their high-quality storage in granary. Hyperspectral imaging (HSI) coupled with convolutional neural network (CNN) based on spectral and spatial attention (Spl-Spal-At) module was proposed for identifying the different types of maize kernels. The HSI data within 380–1000 nm of six classes of sprouted, heat-damaged, insect-damaged, moldy, broken and healthy kernels was collected. The CNN-Spl-At, CNN-Spal-At and CNN-Spl-Spal-At models were established based on the spectra, images and their fusion features as inputs for the recognition of different kernels. Further compared the performances of proposed models and conventional models were built by support vector machine (SVM) and extreme learning machine (ELM). The results indicated that the recognition ability of CNN with attention series models was significantly better than that of SVM and ELM models and fused features were more conducive to expressing the appearance of different kernels than single features. And the CNN-Spl-Spal-At model had an optimal recognition result with high average classification accuracy of 98.04 % and 94.56 % for the training and testing sets, respectively. The recognition results were visually presented on the surface image of kernels with different colors. The CNN-Spl-Spal-At model was built in this study could effectively detect defective maize kernels, and it also had great potential to provide the analysis approaches for the development of non-destructive testing equipment based on HSI technique for maize quality.

An Expert System for Indian Sign Language Recognition Using Spatial Attention–based Feature and Temporal Feature

Sign Language (SL) is the only means of communication for the hearing-impaired people. Normal people have difficulty understanding SL, resulting in a communication barrier between hearing impaired people and hearing community. However, the Sign Language Recognition System (SLRS) has helped to bridge the communication gap. Many SLRs are proposed for recognizing SL; however, a limited number of works are reported for Indian Sign Language (ISL). Most of the existing SLRS focus on global features other than the Region of Interest (ROI). Focusing more on the hand region and extracting local features from the ROI improves system accuracy. The attention mechanism is a widely used technique for emphasizing the ROI. However, only a few SLRS used the attention method. They employed the Convolution Block Attention Module and temporal attention but Spatial Attention (SA) is not utilized in previous SLRS. Therefore, a novel SA based SLRS named Spatial Attention-based Sign Language Recognition Module (SASLRM) is proposed to recognize ISL words for emergency situations. SASLRM recognizes ISL words by combining convolution features from a pretrained VGG-19 model and attention features from a SA module. The proposed model accomplished an average accuracy of 95.627% on the ISL dataset. The proposed SASLRM is further validated on LSA64, WLASL, and Cambridge Hand Gesture Recognition datasets where, the proposed model reached an accuracy of 97.84%, 98.86%, and 98.22%, respectively. The results indicate the effectiveness of the proposed SLRS in comparison with the existing SLRS.

Improving Continuous Sign Language Recognition with Consistency Constraints and Signer Removal

Deep-learning-based continuous sign language recognition (CSLR) models typically consist of a visual module, a sequential module, and an alignment module. However, the effectiveness of training such CSLR backbones is hindered by limited training samples, rendering the use of a single connectionist temporal classification loss insufficient. To address this limitation, we propose three auxiliary tasks to enhance CSLR backbones. First, we enhance the visual module, which is particularly sensitive to the challenges posed by limited training samples, from the perspective of consistency. Specifically, since sign languages primarily rely on signers’ facial expressions and hand movements to convey information, we develop a keypoint-guided spatial attention module that directs the visual module to focus on informative regions, thereby ensuring spatial attention consistency. Furthermore, recognizing that the output features of both the visual and sequential modules represent the same sentence, we leverage this prior knowledge to better exploit the power of the backbone. We impose a sentence embedding consistency constraint between the visual and sequential modules, enhancing the representation power of both features. The resulting CSLR model, referred to as consistency-enhanced CSLR, demonstrates superior performance on signer-dependent datasets, where all signers appear during both training and testing. To enhance its robustness for the signer-independent setting, we propose a signer removal module based on feature disentanglement, effectively eliminating signer-specific information from the backbone. To validate the effectiveness of the proposed auxiliary tasks, we conduct extensive ablation studies. Notably, utilizing a transformer-based backbone, our model achieves state-of-the-art or competitive performance on five benchmarks, including PHOENIX-2014, PHOENIX-2014-T, PHOENIX-2014-SI, CSL, and CSL-Daily. Code and models are available at https://github.com/2000ZRL/LCSA_C2SLR_SRM.

Harmonious Mutual Learning for Facial Emotion Recognition

Facial emotion recognition in the wild is an important task in computer vision, but it still remains challenging since the influence of backgrounds, occlusions and illumination variations in facial images, as well as the ambiguity of expressions. This paper proposes a harmonious mutual learning framework for emotion recognition, mainly through utilizing attention mechanisms and probability distributions without utilizing additional information. Specifically, this paper builds an architecture with two emotion recognition networks and makes progressive cooperation and interaction between them. We first integrate self-mutual attention module into the backbone to learn discriminative features against the influence from emotion-irrelevant facial information. In this process, we deploy spatial attention module and convolutional block attention module for the two networks respectively, guiding to enhanced and supplementary learning of attention. Further, in the classification head, we propose to learn the latent ground-truth emotion probability distributions using softmax function with temperature to characterize the expression ambiguity. On this basis, a probability distribution distillation learning module is constructed to perform class semantic interaction using bi-directional KL loss, allowing mutual calibration for the two networks. Experimental results on three public datasets show the superiority of the proposed method compared to state-of-the-art ones.

BayesNet: Enhancing UAV-Based Remote Sensing Scene Understanding with Quantifiable Uncertainties

Remote sensing stands as a fundamental technique in contemporary environmental monitoring, facilitating extensive data collection and offering invaluable insights into the dynamic nature of the Earth’s surface. The advent of deep learning, particularly convolutional neural networks (CNNs), has further revolutionized this domain by enhancing scene understanding. However, despite the advancements, traditional CNN methodologies face challenges such as overfitting in imbalanced datasets and a lack of precise uncertainty quantification, crucial for extracting meaningful insights and enhancing the precision of remote sensing techniques. Addressing these critical issues, this study introduces BayesNet, a Bayesian neural network (BNN)-driven CNN model designed to normalize and estimate uncertainties, particularly aleatoric and epistemic, in remote sensing datasets. BayesNet integrates a novel channel–spatial attention module to refine feature extraction processes in remote sensing imagery, thereby ensuring a robust analysis of complex scenes. BayesNet was trained on four widely recognized unmanned aerial vehicle (UAV)-based remote sensing datasets, UCM21, RSSCN7, AID, and NWPU, and demonstrated good performance, achieving accuracies of 99.99%, 97.30%, 97.57%, and 95.44%, respectively. Notably, it has showcased superior performance over existing models in the AID, NWPU, and UCM21 datasets, with enhancements of 0.03%, 0.54%, and 0.23%, respectively. This improvement is significant in the context of complex scene classification of remote sensing images, where even slight improvements mark substantial progress against complex and highly optimized benchmarks. Moreover, a self-prepared remote sensing testing dataset is also introduced to test BayesNet against unseen data, and it achieved an accuracy of 96.39%, which showcases the effectiveness of the BayesNet in scene classification tasks.

Spherical image acquisition and defect detection of nuclear fuel based on the line scan

Graphite balls are used as matrix materials of nuclear fuel. To ensure the safe use of nuclear energy, it is necessary to detect the surface defects of graphite balls to screen out the balls that do not meet safety requirements. However, due to the small surface defects and complex background of graphite balls, the detection accuracy based on traditional detection methods is difficult to meet engineering requirements. To solve the above problems, this paper designs an image acquisition device for graphite balls based on line scanning to collect spherical images and designs an improved algorithm based on the U2-Netp network to solve the problem of low accuracy of defect detection. Firstly, the algorithm replaces the up-sampling in the RSU module with cubic linear interpolation to enhance the smoothness of the image. Secondly, spatial channel attention and multi-feature fusion modules are built between the encoder and the decoder to improve the accuracy of the model to obtain interesting information. Finally, the prediction output of Sigmoid and Threshold fusion is designed to solve the problems caused by the gradient disappearance and the uneven polarity of positive and negative samples. Tested on the actual collected data set, the improved U2-Netp algorithm improved the average accuracy of detecting surface defects on graphite balls by 8.1% compared to the original U2-Netp algorithm, reaching 85.3%.

Hybrid channel and spatial attention-UNet for skin lesion segmentation

<span>Melanoma is a type of skin cancer which has affected many lives globally. The American Cancer Society research has suggested that it a serious type of skin cancer and lead to mortality but it is almost 100% curable if it is detected and treated in its early stages. Currently automated computer vision-based schemes are widely adopted but these systems suffer from poor segmentation accuracy. To overcome these issue, deep learning (DL) has become the promising solution which performs extensive training for pattern learning and provide better classification accuracy. However, skin lesion segmentation is affected due to skin hair, unclear boundaries, pigmentation, and mole. To overcome this issue, we adopt UNet based deep learning scheme and incorporated attention mechanism which considers low level statistics and high-level statistics combined with feedback and skip connection module. This helps to obtain the robust features without neglecting the channel information. Further, we use channel attention, spatial attention modulation to achieve the final segmentation. The proposed DL based scheme is instigated on publically available dataset and experimental investigation shows that the proposed Hybrid Attention UNet approach achieves average performance as 0.9715, 0.9962, 0.9710.</span>