Convolutional Attention Research Articles

Music source feature extraction based on improved attention mechanism and phase feature

Music source feature extraction is an important research direction in music information retrieval and music recommendation system. To extract the features of music sources more effectively, the study introduces the jump attention mechanism and combines it with the convolutional attention module. Also, a feature extraction module based on Unet + + and spatial attention module is proposed. In addition, the phase feature information of the mixed music signals is utilized to improve the network performance. Results showed that this model was studied to perform well in music source separation experiments of vocals and accompaniment. For vocal separation on the MIR-1K dataset, the model achieves 11.25 dB, 17.34 dB, and 13.83 dB for each metric, respectively. Meanwhile, for drum separation on the DSD100 dataset, the model achieves a median signal-to-source distortion ratio of 4.36 dB, which is 2.91 dB better than that of the Spectral Hierarchical Network model. For the separation of the bass sound and the human voice, the model's in the separation of bass and human voice, the median distortion ratio of the model is as high as 4.87 dB and 6.09 dB, which is better than that of the Spectral Hierarchical Network model. This indicates the significant performance advantages in feature extraction and separation of music sources, and it has important application values in music production and speech recognition.

CBAM-RIUnet: Breast Tumor Segmentation With Enhanced Breast Ultrasound and Test-Time Augmentation.

This study addresses the challenge of precise breast tumor segmentation in ultrasound images, crucial for effective Computer-Aided Diagnosis (CAD) in breast cancer. We introduce CBAM-RIUnet, a deep learning (DL) model for automated breast tumor segmentation in breast ultrasound (BUS) images. The model, featuring an efficient convolutional block attention module residual inception Unet, outperforms existing models, particularly excelling in Dice and IoU scores. CBAM-RIUnet follows the Unet structure with a residual inception depth-wise separable convolution, and incorporates a convolutional block attention module (CBAM) to eliminate irrelevant features and focus on the region of interest. Evaluation under three scenarios, including enhanced breast ultrasound (EBUS) and test-time augmentation (TTA), demonstrates impressive results. CBAM-RIUnet achieves Dice and IoU scores of 89.38% and 88.71%, respectively, showcasing significant improvements compared to state-of-the-art DL techniques. In conclusion, CBAM-RIUnet presents a highly effective and simplified DL model for breast tumor segmentation in BUS imaging.

MobilenetV2-RC: A lightweight network model for retinopathy classification in retinal OCT images

Abstract Retinopathy is an important ophthalmic disease that causes blindness in the elderly population. With the global old peoples increasing, the ophthalmic healthcare system is more and significant to pre-diagnosis. Optical Coherence Tomography (OCT) is considered the gold standard for ophthalmic treatment and diagnosis. OCT technologies and equipment continue to develop towards the intelligence and convenience for requirements of rapid diagnosis in the remote and poverty-stricken areas. We proposed an improved MobilenetV2 lightweight model for retinopathy classification (MobilenetV2-RC), which incorporates Spatial and Channel Reconstruction Convolution (SCConv) and the improved Convolutional Block Attention Module (CBAM) attention mechanism into the framework. Not only can it effectively limit feature redundancy to reduce model parameters, but also enhance the ability of feature representation to improve classification accuracy. The parameters of the proposed model are only 1.96 M with an overall accuracy of 98.96%, which is higher 3.32% than the original MobilenetV2. Compared with ResNet18, InceptionV3, and VGG16_BN, the overall accuracy is increased by 4.6%, 6.3%, and 3.9%, respectively. The test results of UCSD and Duke open-source datasets are more remarkable. Experimental results show that our proposed algorithm has strong reliability and generalization for the accurate classification of retinopathy, and a greater application prospect in the intelligent diagnosis of ophthalmology and mobile detection terminals.

ST2SI: Image Style Transfer via Vision Transformer using Spatial Interaction

While retaining the original content structure, image style transfer uses style image to render it to obtain stylized images with artistic features. Because the content image contains different detail units and the style image has various style patterns, it is easy to cause the distortion of the stylized image. We proposes a new Style Transfer based on Vision Transformer using Spatial Interaction (ST2SI), which takes advantage of Spatial Interactive Convolution (SIC) and Spatial Unit Attention (SUA) to further enhance the content and style representation, so that the encoder can not only better learn the features of the content domain and the style domain, but also maintain the structural integrity of the image content and the effective integration of style features. Concretely, the high-order spatial interaction ability of Spatial Interactive Convolution can capture complex style patterns, and Spatial Unit Attention can balance the content information of different detail units through the change of attention weight, thus solving the problem of image distortion. Comprehensive qualitative and quantitative experiments prove the efficacy of our approach.

Septic Arthritis Modeling Using Sonographic Fusion with Attention and Selective Transformation: a Preliminary Study.

Conventionally diagnosing septic arthritis relies on detecting the causal pathogens in samples of synovial fluid, synovium, or blood. However, isolating these pathogens through cultures takes several days, thus delaying both diagnosis and treatment. Establishing a quantitative classification model from ultrasound images for rapid septic arthritis diagnosis is mandatory. For the study, a database composed of 342 images of non-septic arthritis and 168 images of septic arthritis produced by grayscale (GS) and power Doppler (PD) ultrasound was constructed. In the proposed architecture of fusion with attention and selective transformation (FAST), both groups of images were combined in a vision transformer (ViT) with the convolutional block attention module, which incorporates spatial, modality, and channel features. Fivefold cross-validation was applied to evaluate the generalized ability. The FAST architecture achieved the accuracy, sensitivity, specificity, and area under the curve (AUC) of 86.33%, 80.66%, 90.25%, and 0.92, respectively. These performances were higher than using conventional ViT (82.14%) and significantly better than using one modality alone (GS 73.88%, PD 72.02%), with the p-value being less than 0.01. Through the integration of multi-modality and the extraction of multiple channel features, the established model provided promising accuracy and AUC in septic arthritis classification. The end-to-end learning of ultrasound features can provide both rapid and objective assessment suggestions for future clinic use.

Prune and Quantize Semantic Segmentation Network for Aerial Objects Recognition

Semantic segmentation of aerial and satellite images is crucial for applications in environmental management, urban planning, and traffic safety. While deep learning techniques with convolutional neural networks (CNNs) and attention mechanisms have achieved superior accuracy compared to traditional methods, they often struggle with model complexity and resource constraints. This paper introduces two novel techniques - pruning and quantization - to enhance both the performance and efficiency of semantic segmentation models for remote sensing images (RSIs). Pruning reduces model complexity by eliminating less significant weights, while quantization decreases memory usage by converting weights into a more compact format. We applied these techniques to the DeepLabV3+ model with ResNet18 and ResNet50 backbones and assessed their performance across multiple RSI datasets. Our results show that pruning and quantization effectively balance accuracy and computational efficiency, achieving a mean IoU of 81.24% with a memory footprint of 135.19 MB for pruning, and 81.04% mean IoU with a memory footprint of 33.79 MB for quantization on the ISPRS Vaihingen dataset. These methods offer a viable solution for deploying semantic segmentation models on resource-constrained hardware.

TPAFNet: Transformer-Driven Pyramid Attention Fusion Network for 3D Medical Image Segmentation.

The field of 3D medical image segmentation is witnessing a growing trend in the utilization of combined networks that integrate convolutional neural networks and transformers. Nevertheless, prevailing hybrid networks are confronted with limitations in their straightforward serial or parallel combination methods and lack an effective mechanism to fuse channel and spatial feature attention. To address these limitations, we present a robust multi-scale 3D medical image segmentation network, the Transformer-Driven Pyramid Attention Fusion Network, which is denoted as TPAFNet, leveraging a hybrid structure of CNN and transformer. Within this framework, we exploit the characteristics of atrous convolution to extract multi-scale information effectively, thereby enhancing the encoding results of the transformer. Furthermore, we introduce the TPAF block in the encoder to seamlessly fuse channel and spatial feature attention from multi-scale feature inputs. In contrast to conventional skip connections that simply concatenate or add features, our decoder is enriched with a TPAF connection, elevating the integration of feature attention between low-level and high-level features. Additionally, we propose a low-level encoding shortcut from the original input to the decoder output, preserving more original image features and contributing to enhanced results. Finally, the deep supervision is implemented using a novel CNN-based voxel-wise classifier to facilitate better network convergence. Experimental results demonstrate that TPAFNet significantly outperforms other state-of-the-art networks on two public datasets, indicating that our research can effectively improve the accuracy of medical image segmentation, thereby assisting doctors in making more precise diagnoses.

Boundary-aware convolutional attention network for liver segmentation in ultrasound images

Liver ultrasound is widely used in clinical practice due to its advantages of non-invasiveness, non-radiation, and real-time imaging. Accurate segmentation of the liver region in ultrasound images is essential for accelerating the auxiliary diagnosis of liver-related diseases. This paper proposes BACANet, a deep learning algorithm designed for real-time liver ultrasound segmentation. Our approach utilizes a lightweight network backbone for liver feature extraction and incorporates a convolutional attention mechanism to enhance the network's ability to capture global contextual information. To improve early localization of liver boundaries, we developed a selective large kernel convolution module for boundary feature extraction and introduced explicit liver boundary supervision. Additionally, we designed an enhanced attention gate to efficiently convey liver body and boundary features to the decoder to enhance the feature representation capability. Experimental results across multiple datasets demonstrate that BACANet effectively completes the task of liver ultrasound segmentation, achieving a balance between inference speed and segmentation accuracy. On a public dataset, BACANet achieved a DSC of 0.921 and an IOU of 0.854. On a private test dataset, BACANet achieved a DSC of 0.950 and an IOU of 0.907, with an inference time of approximately 0.32 s per image on a CPU processor.

A novel domain independent scene text localizer

Text localization across multiple domains is crucial for applications like autonomous driving and tracking marathon runners. This work introduces DIPCYT, a novel model that utilizes Domain Independent Partial Convolution and a Yolov5-based Transformer for text localization in scene images from various domains, including natural scenes, underwater, and drone images. Each domain presents unique challenges: underwater images suffer from poor quality and degradation, drone images suffer from tiny text and loss of shapes, and scene images suffer from arbitrarily oriented, shaped text. Additionally, license plates in drone images may not provide rich semantic information compared to other text types due to loss of contextual information between characters. To tackle these challenges, DIPCYT employs new partial convolution layers within Yolov5 and integrates Transformer detection heads with a novel Fourier Positional Convolutional Block Attention Module (FPCBAM). This approach leverages common text properties across domains, such as contextual (global) and spatial (local) relationships. Experimental results demonstrate that DIPCYT outperforms existing methods, achieving F-scores of 0.90, 0.90, 0.77, 0.85, 0.85, and 0.88 on Total-Text, ICDAR 2015, ICDAR 2019 MLT, CTW1500, Drone, and Underwater datasets, respectively.

An ultra lightweight neural network for automatic modulation classification in drone communications

Unmanned aerial vehicle (UAV)-assisted communication based on automatic modulation classification (AMC) technology is considered an effective solution to improve the transmission efficiency of wireless communication systems, as it can adaptively select the most suitable modulation method according to the current communication environment. However, many existing deep learning (DL)-based AMC methods cannot be directly applied to UAV platform with limited computing power and storage space, because of the contradiction between accuracy and efficiency. This paper mainly studies the lightweight of DL-based AMC networks to improve adaptability in resource-constrained scenarios. To address this challenge, we propose an ultra-lightweight neural network (ULNN). This network incorporates a lightweight convolutional structure, attention mechanism, and cross-channel feature fusion technique. Additionally, we introduce data augmentation (DA) based on signal phase offsets during the model training process, aimed at improving the model’s generalization ability and preventing overfitting. Through experimental validation on the public dataset RML2016.10 A, the ULNN we proposed achieves an average precision of 62.83% with only 8815 parameters and reaches a peak classification accuracy of 92.11% at SNR = 10 dB. The experimental results show that ULNN can achieve high recognition accuracy while keeping the model lightweight, and is suitable for UAV platform with limited resources.

An Improved Instance Segmentation Method for Complex Elements of Farm UAV Aerial Survey Images.

Farm aerial survey layers can assist in unmanned farm operations, such as planning paths and early warnings. To address the inefficiencies and high costs associated with traditional layer construction, this study proposes a high-precision instance segmentation algorithm based on SparseInst. Considering the structural characteristics of farm elements, this study introduces a multi-scale attention module (MSA) that leverages the properties of atrous convolution to expand the sensory field. It enhances spatial and channel feature weights, effectively improving segmentation accuracy for large-scale and complex targets in the farm through three parallel dense connections. A bottom-up aggregation path is added to the feature pyramid fusion network, enhancing the model's ability to perceive complex targets such as mechanized trails in farms. Coordinate attention blocks (CAs) are incorporated into the neck to capture richer contextual semantic information, enhancing farm aerial imagery scene recognition accuracy. To assess the proposed method, we compare it against existing mainstream object segmentation models, including the Mask R-CNN, Cascade-Mask, SOLOv2, and Condinst algorithms. The experimental results show that the improved model proposed in this study can be adapted to segment various complex targets in farms. The accuracy of the improved SparseInst model greatly exceeds that of Mask R-CNN and Cascade-Mask and is 10.8 and 12.8 percentage points better than the average accuracy of SOLOv2 and Condinst, respectively, with the smallest number of model parameters. The results show that the model can be used for real-time segmentation of targets under complex farm conditions.

Efficient Multi-View Graph Convolutional Network with Self-Attention for Multi-Class Motor Imagery Decoding.

Research on electroencephalogram-based motor imagery (MI-EEG) can identify the limbs of subjects that generate motor imagination by decoding EEG signals, which is an important issue in the field of brain-computer interface (BCI). Existing deep-learning-based classification methods have not been able to entirely employ the topological information among brain regions, and thus, the classification performance needs further improving. In this paper, we propose a multi-view graph convolutional attention network (MGCANet) with residual learning structure for multi-class MI decoding. Specifically, we design a multi-view graph convolution spatial feature extraction method based on the topological relationship of brain regions to achieve more comprehensive information aggregation. During the modeling, we build an adaptive weight fusion (Awf) module to adaptively merge feature from different brain views to improve classification accuracy. In addition, the self-attention mechanism is introduced for feature selection to expand the receptive field of EEG signals to global dependence and enhance the expression of important features. The proposed model is experimentally evaluated on two public MI datasets and achieved a mean accuracy of 78.26% (BCIC IV 2a dataset) and 73.68% (OpenBMI dataset), which significantly outperforms representative comparative methods in classification accuracy. Comprehensive experiment results verify the effectiveness of our proposed method, which can provide novel perspectives for MI decoding.

Wideband Adaptive Beamforming via Multiscale Channel Attention Convolutional Neural Network

Wideband Adaptive Beamforming via Multiscale Channel Attention Convolutional Neural Network

HybridCBAMNet: Enhancing time series binary classification with convolutional recurrent networks and attention mechanisms

The rapid advancement of Internet of Things technology and the increasing availability of big data have resulted in an exponential growth of time series data, highlighting a pressing need for effective classification methods. This study introduces HybridCBAMNet, a novel convolutional recurrent neural network model enhanced with recurrent networks and attention mechanisms for binary time series classification. The architecture integrates Conv1D-based feature extraction modules to extract relevant features, alongside attention enhancement modules and convolutional block attention modules. Additionally, bidirectional recurrent units are utilized to capture temporal dependencies and contextual information in both forward and backward directions. The model achieves top F1-scores in seven out of thirty-six binary classification tasks, significantly surpassing the performance of fourteen existing state-of-the-art models from the UCR archive. These results demonstrate that HybridCBAMNet not only enhances classification accuracy but also improves the model generalization capabilities, contributing valuable insights to the field of time series analysis.

LungXpertAI: A deep multi-task learning model for chest CT scan analysis and COVID-19 detection

Addressing the urgent need for accurate COVID-19 identification and lung infection segmentation in CT scans, our study introduces LungXpertAI, a novel Multi-Task Learning architecture. Previous studies on multi-task structures have been limited in providing detailed insights into their architectures. Addressing these details is crucial for enhancing the performance of multi-task structures. Our proposed multi-task structure, based on the U-Net architecture, incorporates a shared encoder, a specialized segmentation decoder, and a multi-layer perceptron for classification. In the proposed approach, image processing algorithms, including histogram equalization, median filtering, and morphological operations have been employed in the pre-processing stage of input images. Also, by combining these algorithms in pairs, an attempt has been made to enhance the performance of tasks. Initial preprocessing and the integration of the Convolutional Block Attention Module (CBAM) contribute to effective feature extraction. Post-processing with Conditional Random Field (CRF) further refines segmentation outputs. CRF considers spatial dependencies within the segmentation results, contributing to more precise delineation of COVID-19-affected areas in CT scans. Evaluation on using dataset demonstrates the effectiveness of pairwise combinations of image processing algorithms, achieving superior results with a segmentation accuracy of 95.66 % and a classification accuracy of 95.62 %. The incorporation of the CBAM module improves classification accuracy to 96.22 %. Moreover, Integrating CBAM and CRF significantly improves segmentation accuracy to 96.51 %. The application of our proposed method steps to U-Net++ and ResUnet showcases their potential for enhancing multi-task structures. This study establishes new COVID-19 detection standards, promising progress in medical image analysis.

Short-term power load forecasting based on Seq2Seq model integrating Bayesian optimization, temporal convolutional network and attention

Power load forecasting is of great significance to the electricity management. However, extant research is insufficient in comprehensively combining data processing and further optimization of existing prediction models. Therefore, this paper propose an improved power load prediction methods from two aspects: data processing and optimization of Sequence to Sequence (Seq2Seq) model. Firstly, in the data processing, Extreme Gradient Boosting (XGBoost) is adopted to eliminate the redundant features for feature extraction. Meanwhile, Successive Variational Mode Decomposition (SVMD) is employed to solve the unsteadiness and nonlinearities present in electricity data during the decomposition process. Secondly, the Seq2Seq model is selected and improved with a variety of machine learning methods. Specifically, input data features are extracted using Convolutional Neural Networks (CNN), enhancing the decoder's focus on vital information with the Attention mechanism (AM). Temporal Convolutional Network (TCN) serves as both the encoder and decoder of Seq2Seq, with further optimization of the model parameters through the Bayesian Optimization (BO) algorithm. Finally, the cases of two real power market datasets in Switzerland and Singapore illustrate the efficiency and superiority of proposed hybrid forecasting method. Through a comprehensive comparison and analysis with the other six models and four commonly used evaluation metrics, it is evident that the proposed method excels in performance, attaining the highest level of prediction accuracy, with the highest accuracy rate of 95.83 %. Consequently, it exhibits significant practical utility in the realm of power load forecasting.

AFC-Unet: Attention-fused full-scale CNN-transformer unet for medical image segmentation

In the field of medical image segmentation, although U-Net has achieved significant achievements, it still exposes some inherent disadvantages when dealing with complex anatomical structures and small targets, such as inaccurate target localization, blurry edges, and insufficient integration of contextual information. To address these challenges, this study proposes the Attention-Fused Full-Scale CNN-Transformer Unet (AFC-Unet), aiming to effectively overcome the limitations of traditional U-Net through multi-scale feature fusion, attention mechanisms, and CNN-Transformer hybrid modules. Specifically, we adopt an encoder–decoder architecture, incorporating full-scale feature block fusion and pyramid sampling modules to enhance the model’s ability to recognize fine to overall structural features by integrating cross-level multi-scale features. We propose the Multi-feature Fusion Attention Gates (MFAG) module, which focuses on and highlights discriminative information of potential lesions and key anatomical boundaries, effectively suppressing irrelevant background interference. We design a module Convolutional Hybrid Attention Transformer (CHAT) that integrates CNN and Transformer to address the shortcomings of traditional single models in handling long-range dependencies and global context understanding. Experimental results on three datasets of different scales demonstrate that the model’s segmentation performance for medical images surpasses state-of-the-art models, showcasing high generalization ability.

Tunnel lining quality detection based on the YOLO-LD algorithm

Tunnel lining defects, such as discontinuous reinforcing steel, concrete dehollowing, and incomplete pouring, can substantially undermine structural durability and stability. Addressing limitations such as strong subjectivity and low accuracy in traditional quality assessment methods, we introduce the YOLO-LD tunnel lining quality detection algorithm. This model is an adaptation of the original YOLOv7 algorithm, where the original feature pyramid network is substituted by an asymptotic feature pyramid network, and a convolutional block attention module is added subsequently to backbone extraction. Electromagnetic wave propagation is simulated using gprMax3.0, while the tunnel lining structure is imaged through ground-penetrating radar employing the finite-difference time-domain method. These simulations yield a comprehensive radar image dataset for tunnel lining quality evaluation. A performance comparison of YOLO-LD with four other established algorithms reveals its superiority in detecting the three aforementioned defects, yielding an mF1 score of 91.07 % and a mAP score of 94.13 %. The model demonstrates robust performance in comprehensive defect detection and generalization.

CCM-Net: Color compensation and coordinate attention guided underwater image enhancement with multi-scale feature aggregation

Due to the light scattering and wavelength absorption in water, underwater images exhibit blurred details, low contrast, and color deviation. Existing underwater image enhancement methods are divided into traditional methods and deep learning-based methods. Traditional methods either rely on scene prior and lack robustness, or are not flexible enough resulting in poor enhancement effects. Deep learning methods have achieved good results in the field of underwater image enhancement due to their powerful feature representation ability. However, these methods cannot enhance underwater images with various degradations because they do not consider the inconsistent attenuation of different color channels and spatial regions. In this paper, we propose a novel asymmetric encoder-decoder network for underwater image enhancement, called CCM-Net. Concretely, we first introduce the prior knowledge-based encoder, which includes color compensation (CC) modules and feature extraction modules that consist of depth-wise separable convolution and global-local coordinate attention (GLCA). Then, we design a multi-scale feature aggregation (MFA) module to integrate shallow, middle, and deep features. Finally, we deploy a decoder to reconstruct the underwater images with the extracted features. Extensive experiments on publicly available datasets demonstrate that our CCM-Net effectively improves the visual quality of underwater images and achieves impressive performance.

MFR-Net: A multi-feature fusion phase unwrapping method for different speckle noises

Phase unwrapping is a crucial step in laser interferometry for obtaining accurate physical measurement of object. To reduce the impact of speckle noise on wrapped phase during actual measurement and improve the subsequent measurement accuracy, a multi-feature fusion phase unwrapping method for different speckle noises named MFR-Net is proposed in this paper. The network is composed of a front-end multi-module filter processing layer and a back-end network with dilated convolution and coordinate attention mechanism. By reducing random phase differences introduced by different levels of noise, the network enhances its capability to extract spatial features such as gradient information between pixels under speckle noise, so that it successfully unwraps the wrapped phase with different speckle noises and accurately recovers the real phase information. Taking the wrapped phases with multiplicative speckle noise and additive random noise as dataset, the results of ablation and comparison experiments show that the MFR-Net has superior unwrapped results. Under three different levels of speckle noise, the average values of MSE, SSIM, PSNR and AU for MFR-Net are at least improved by 84.80 %, 10.99 %, 29.00 % and 7.72 %, respectively, compared to PDVQG, TIE, DLPU and VURNet algorithms. When the standard deviation of speckle noise varies continuously in the range [1.0, 2.0], the average values of four indexes reaches 0.12 rad, 0.91, 31.80 dB and 99.96 %, respectively, indicating the stronger robustness of MFR-Net. In addition, the phase step unwrapping is performed by MFR-Net. Compared to DLPU and VURNet, MFR-Net method reduced MSE by 80 % and 87.35 %, respectively, demonstrating the outstanding generalization capability. The proposed MFR-Net can realize the correct phase unwrapping under different speckle noises. It may be applied in laser interferometry applications such as digital holography and interferometric synthetic aperture radar.