Residual Attention Network Research Articles

Enhanced Quasi-Meshing Hotspot Effect Integrated Embedded Attention Residual Network for Culture-Free SERS Accurate Determination of Fusarium Spores

Enhanced Quasi-Meshing Hotspot Effect Integrated Embedded Attention Residual Network for Culture-Free SERS Accurate Determination of Fusarium Spores

Diabetic retinopathy grading based on multi-scale residual network and cross-attention module

Diabetic retinopathy (DR) is a severe effect of diabetes mellitus that mainly impacts the retinal tissue and carries a high risk of blindness. Ophthalmologists face challenges in assessing the severity of DR due to its complexity and time constraints. Consequently, there is an urgent need for the development of automated methods that employ retinal fundus images to detect DR. This study introduces a novel deep learning architecture that utilizes a multi-scale residual attention block (MSRAB) and a cross-attention block (CrAB) for DR grading. The proposed MSRAB employs a convolutional neural network (CNN) with diverse dilation rates to expand its field of view. MSRAB adaptively concentrate on pertinent features and incorporates a residual attention network to improve grading performance. Integrating the residual attention network enables MSRAB to prioritize critical characteristics in retinal fundus images and enhance performance. Similarly, the CrAB integrates channel and spatial attention mechanisms to capture inter-channel interactions and spatial dependencies within the input features. This comprehensive methodology enables the model to concentrate more efficiently on critical regions to discriminate irrelevant features and to capture interrelations across different channels and spatial regions. The proposed model employs a pre-trained backbone network to extract local and global features to capture complex features for precisely identifying DR. This process enhances model efficiency, which is helpful in methods with limited data and processing limitations. The suggested methodology surpasses the existing methodologies in accuracy, recall, precision, and Area Under the Curve.

Intelligent road surface state recognition method based on multi-layer attention residual network

Abstract Data-driven road surface state recognition enhances the efficiency and accuracy of road management, contributing to increased safety and reliability in road traffic. However, traditional machine learning and deep learning-based road surface state recognition typically rely on extensive data for model training, making it challenging to adapt to complex tasks in diverse scenarios. Therefore, this paper proposes a Multi-layer Attention Residual Network (MARN)-based intelligent road surface state recognition method. First, a Residual Convolutional Neural Network (ResNet) is constructed as the backbone model of MARN to mitigate the gradient vanishing problem, allowing the network to extract deeper features. Subsequently, an adaptive multi-layer attention mechanism is introduced in each convolutional layer, enabling adaptive weighting of each feature channel in the dataset to enhance the model’s focus on different features for better feature extraction. Furthermore, a cosine annealing learning rate adjuster is designed to improve the accuracy, robustness, and convergence during the model training process. Finally, the proposed MARN is validated using an image dataset containing six different road surface states. Comparative studies are conducted on the recognition accuracy of the proposed MARN, original ResNet, Visual Geometry Group network (VGG16), and Convolutional Neural Network (CNN). The impact of different batch sizes on the convergence speed of road surface state recognition under MARN is also analyzed. Results demonstrate that MARN achieves a training set accuracy of over 95%, surpassing VGG16 and CNN with accuracies below 85%. Compared to ResNet, MARN exhibits a 1.3% higher training set accuracy and a 0.25 lower validation set loss, showcasing superior accuracy and robustness in road surface state recognition.

A novel intelligent fault diagnosis method for gearbox based on multi-dimensional attention denoising convolution.

In the field of intelligent fault diagnosis, particularly concerning rotating machinery, convolutional neural networks (CNNs) face significant challenges when applied to real industrial vibration data. These data are not only contaminated by various types of noise but also exhibit fault features that vary across different scales. Consequently, the effective suppression of extraneous noise and accurate extraction of multi-scale fault features are crucial issues. To address these challenges, this study proposes a novel deep neural network framework, termed the Multidimensional Fusion Residual Attention Network (MFRANet), for gearbox fault diagnosis. The MFRANet employs a multi-scale deep separable convolution module to thoroughly investigate the fundamental characteristics of the original vibration signals in both the time and time-frequency domains. To enhance the detailed analysis of diagnostic data and mitigate the risks of overfitting and noise interference, an efficient residual channel attention module is incorporated to weight and denoise the feature maps. Additionally, an external attention module is introduced to create implicit connections between the denoised multi-scale feature maps and to highlight potential correlations within the sample data, thereby improving the accuracy of fault diagnosis. Experimental evaluations on a gearbox fault dataset demonstrate that the proposed method surpasses several benchmark and state-of-the-art techniques in terms of diagnostic performance, exhibiting robust noise resilience across various noise levels. This indicates enhanced reliability and accuracy in gearbox fault diagnosis, providing an innovative and efficient solution for fault diagnosis in rotating machinery. The study underscores the contributions of artificial intelligence through the innovative structure of the method and the integration of advanced deep learning modules, while its engineering application is evidenced by addressing practical challenges in rotating machinery fault diagnosis. This work meets the urgent need for reliable diagnostic methods in industrial environments.

Design of pattern image generation technique based on self-attentive residual conditional generative adversarial network

Image generation techniques have made remarkable progress in the digital image processing and computer vision. However, traditional generation models cannot meet the complexity and diversity requirements of patterned images. In view of this, the study aims to enhance the quality of generated pattern images, which uses improved residual block, and introduces a self-attention mechanism to compute the weight parameters of the input features to enhance the accuracy. Comparing with three image generation models, the research model shows lower Frechette initial distance, which is better than the other three methods, and the average Frechette initial distance values in the four scenes are 175.23, 176.41, 174.41, and 165.23. Generated mouths and eyes: the average values of Frechette initial distances reach 98.23 and 97.24, respectively. For emotion classification, the Frechette initial distance averages for sad, excited, and calm emotions were 82.34, 75.63, and 70.21, respectively. The model was trained up to 2500 iterations, and the loss value was reduced to 0.54, with an accuracy of 98.23%, confirming its effectiveness and high performance. The self attention residual network enhances the model's ability to capture image details, effectively improving the quality and accuracy of image generation, and providing a new technological path for radiation imaging data processing and analysis in radiation science.

An unsupervised transfer learning gear fault diagnosis method based on parameter-optimized VMD and residual attention networks

An unsupervised transfer learning gear fault diagnosis method based on parameter-optimized VMD and residual attention networks

Research on Fraud Detection Method of Financial Data of Listed Companies Based on HMCRAN

This study investigates the application of deep learning technology to improve accuracy and efficiency in detecting financial fraud in listed companies. Using data from the China Economic and Financial Research Database on Main Board A-share listed companies with disclosed fraudulent behaviors, a hybrid multi-channel residual attention network (HMCRAN) was constructed, combining multi-channel residual networks and transformers. The study involved feature extraction, data preprocessing, model training, and performance evaluation, validating the superiority of the HMCRAN model for financial fraud detection. Experimental results demonstrate that the model excels in handling high-dimensional financial data and capturing long-term dependencies, significantly improving detection accuracy and robustness. The HMCRAN model outperforms traditional machine learning methods across various evaluation metrics, demonstrating its high practical application value.

Complementary Parallel Residual Attention Network for Single Image Deraining

Complementary Parallel Residual Attention Network for Single Image Deraining

DLRA-Net: Deep Local Residual Attention Network with Contextual Refinement for Spectral Super-Resolution

DLRA-Net: Deep Local Residual Attention Network with Contextual Refinement for Spectral Super-Resolution

ScDRMAE: integrating masked autoencoder with residual attention networks to leverage omics feature dependencies for accurate cell clustering.

Cell clustering is foundational for analyzing the heterogeneity of biological tissues using single-cell sequencing data. With the maturation of single-cell multi-omics sequencing technologies, we can integrate multiple omics data to perform cell clustering, thereby overcoming the limitations of insufficient information from single omics data. Existing methods for cell clustering often only consider the differences in data patterns during the analysis of multi-omics data, but the dependencies between omics features of different cell types also significantly influence cell clustering. Moreover, the high dropout rates in scRNA-seq and scATAC-seq data can impact the performance of cell clustering. We propose a cell clustering model based on a masked autoencoder, scDRMAE. Utilizing a masking mechanism, scDRMAE effectively learns the relationships between different features and imputes false zeros caused by dropout events. To differentiate the importance of various omics data in cell clustering, we dynamically adjust the weights of different omics data through an attention mechanism. Finally, we use the K-means algorithm for cluster analysis of the fused multi-omics data. On commonly used sets of 15 multi-omics datasets, our method demonstrates superior cell clustering performance on multiple metrics compared to other computational methods. In addition, when datasets exhibit varying degrees of dropout noise, our method shows better performance and stronger stability on multiple metrics compared to other methods. Moreover, by analyzing the cell clusters classified by scDRMAE, we identified several biologically significant biomarkers that have been validated, further confirming the effectiveness of scDRMAE in cell clustering from a biological perspective.

Sensitivity Matrix Update Method for Electrical Resistance Tomography Based on Error-Constrained Cross Fusion Residual Attention Network

Abstract Electrical resistance tomography (ERT) is an imaging technique of conductivity distribution. The image reconstruction algorithm based on the empty field sensitivity matrix is widely used because it provides an approximate solution to the complex ERT inverse problems. However, due to the soft field effect, the sensitivity matrix changes with the media distribution, leading to significant errors in image reconstruction. To address this issue, an error-constrained deep learning scheme for bubble flow, namely cross fusion residual attention network (CFRA-Net) is designed to update the sensitivity matrix during gas-liquid-solid fluidized bed operation. CFRA-Net employs a multi-head self-attention mechanism to enhance bubble features in boundary measurements, a multilevel cross fusion module to fuse empty field strength and boundary measurement information, and a novel serial-channel residual spatial attention block to boost the feature extraction capability of the model. A weighted loss function is designed to give more weight to the gas phase distribution region. Additionally, this paper establishes a dataset for gas-liquid-solid three-phase flow, considering background field conductivity variations. Validation and reliability of the proposed method are assessed through ablation, comparison, and noise experiments. The experimental results demonstrate that CFRA-Net achieves rapid updates of the sensitivity matrix, high imaging accuracy, and robust noise immunity.

A lightweight parallel attention residual network for tile defect recognition

In modern industrial production, permanent magnet motors are an indispensable part of industrial manufacturing. The quality of the magnetic tiles directly affects the working performance of the permanent magnet motors, making the detection of defects on the surface of magnetic tiles critically important. However, due to the small size of defects on the tile image and the reflectivity of the defective surface, the details of image characteristics are not prominently acquired.These problems bring a lot of difficulties for the recognition of magnetic tile defects. In this paper, a magnetic tile defect detection method is proposed for the probAlems of unclear image features and small defects. First, the image is processed using linear variation to enhance the image detail features. Then, by introducing the inverted bottleneck block structure in MobileNetV2, the Attention Parallel Residual Convolution Block (APR) is proposed, and the Lightweight Parallel Attention Residual Network (LPAR-Net) is built. In APR Block, 7 × 7 convolution is introduced so that the model can extract spatial features from a larger range, and weighted fusion of input images by residual structure. In addition, in this paper, CBAM is improved, split into two parts and inserted into APR Block. Finally, the mainstream image classification models and the LPAR-Net proposed in this paper are used for comparison, respectively. The experimental results show that the method achieves 93.63% accuracy on the adopted dataset, which is better than the existing mainstream image classification network models DenseNet, MobileNet, ConvNext and so on. In addition, this paper introduces a strip steel surface defect dataset and compares it with the above image classification model, which verifies that the detection method proposed in this paper still has strong recognition capability.

Classification of infection grade for anthracnose in mango leaves under complex background based on CBAM-DBIRNet

Anthracnose is one of the most harmful fungal diseases in mango orchards. Accurate and rapid detection of anthracnose infection grade for mango leaves in complex natural environments is a prerequisite of precise variable pesticide spraying by robots in intelligent orchards. In this paper, A lightweight double branch inverted residual attention network (CBAM-DBIRNet) model was proposed by combining the inverted residual network and the attention mechanism, which was used to detect the infection grade of anthracnose for mango leaves in natural environment. The model designed two parallel residual branch structures to extract multi-scale features of mango leaves with different infection grades through different dilation coefficients. The inverted residual structure and depthwise separable convolution were used to reduce the complexity of the model. The CBAM attention module was integrated into the model to suppress the interference of complex background information. The mango leaves collected in the natural orchard were used for experiments. The results showed that the accuracy, recall and F1 of the CBAM-DBIRNet model for the classification of anthracnose infection grade were 98.42 %, 97.80 % and 97.89 %, respectively. The model size and parameter were 0.64 MB and 0.15 M, respectively. This study provides technical support for smart orchard management and robotic precision variable application operations.

Multi-timescale attention residual shrinkage network with adaptive global-local denoising for rolling-bearing fault diagnosis

In actual engineering scenarios, bearing fault signals are inevitably overwhelmed by strong background noise from various sources. However, most deep-learning-based diagnostic models tend to broaden the feature extraction scale to extract rich fault features for bearing-fault identification under noise interference, with little attention paid to multi-timescale discriminative feature mining with adaptive noise rejection, which affects the diagnostic performance. Thus, a multi-timescale attention residual shrinkage network with adaptive global-local denoising (AMARSN) was proposed for rolling-bearing fault diagnosis by learning discriminative multi-timescale fault features from signals and fully eliminating noise components in the multi-timescale fault features. First, a multi-timescale attention learning module (MALMod) was developed to capture multi-timescale fault features and enhance their discriminability under noise interference. Subsequently, an adaptive global-local denoising module (AGDMod) was constructed to fully eliminate noise in multiscale fault features by constructing specific global-local denoising thresholds and designing an adaptive smooth soft thresholding function. Finally, end-to-end bearing fault diagnosis tasks were realized using a softmax classifier located at the end of the AMARSN. The AMARSN was validated using two bearing datasets. The extensive results demonstrated that the AMARSN can mine more effective fault features from signals and achieve average diagnostic accuracies of 85.24% and 80.09% under different noise with different levels.

ERABiLNet: enhanced residual attention with bidirectional long short-term memory

Alzheimer’s Disease (AD) causes slow death in brain cells due to shrinkage of brain cells which is more prevalent in older people. In most cases, the symptoms of AD are mistaken as age-related stresses. The most widely utilized method to detect AD is Magnetic Resonance Imaging (MRI). Along with Artificial Intelligence (AI) techniques, the efficacy of identifying diseases related to the brain has become easier. But, the identical phenotype makes it challenging to identify the disease from the neuro-images. Hence, a deep learning method to detect AD at the beginning stage is suggested in this work. The newly implemented “Enhanced Residual Attention with Bi-directional Long Short-Term Memory (Bi-LSTM) (ERABi-LNet)” is used in the detection phase to identify the AD from the MRI images. This model is used for enhancing the performance of the Alzheimer’s detection in scale of 2–5%, minimizing the error rates, increasing the balance of the model, so that the multi-class problems are supported. At first, MRI images are given to “Residual Attention Network (RAN)”, which is specially developed with three convolutional layers, namely atrous, dilated and Depth-Wise Separable (DWS), to obtain the relevant attributes. The most appropriate attributes are determined by these layers, and subjected to target-based fusion. Then the fused attributes are fed into the “Attention-based Bi-LSTM”. The final outcome is obtained from this unit. The detection efficiency based on median is 26.37% and accuracy is 97.367% obtained by tuning the parameters in the ERABi-LNet with the help of Modified Search and Rescue Operations (MCDMR-SRO). The obtained results are compared with ROA-ERABi-LNet, EOO-ERABi-LNet, GTBO-ERABi-LNet and SRO-ERABi-LNet respectively. The ERABi_LNet thus provides enhanced accuracy and other performance metrics compared to such deep learning models. The proposed method has the better sensitivity, specificity, F1-Score and False Positive Rate compared with all the above mentioned competing models with values such as 97.49%.97.84%,97.74% and 2.616 respective;y. This ensures that the model has better learning capabilities and provides lesser false positives with balanced prediction.

Exploiting histopathological imaging for early detection of lung and colon cancer via ensemble deep learning model

Cancer seems to have a vast number of deaths due to its heterogeneity, aggressiveness, and significant propensity for metastasis. The predominant categories of cancer that may affect males and females and occur worldwide are colon and lung cancer. A precise and on-time analysis of this cancer can increase the survival rate and improve the appropriate treatment characteristics. An efficient and effective method for the speedy and accurate recognition of tumours in the colon and lung areas is provided as an alternative to cancer recognition methods. Earlier diagnosis of the disease on the front drastically reduces the chance of death. Machine learning (ML) and deep learning (DL) approaches can accelerate this cancer diagnosis, facilitating researcher workers to study a vast majority of patients in a limited period and at a low cost. This research presents Histopathological Imaging for the Early Detection of Lung and Colon Cancer via Ensemble DL (HIELCC-EDL) model. The HIELCC-EDL technique utilizes histopathological images to identify lung and colon cancer (LCC). To achieve this, the HIELCC-EDL technique uses the Wiener filtering (WF) method for noise elimination. In addition, the HIELCC-EDL model uses the channel attention Residual Network (CA-ResNet50) model for learning complex feature patterns. Moreover, the hyperparameter selection of the CA-ResNet50 model is performed using the tuna swarm optimization (TSO) technique. Finally, the detection of LCC is achieved by using the ensemble of three classifiers such as extreme learning machine (ELM), competitive neural networks (CNNs), and long short-term memory (LSTM). To illustrate the promising performance of the HIELCC-EDL model, a complete set of experimentations was performed on a benchmark dataset. The experimental validation of the HIELCC-EDL model portrayed a superior accuracy value of 99.60% over recent approaches.

A Multi-Scale Convolutional Hybrid Attention Residual Network for Enhancing Underwater Image and Identifying Underwater Multi-Scene Sea Cucumber

A Multi-Scale Convolutional Hybrid Attention Residual Network for Enhancing Underwater Image and Identifying Underwater Multi-Scene Sea Cucumber

Adaptive edge prior-based deep attention residual network for low-dose CT image denoising

Improving the diagnostic quality of low-dose CT (LDCT) images relies on effective noise removal. Recent advancements have highlighted the widespread use of deep residual networks for LDCT image denoising. These networks possess properties that aid in preserving image integrity and optimizing model performance. However, the denoising process faces challenges due to the complex patterns and intensity similarities between edge details and lesion regions. To address this issue, this paper introduces a novel approach called the cross-scale attentional residual network (RCANet), which utilizes an adaptive edge prior for LDCT image denoising. The adaptive edge prior enhances the denoising network’s ability to retain image boundary features and uniqueness. To distinguish subtle differences between LDCT image edge details and lesion areas, a cross-scale mapping dual-element module (CMDM) is designed to preserve rich edge texture information during model training. To prevent over-smoothing of denoised results, a compound loss function is proposed, combining MSE loss and multi-scale attention residual perception loss. To validate the effectiveness of the method, experiments were conducted on the AAPM-Mayo Clinic LDCT Grand Challenge dataset. The results demonstrate that RCANet surpasses state-of-the-art residual structure-based network models and performs comparably to leading denoising algorithms.

Lightweight pyramid attention residual network for intelligent fault diagnosis of machine under sharp speed variation

Deep learning based diagnostic methods have obtained great success in intelligent fault diagnosis of machines. However, most of the existing methods are developed for fault diagnosis task under stable working conditions, and cannot handle the data generated under sharp speed variation favorably. Besides, the large model parameters and high computing costs of these methods fail to meet the requirement of small memory and fast-precise prediction in practical applications. To address these challenges, a lightweight pyramid attention residual network (PARNet) is proposed in this paper. First, an efficient pyramid squeeze convolution module is proposed to capture multi-scale information from raw signals. Then, a soft squeeze-and-excitation attention is designed to establish long-range channel dependency, and fuse features of different scales. Based on the above improvements, a pyramid attention residual block is proposed as basic feature learning unit, and a concise CNN-based model PARNet is built using PAR block. Extensive experiments in two case studies are conducted to validate the effectiveness and adaptability of PARNet under different speed variation conditions. The test results demonstrate the superiority of the proposed method for machine fault diagnosis and reliability assessment in terms of recognition accuracy and model parameter complexity, compared with other advanced diagnostic methods.

Drug repositioning based on residual attention network and free multiscale adversarial training

BackgroundConducting traditional wet experiments to guide drug development is an expensive, time-consuming and risky process. Analyzing drug function and repositioning plays a key role in identifying new therapeutic potential of approved drugs and discovering therapeutic approaches for untreated diseases. Exploring drug-disease associations has far-reaching implications for identifying disease pathogenesis and treatment. However, reliable detection of drug-disease relationships via traditional methods is costly and slow. Therefore, investigations into computational methods for predicting drug-disease associations are currently needed.ResultsThis paper presents a novel drug-disease association prediction method, RAFGAE. First, RAFGAE integrates known associations between diseases and drugs into a bipartite network. Second, RAFGAE designs the Re_GAT framework, which includes multilayer graph attention networks (GATs) and two residual networks. The multilayer GATs are utilized for learning the node embeddings, which is achieved by aggregating information from multihop neighbors. The two residual networks are used to alleviate the deep network oversmoothing problem, and an attention mechanism is introduced to combine the node embeddings from different attention layers. Third, two graph autoencoders (GAEs) with collaborative training are constructed to simulate label propagation to predict potential associations. On this basis, free multiscale adversarial training (FMAT) is introduced. FMAT enhances node feature quality through small gradient adversarial perturbation iterations, improving the prediction performance. Finally, tenfold cross-validations on two benchmark datasets show that RAFGAE outperforms current methods. In addition, case studies have confirmed that RAFGAE can detect novel drug-disease associations.ConclusionsThe comprehensive experimental results validate the utility and accuracy of RAFGAE. We believe that this method may serve as an excellent predictor for identifying unobserved disease-drug associations.