Deep Residual Attention Network Research Articles

DRANetSplicer: A Splice Site Prediction Model Based on Deep Residual Attention Networks.

The precise identification of splice sites is essential for unraveling the structure and function of genes, constituting a pivotal step in the gene annotation process. In this study, we developed a novel deep learning model, DRANetSplicer, that integrates residual learning and attention mechanisms for enhanced accuracy in capturing the intricate features of splice sites. We constructed multiple datasets using the most recent versions of genomic data from three different organisms, Oryza sativa japonica, Arabidopsis thaliana and Homo sapiens. This approach allows us to train models with a richer set of high-quality data. DRANetSplicer outperformed benchmark methods on donor and acceptor splice site datasets, achieving an average accuracy of (96.57%, 95.82%) across the three organisms. Comparative analyses with benchmark methods, including SpliceFinder, Splice2Deep, Deep Splicer, EnsembleSplice, and DNABERT, revealed DRANetSplicer's superior predictive performance, resulting in at least a (4.2%, 11.6%) relative reduction in average error rate. We utilized the DRANetSplicer model trained on O. sativa japonica data to predict splice sites in A. thaliana, achieving accuracies for donor and acceptor sites of (94.89%, 94.25%). These results indicate that DRANetSplicer possesses excellent cross-organism predictive capabilities, with its performance in cross-organism predictions even surpassing that of benchmark methods in non-cross-organism predictions. Cross-organism validation showcased DRANetSplicer's excellence in predicting splice sites across similar organisms, supporting its applicability in gene annotation for understudied organisms. We employed multiple methods to visualize the decision-making process of the model. The visualization results indicate that DRANetSplicer can learn and interpret well-known biological features, further validating its overall performance. Our study systematically examined and confirmed the predictive ability of DRANetSplicer from various levels and perspectives, indicating that its practical application in gene annotation is justified.

Prediction of EV Charging Behavior using BOA-based Deep Residual Attention Network

In smart city applications, electric vehicles (EVs) are rapidly gaining popularity due to their ability to help cut down on carbon emissions. Numerous environmental conditions, including terrain, traffic, driving style, temperature, and so on, affect the amount of energy an EV needs to operate. However, the burden on power grid infrastructure from widespread EV deployment is one of the biggest obstacles. Smart scheduling algorithms can be used to handle the rising public charging demand. Scheduling algorithms can be improved using data-driven tools and procedures to study EV charging behaviour. Predictions of behaviour, including temperature, departure time, and energy requirements, have been the focus of research on past charging data. Weather, traffic, and surrounding events are all factors that have been mostly ignored but which could improve representations and predictions. The DRA-Net, or Deep Residual Attention Network, was developed by the researchers and is used to recognize EV charging patterns. To minimize data loss, the Res-Attention component utilized tighter connections and smaller convolutional kernels (3 x 3). In addition, an Artificial Butterfly Optimisation Algorithm (BOA) model is used to fine-tune the DRA-Net's hyper-parameters. We highlight the significance of traffic and weather info for charging behaviour predictions, and the study's experimental forecasts show a considerable improvement over prior work on the same dataset. The future of electric vehicle (EV) research has been mapped out thanks to in-depth study, and as a result, EVs will soon significantly impact the auto industry.

Deep residual attention network for human defecation prediction using bowel sounds

Deep residual attention network for human defecation prediction using bowel sounds

Super-resolution reconstruction of medical images based on deep residual attention network

Super-resolution reconstruction of medical images based on deep residual attention network

FlowRAU-Net: Accelerated 4D Flow MRI of Aortic Valvular Flows With a Deep 2D Residual Attention Network.

In this work, we propose a novel deep learning reconstruction framework for rapid and accurate reconstruction of 4D flow MRI data. Reconstruction is performed on a slice-by-slice basis by reducing artifacts in zero-filled reconstructed complex images obtained from undersampled k-space. A deep residual attention network FlowRAU-Net is proposed, trained separately for each encoding direction with 2D complex image slices extracted from complex 4D images at each temporal frame and slice position. The network was trained and tested on 4D flow MRI data of aortic valvular flow in 18 human subjects. Performance of the reconstructions was measured in terms of image quality, 3-D velocity vector accuracy, and accuracy in hemodynamic parameters. Reconstruction performance was measured for three different k-space undersamplings and compared with one state of the art compressed sensing reconstruction method and three deep learning-based reconstruction methods. The proposed method outperforms state of the art methods in all performance measures for all three different k-space undersamplings. Hemodynamic parameters such as blood flow rate and peak velocity from the proposed technique show good agreement with reference flow parameters. Visualization of the reconstructed image and velocity magnitude also shows excellent agreement with the fully sampled reference dataset. Moreover, the proposed method is computationally fast. Total 4D flow data (including all slices in space and time) for a subject can be reconstructed in 69 seconds on a single GPU. Although the proposed method has been applied to 4D flow MRI of aortic valvular flows, given a sufficient number of training samples, it should be applicable to other arterial flows.

Deep learning super‐resolution electron microscopy based on deep residual attention network

AbstractField‐emission scanning electron microscopy has become a fundamental research tool in the fields of medicine and materials science owing to its effectiveness. However, an inherent contradiction exists between the resolution of field‐emission scanning electron microscopy and its field‐of‐view. To solve this problem, we propose a deep learning‐based method for electron microscopy that can simultaneously obtain a large field‐of‐view and ultrahigh resolution. To solve the super‐resolution problem, a deep residual attention network is designed based on residual learning and the attention mechanism, wherein the backbone network employs several residual groups to effectively extract the features; moreover, attention groups append the backbone part to refine and fuse the features. Besides, a high‐frequency information retention module is added to acquire high‐frequency signals, acting as an effective complement to the deep residual attention network. Owing to the lack of super‐resolution datasets for electron microscopy, we created a microscopic butterfly wing dataset. In the experiments, MixUp was also applied to super‐resolution problem as a simple and effective data augmentation method, to provide more data to train the model. To evaluate the proposed method, we used standard and self‐made datasets with peak signal‐to‐noise ratio (PSNR) and structural similarity (SSIM) as criteria. The results show that the proposed model exhibits a state‐of‐the‐art performance with a PSNR of 25.12 dB and an SSIM of 0.64, and makes field‐emission scanning electron microscopy more practical and promising in the field of optical device optimization.

Novel multi‐scale deep residual attention network for facial expression recognition

Recently, the deep convolutional neural networks (CNNs) have shown great success and for facial expression recognition (FER). These CNN-based approaches have made breakthroughs in the accuracy by using deeper networks. As the amount of train data is very less, these models easily fall into the overfitting in the training. In this study, a novel multi-scale deep residual attention network (Ms-RAN) is proposed for FER. The proposed Ms-RAN is mainly based on the multi-scale residual attention unit, which consists of two different scale sub-units. Each sub-unit is composed of the convolutional layers, parametric rectified linear units (PReLUs), and the residual attention connection. By focusing on the relationship between channels and automatically learning the importance of different channel features, the proposed Ms-RAN can make the proposed model pay more attention to the most informative channel features, while suppressing those unimportant channel features. Owing to the unique design of Ms-RAN, the combination of various levels features can be enhanced in the proposed method, and valuable and different ranges of expressive information can also be provided for recognition. The experimental results demonstrate that the proposed method achieves superior performance than other state-of-the-art approaches on five databases, CK+, Oulu-CASIA, BU-3DFE, BP4D+, and MMI.

Learning Expression Features via Deep Residual Attention Networks for Facial Expression Recognition From Video Sequences

Facial expression recognition from video sequences is currently an interesting research topic in computer vision, pattern recognition, artificial intelligence, etc. Considering the problem of semantic gap between the extracted hand-designed features in affective videos and subjective emotions, recognizing facial expressions from video sequences is a challenging subject. To tackle this problem, this paper proposes a new method of facial expression recognition from video sequences via deep residual attention network. Firstly, due to the difference in the intensity of emotional representation of each local area in a facial image, deep residual attention networks are employed to learn high-level affective expression features for each frame of facial expression images in video sequences. The used deep residual attention networks integrate deep residual networks with a spatial attention mechanism. Then, average-pooling is performed to produce fixed-length global video-level feature representations. Finally, the global video-level feature representations are utilized as inputs of a multi-layer perceptron to conduct facial expression classification tasks in video sequences. Experimental results on two public video emotional datasets, i.e. BAUM-1s and RML, demonstrate the effectiveness of the proposed method.

A Study on Tuberculosis Classification in Chest X-ray Using Deep Residual Attention Networks.

The introduction of deep learning techniques for the computer-aided detection scheme has shed a light for real incorporation into the clinical workflow. In this work, we focus on the effect of attention in deep neural networks on the classification of tuberculosis x-ray images. We propose a Convolutional Block Attention Module (CBAM), a simple but effective attention module for feed-forward convolutional neural networks. Given an intermediate feature map, our module infers attention maps and multiplied it to the input feature map for adaptive feature refinement. It achieves high precision and recalls while localizing objects with its attention. We validate the performance of our approach on a standard-compliant data set, including a dataset of 4990 x-ray chest radiographs from three hospitals and show that our performance is better than the models used in previous work.