Multi-attention Mechanism Research Articles

Neural Machine Translation of Electrical Engineering Based on Vector Fusion

The development of neural machine translation has achieved a good translation effect on large-scale general corpora, but there are still many problems in the translation of low resources and specific fields. This paper studies the problem of machine translation in the field of electrical engineering and fuses the multi-layer vectors at the encoder side of the model. On this basis, the decoder unit of the translation model is improved, and a multi-attention mechanism translation model based on vector fusion is proposed, which improves the ability of the model to extract features and achieves a better translation effect on Chinese-English translation tasks. The experimental results show that the BLEU (bilingual evaluation understudy) value of the improved translation system in the field of electrical engineering has increased by 0.15–1.58 percentage points.

Pillar-Based 3D Object Detection from Point Cloud with Multiattention Mechanism

Object detection in point clouds is a critical component in most autonomous driving systems. In this paper, in order to improve the effectiveness of image feature extraction and the accuracy of detection of point clouds, a pillar-based 3D point cloud object detection algorithm with multiattention mechanism is proposed, which includes three attention mechanisms SOCA, SOPA, and SAPI. The results show that the recognition accuracy of the optimized algorithm for cars, pedestrians, and cyclists on KITTI dataset is significantly improved on the detection benchmarks of BEV and 3D. Despite using only LiDAR, our algorithm outperforms PointPillars, which is one of the state-of-the-art algorithms for 3D object detection, with respect to both 3D and BEV view KITTI benchmarks while maintaining a relatively competitive speed.

Point Cloud Registration Using Multiattention Mechanism and Deep Hybrid Features

Due to some unfavorable factors, how to accurately register point clouds is still a challenging task. In this article, an effective point cloud registration network is proposed with multiple attention mechanism and deep hybrid features. For the features obtained with a graph neural network, three attention modules, namely the spatial attention module, channel attention module, and self-geometric attention module, are utilized to mine various areas of regional information. An attention-based feature fusion module, which consists of three consecutive residual blocks, is devised to fuse the features from the three attention modules. Moreover, the capability of the network for correctly matching point clouds is enhanced, by using deep hybrid features to guide the correspondence search and the calculation of matching confidence. Experimental results on several widely used datasets demonstrate the effectiveness of the proposed point cloud registration network.

Grouped Multi-Attention Network for Hyperspectral Image Spectral-Spatial Classification

Deep learning has been a powerful tool for hyperspectral image (HSI) classification. However, it is still an open issue to effectively learn highly discriminative features from the HSI, due to the high-dimensionality and complex spectral-spatial characteristics. To settle this issue, we propose a new band-grouping guided multi-attention module for the performance promotion of spectral-spatial feature learning. First, based on the fact of high relevance between adjacent spectral bands and low dependencies across long-range ones, all the spectral bands are adaptively divided into multiple non-overlapping groups where relevant bands are included. The advantage is to reduce the spectral dimension and data complexity when processing and analyzing each group. Then, a multi-attention mechanism, which not only explore the intra-group salient information but also propagate the inter-group difference information, is embedded into the convolutional neural networks to learn group-specific spectral-spatial features. By emphasizing useful spectral/spatial information and squeezing useless information with attention mechanism, the severability of learned features is enhanced. Based on this module, a spectral-spatial classification network is built, named by grouped multi-attention network (GMA-Net). The GMA-Net contains a two-branch architecture, i.e., pixel-wise spectral feature learning and patch-wise spectral-spatial feature learning. Via fusing the features from two branches, the complementary and discriminative features provided by pixel-wise and patch-wise learning manner can be integrated to further boost classification performance. Experimental results demonstrate that the proposed method is superior than several state-of-the-art approaches. Codes are available at: https://github.com/luting-hnu.

CEMA-LSTM: Enhancing Contextual Feature Correlation for Radar Extrapolation Using Fine-Grained Echo Datasets

Accurate precipitation nowcasting can provide great convenience to the public so they can conduct corresponding arrangements in advance to deal with the possible impact of upcoming heavy rain. Recent relevant research activities have shown their concerns on various deep learning models for radar echo extrapolation, where radar echo maps were used to predict their consequent moment, so as to recognize potential severe convective weather events. However, these approaches suffer from an inaccurate prediction of echo dynamics and unreliable depiction of echo aggregation or dissipation, due to the size limitation of convolution filter, lack of global feature, and less attention to features from previous states. To address the problems, this paper proposes a CEMA-LSTM recurrent unit, which is embedded with a Contextual Feature Correlation Enhancement Block (CEB) and a Multi-Attention Mechanism Block (MAB). The CEB enhances contextual feature correlation and supports its model to memorize significant features for near-future prediction; the MAB uses a position and channel attention mechanism to capture global features of radar echoes. Two practical radar echo datasets were used involving the FREM and CIKM 2017 datasets. Both quantification and visualization of comparative experimental results have demonstrated outperformance of the proposed CEMA-LSTM over recent models, e.g., PhyDNet, MIM and PredRNN++, etc. In particular, compared with the second-ranked model, its average POD, FAR and CSI have been improved by 3.87%, 1.65% and 1.79%, respectively on the FREM, and by 1.42%, 5.60% and 3.16%, respectively on the CIKM 2017.

Data Augmentation and Few-Shot Change Detection in Forest Remote Sensing

Forest remote sensing change detection provides an important technical support for forest management decisions and analysis of forest disturbance factors. However, lack of data in specialized fields leads to the detection accuracy improved difficulty. In this study, a forest remote sensing change detection model in the context of few-shot learning is proposed. The proposed model achieves end-to-end change detection algorithm for forest scenes from two perspectives of data augmentation and updated few-shot algorithm. Firstly, forest fragments images are jointly generated by using feature extraction network and generative adversarial network. Then, forest fragments are blended into the original change detection dataset by Poisson blending method to achieve effective augmentation. Furthermore, the end-to-end change detection network is also updated using a few-shot learning. Additionally, Meta-learning module is added to the slow feature analysis algorithm based on the multi-attention mechanism to realize the detection effect improvements. The proposed model improves the F1 score from 86% to 91% on the two datasets. Moreover, it increases the F1 score by 6.52% on average.

MIGN: Multiscale Image Generation Network for Remote Sensing Image Semantic Segmentation

With the development of computer vision, the semantic segmentation of remote sensing images, which has become an important topic, has been utilized in various applications for image content analysis and understanding, such as urban planning, natural disaster monitoring, and land resource management. Many approaches have been proposed to address these problems. However, due to obvious differences in resolution, spatial structure, and semantics between remote sensing images and ordinary images, the semantic segmentation of remote sensing images is still challenging. In this paper, we propose a novel multiscale image generation network (MIGN) that can efficiently generate high-resolution segmentation results by considering both details and boundary information. In particular, a multi-attention mechanism method for semantic segmentation of remote sensing images is designed. The attention weight is calculated by capturing the interaction of cross dimensions in a two-branch structure, which can learn the underlying feature information and guarantee the performance of each pixel feature for final classification. We also propose an edge supervised module to ensure that the segmentation boundary has a more accurate performance. A multiscale image fusion algorithm based on the Bayes model is proposed to improve the accuracy of the segmentation module. The performance of our model is evaluated on the ISPRS Vaihingen and Potsdam datasets. The results show that our method is superior to the most advanced image segmentation methods in terms of MIoU and pixel accuracy.

Face frontalization with deep GAN via multi-attention mechanism

Face frontalization with deep GAN via multi-attention mechanism

Multitask Residual Shrinkage Convolutional Neural Network for Sleep Apnea Detection Based on Wearable Bracelet Photoplethysmography

Sleep apnea syndrome (SAS) is a common chronic respiratory disorder, which seriously harms human health. In order to realize the large-scale promotion of SAS detection, the SAS detection method based on wearable devices has attracted the attention of some researchers. However, in view of the poor signals collected by wearable devices and the less variation between signals of different classes (normal/SAS), the detection methods in past studies are often unsatisfactory. In this study, a wearable SAS detection method based on 1-D multitask multiattention residual shrinkage convolution neural network (1D-MMResSNet) model and cost-sensitive (CS) classifier was proposed. First, the photoplethysmography (PPG) sleep data of 92 subjects were collected by using wearable smart Bracelet devices. Second, a backbone network composed of the residual multiattention mechanism convolution block and residual shrinkage convolution block was proposed to realize feature selection and feature extraction for pulse rate variability signals. At the same time, the single supervised learning task of the network was improved, and a multitask learning method was proposed to enhance the ability of the network to mine subtle differences between data, thereby effectively learning the discriminative network features. Finally, the AdaCost cost-sensitive algorithm was introduced to alleviate the class imbalance problem in data samples. In segment detection, the accuracy, sensitivity, and specificity were 81.82%, 70.27%, and 85.81%, respectively; In individual detection, the accuracy, sensitivity, and specificity were 95.65%, 88.89%, and 97.30%, respectively. Experimental results show that the proposed model has excellent detection performance and is expected to be embedded in wearable devices.

A novel denoising method for CT images based on U-net and multi-attention

Reducing the radiation dose may lead to increased noise in medical computed tomography (CT), which can adversely affect the radiologists' judgment. Many efforts have been devoted to the denoising of low-dose CT (LDCT) images. However, it is often observed that denoised medical images usually lose some important clinical lesion edge information and may affect doctors’ clinical diagnosis. For a denoising neural network, it is expected that the neural network can well retain the detailed features and make the network more anthropomorphic, and to simulate the attention mechanism of observation, being a valuable feature of the thinking process of human brain. Based on U-network (U-Net) and multi-attention mechanism, a novel denoising method for medical CT images is proposed in this study. To obtain different feature information in CT images, three attention modules are proposed in our method. The local attention module is developed to localize the surrounding information of the feature map and calculate each pixel from the context extracted from the feature map. The multi-feature channel attention module can automatically learn and extract features, suppress some invalid information and add different weights to each channel in the feature map according to different tasks. The hierarchical attention module allows the deep neural network to extract a large amount of feature information. This study also introduces an enhanced learning module to learn and retain the detail information in the image by stacking multi-layer convolution layer, batch normalization (BN) layer and activation function layer to increase the network depth. Experimental studies are conducted, and comparisons with the state-of-the-art networks are made, and the results demonstrate that the developed method can effectively remove the noise in CT images and improve the image quality in the evaluation metrics of peak signal to noise ratio (PSNR) and structural similarity (SSIM). Our method achieved 34.7329 of PSNR and 0.9293 of SSIM for σ = 10 on the QIN_LUNG_CT dataset, and achieved 28.9163 of PSNR and 0.8602 of SSIM on the Mayo Clinic LDCT Grand Challenge dataset.

LACN: A lightweight attention-guided ConvNeXt network for low-light image enhancement

Images captured under low-light conditions usually have poor visual quality, and hence greatly reduce the accuracy of subsequent tasks such as image segmentation and detection. In the low-light image enhancement task, noises in the dark areas are generally amplified while the images’ brightness is enhanced. It should be pointed out that many deep learning methods cannot effectively suppress the noise at this stage and capture important feature information. To address the above problem, this paper proposes a Lightweight Attention-guided ConvNeXt Network (LACN) for low-light image enhancement. A novel Attention ConvNeXt Module (ACM) is first proposed by introducing a parameter-free attention module (i.e. SimAM) into the ConvNeXt backbone network. Then, a nontrivial lightweight network LACN based on a multi-attention mechanism is established through stacking two ACMs and fusing their features. In what follows, an improved hybrid attention mechanism, Selective Kernel Attention Module (SKAM), is adopted to effectively extract both global and local information. Such a module realizes the evaluation of lighting conditions for the whole image and the adaptive adjustment of the receptive field. Finally, through the feature fusion module, the features of different stages are aggregated to improve the ability of network to retain color information. Numerous experiments on low-light image enhancement are implemented via comparison with other state-of-the-art methods. Experiments show that the proposed method significantly improves the brightness and contrast of low-illumination images, preserves color information, and suppresses the generation of noises after image brightening.

HMA-Net: A deep U-shaped network combined with HarDNet and multi-attention mechanism for medical image segmentation.

Automatic segmentation of lesion, organ, and tissue from the medical image is an important part of medical image analysis, which are useful for improving the accuracy of disease diagnosis and clinical analysis. For skin melanomas lesions, the contrast ratio between lesions and surrounding skin is low and there are many irregular shapes, uneven distribution, and local and boundary features. Moreover, some hair covering the lesions destroys the local context. Polyp characteristics such as shape, size, and appearance vary at different development stages. Early polyps with small sizes have no distinctive features and could be easily mistaken for other intestinal structures, such as wrinkles and folds. Imaging positions and illumination conditions would alter polyps' appearance and lead to no visible transitions between polyps and surrounding tissue. It remains a challenging task to accurately segment the skin lesions and polyps due to the high variability in the location, shape, size, color, and texture of the target object. Developing a robust and accurate segmentation method for medical images is necessary. To achieve better segmentation performance while dealing with the difficulties above, a U-shape network based on the encoder and decoder structure is proposed to enhance the segmentation performance in target regions. In this paper, a novel deep network of the encoder-decoder model that combines HarDNet, dual attention (DA), and reverse attention (RA) is proposed. First, HarDNet68 is employed to extract the backbone features while improving the inference speed and computational efficiency. Second, the DA block is adopted to capture the global feature dependency in spatial and channel dimensions, and enrich the contextual information on local features. At last, three RA blocks are exploited to fuse and refine the boundary features to obtain the final segmentation results. Extensive experiments are conducted on a skin lesion dataset which consists of ISIC2016, ISIC2017, and ISIC 2018, and a polyp dataset which consists of several public datasets, that is, Kvasir, CVC-ClinicDB, CVC-ColonDB, ETIS, Endosece. The proposed method outperforms some state-of-art segmentation models on the ISIC2018, ISIC2017, and ISIC2016 datasets, with Jaccard's indexes of 0.846, 0.881, and 0.894, mean Dice coefficients of 0.907, 0.929, and 03939, precisions of 0.908, 0.977, and 0.968, and accuracies of 0.953, 0.975, and 0.972. Additionally, the proposed method also performs better than some state-of-art segmentation models on the Kvasir, CVC-ClinicDB, CVC-ColonDB, ETIS, and Endosece datasets, with mean Dice coefficients of 0.907, 0.935, 0.716, 0.667, and 0.887, mean intersection over union coefficients of 0.850, 0.885, 0.644, 0.595, and 0.821, structural similarity measures of 0.918, 0.953, 0.823, 0.807, and 0.933, enhanced alignment measures of 0.952, 0.983, 0.850, 0.817, and 0.957, mean absolute errors of 0.026, 0.007, 0.037, 0.030, and 0.009. The proposed deep network could improve lesion segmentation performance in polyp and skin lesion images. The quantitative and qualitative results show that the proposed method can effectively handle the challenging task of segmentation while revealing the great potential for clinical application.

Recognition of terminal buds of densely-planted Chinese fir seedlings using improved YOLOv5 by integrating attention mechanism.

Accurate and timely information on the number of densely-planted Chinese fir seedlings is essential for their scientific cultivation and intelligent management. However, in the later stage of cultivation, the overlapping of lateral branches among individuals is too severe to identify the entire individual in the UAV image. At the same time, in the high-density planting nursery, the terminal bud of each seedling has a distinctive characteristic of growing upward, which can be used as an identification feature. Still, due to the small size and dense distribution of the terminal buds, the existing recognition algorithm will have a significant error. Therefore, in this study, we proposed a model based on the improved network structure of the latest YOLOv5 algorithm for identifying the terminal bud of Chinese fir seedlings. Firstly, the micro-scale prediction head was added to the original prediction head to enhance the model’s ability to perceive small-sized terminal buds. Secondly, a multi-attention mechanism module composed of Convolutional Block Attention Module (CBAM) and Efficient Channel Attention (ECA) was integrated into the neck of the network to enhance further the model’s ability to focus on key target objects in complex backgrounds. Finally, the methods including data augmentation, Test Time Augmentation (TTA) and Weighted Boxes Fusion (WBF) were used to improve the robustness and generalization of the model for the identification of terminal buds in different growth states. The results showed that, compared with the standard version of YOLOv5, the recognition accuracy of the improved YOLOv5 was significantly increased, with a precision of 95.55%, a recall of 95.84%, an F1-Score of 96.54%, and an mAP of 94.63%. Under the same experimental conditions, compared with other current mainstream algorithms (YOLOv3, Faster R-CNN, and PP-YOLO), the average precision and F1-Score of the improved YOLOv5 also increased by 9.51-28.19 percentage points and 15.92-32.94 percentage points, respectively. Overall, The improved YOLOv5 algorithm integrated with the attention network can accurately identify the terminal buds of densely-planted Chinese fir seedlings in UAV images and provide technical support for large-scale and automated counting and precision cultivation of Chinese fir seedlings.

A Deep Learning-Based Approach for the Diagnosis of Acute Lymphoblastic Leukemia

Leukemia is a deadly disease caused by the overproduction of immature white blood cells (WBS) in the bone marrow. If leukemia is detected at the initial stages, the chances of recovery are better. Typically, morphological analysis for the identification of acute lymphoblastic leukemia (ALL) is performed manually on blood cells by skilled medical personnel, which has several disadvantages, including a lack of medical personnel, sluggish analysis, and prediction that is dependent on the medical personnel’s expertise. Therefore, we proposed the Multi-Attention EfficientNetV2S and EfficientNetB3 state-of-the-art deep learning architectures using transfer learning-based fine-tuning approach to distinguish the normal and blast cells from microscopic blood smear images that both are pretrained on large-scale ImageNet database. We simply modified the last block of both models and added additional layers to both models. After including this Multi-Attention Mechanism, it not only reduces the model’s complexities but also generalizes its network quite well. By using the proposed technique, the accuracy has improved and the overall loss is also minimized. Our Multi-Attention EfficientNetV2S and EfficientNetB3 models achieved 99.73% and 99.25% accuracy, respectively. We have further compared the proposed model’s performance to other individual and ensemble models. Upon comparison, the proposed model outclassed the existing literature and other benchmark models, thus proving its efficiency.

MAEF-Net: Multi-attention efficient feature fusion network for left ventricular segmentation and quantitative analysis in two-dimensional echocardiography

The segmentation of cardiac chambers and the quantification of clinical functional metrics in dynamic echocardiography are the keys to the clinical diagnosis of heart disease. Identifying the end-diastolic frames (EDFs) and end-systolic frames (ESFs) and manually segmenting the left ventricle in the echocardiographic cardiac cycle before obtaining the left ventricular ejection fraction (LVEF) is a time-consuming and tedious task for clinicians. In this work, we proposed a deep learning-based fully automated echocardiographic analysis method. We proposed a multi-attention efficient feature fusion network (MAEF-Net) to automatically segment the left ventricle. Then, EDFs and ESFs in all cardiac cycles were automatically detected to compute LVEF. The MAEF-Net method used a multi-attention mechanism to guide the network to capture heartbeat features effectively, while suppressing noise, and incorporated deep supervision mechanism and spatial pyramid feature fusion to enhance feature extraction capabilities. The proposed method was validated on the public EchoNet-Dynamic dataset (n = 1226). The Dice similarity coefficient (DSC) of the left ventricular segmentation reached (93.10 ± 2.22)%, and the mean absolute error (MAE) of cardiac phase detection was (2.36 ± 2.23) frames. The MAE for predicting LVEF was 6.29 %. The proposed method was also validated on a private clinical dataset (n = 22). The DSC of the left ventricular segmentation reached (92.81 ± 2.85)%, and the MAE of cardiac phase detection was (2.25 ± 2.27) frames. The MAE for predicting LVEF was 5.91 %, and the Pearson correlation coefficient r reached 0.96. The proposed method may be used as a new method for automatic left ventricular segmentation and quantitative analysis in two-dimensional echocardiography. Our code and trained models will be made available publicly at https://github.com/xiaojinmao-code/MAEF-Net.

Dynamic Prediction of Internet Financial Market Based on Deep Learning.

P2P lending is an important part of Internet finance, which is popular among users because of its efficiency, low cost, wide range, and ease of operation. The problem of predicting loan defaults is affected by many factors, such as the linear and nonlinear nature of the data itself and time dependence and multiple external factors, which have not been well captured in the previous work. In this paper, we propose a multiattention mechanism to capture the different effects of various time slices and various external factors on the results, introduce ARIMA and LSTM to capture the linear and nonlinear characteristics of the lending data respectively, and establish a Time Series Multiattention Prediction Model (MAT-ALSTM) based on LSTM and ARIMA. This paper uses the Lending Club dataset from the United States to prove that our model is superior to ANN, SVM, LSTM, GRU, and ARIMA models in the prediction effect of MAE, RMSE, and DA.

SRMANet: Toward an Interpretable Neural Network with Multi-Attention Mechanism for Gearbox Fault Diagnosis

Deep neural network (DNN), with the capacity for feature inference and nonlinear mapping, has demonstrated its effectiveness in end-to-end fault diagnosis. However, the intermediate learning process of the DNN architecture is invisible, making it an uninterpretable black-box model. In this paper, a stacked residual multi-attention network (SRMANet) is proposed as a means of feature extraction of vibration signals, and visualizing the model training process, designing Squeeze-excitation residual (SE-Res) blocks to obtain additive features with minimal redundancy and sparsity. This study recommends the use of the attention fusion unit to ensure the interpretability of the model and ultimately to obtain representative features. By feeding the output gradient of the attention layer back to the original signal, the key feature components in the time domain signal can be effectively captured. Finally, the interpretability, identification accuracy and adaptability of the model under different operating conditions are verified on 12 different fault tasks in the planetary gearbox.

Comparative Convolutional Dynamic Multi-Attention Recommendation Model.

Recently, an attention mechanism has been used to help recommender systems grasp user interests more accurately. It focuses on their pivotal interests from a psychology perspective. However, most current studies based on it only focus on part of user interests; they have not mined user preferences thoroughly. To address the above problem, we propose a novel recommendation model: comparative convolutional dynamic multi-attention (CCDMA). This model provides a more accurate approach to represent user and item features and uses multi-attention-based convolutional neural networks to extract user and item latent feature vectors dynamically. The multi-attention mechanism considers both self-attention and cross-attention. Self-attention refers to the internal attention within users and items; cross-attention is the mutual attention between users and items. Moreover, we propose an optimized comparative learning framework that can mine the ternary relationships between one user and a pair of items, focusing on their relative relationship and the internal link between a pair of items. Extensive experiments on several real-world data sets show that the CCDMA model significantly outperforms state-of-the-art baselines in terms of different evaluation metrics.

Fan Fault Diagnosis Based on Lightweight Multiscale Multiattention Feature Fusion Network

Although the deep learning diagnosis model has been widely used in the fault diagnosis of rotating machinery. However, these methods lack the interpretability of the diagnostic process. In other words, it is still a difficult problem to understand that the structural function and the diagnosis process in the model correspond to each other. Therefore, this article discusses how to add multiscale and multiattention mechanism to lightweight network. From different scales, different dimensions, combined with the fault signal characteristics of centrifugal fan, the attention structure of cross layer fusion is designed. How to integrate different functions continuously and effectively to achieve better diagnostic performance is answered. The proposed lightweight multiscale multiattention feature fusion network adaptively recalibrates feature weights, which effectively enhances the fault feature learning ability and antinoise ability. Experimental results show that this network is stronger than other advanced diagnostic models.

Design and Application of Preschool Education System Based on Mobile Application

With the rapid development of information technology in recent years, the country’s information reform in the field of education is advancing gradually. As the first link in the chain of lifelong education, the development of early childhood education informatization has become an essential component of the national education informatization construction. In order to better realize the gamification of early childhood education courses and strengthen the management and deepening of course resources, this paper analyzes the significance of the application of children’s games in teaching and the significance of the application of new media resources in children’s course games by combining the actual situation of education and the teaching advantages of new media resources. The paper studies the application of early childhood education teaching and the research on the development of gamification in early childhood education courses. In light of the fact that emerging preschool education products on the market cannot fully meet the requirements of preschool education informatization construction at the level of function and technology, this paper proposes a preschool education system based on mobile applications as well as a child-appropriate recommendation algorithm based on a multiattention mechanism. Experiments conducted on public data sets demonstrate that our algorithm is considerably superior to the benchmark algorithm.