Multi-attention Network Research Articles

Multi-attention network with redundant information filtering for multi-horizon forecasting in multivariate time series

Multi-horizon forecasting of multivariate time series has always been a prominent research topic in domains such as finance and transportation. While prediction models that integrate attention mechanisms and Seq2Seq frameworks effectively capture the temporal dependencies between sequences, there might exist distinctions in the contributions of different input variables to the predictions. RNN-based encoders often overlook these distinctions. Hence, we propose a Multi-attention Network with Redundant Information Filtering (MNRIF) for Multi-horizon Forecasting in Multivariate Time Series, comprising the underlying component Two-stage Feature Fusion Network (TSFFN) and the upper-level structure Temporal Attention Network(TAN). TSFFN first designs a variable-level attention to screen the spatial semantics of explanatory variables during multi-step recursive iterations, effectively reducing noisy information, then builds a feature gate fusion module to adaptively weigh the degree of association between prediction targets and spatial semantics as well as target variables, thereby accomplishing secondary filtering of redundant information. TAN is a task-oriented learner that combines the Encoder–Decoder framework with attention mechanism. By means of dual information filtering based on attention mechanism and recurrent gate, it enhances effective mining of long-term temporal dependency features in sequences. Extensive experiments conducted on three real-world benchmark datasets demonstrate MNRIF’s superior performance compared to various state-of-the-art methods of different types.

DiffCAS: diffusion based multi-attention network for segmentation of 3D coronary artery from CT angiography

DiffCAS: diffusion based multi-attention network for segmentation of 3D coronary artery from CT angiography

LMA-EEGNet: A Lightweight Multi-Attention Network for Neonatal Seizure Detection Using EEG signals

Neonatal epilepsy is an early postnatal brain disorder, and automatic seizure detection is crucial for timely diagnosis and treatment to reduce potential brain damage. This work proposes a novel Lightweight Multi-Attention Network, LMA-EEGNet, for diagnosing neonatal epileptic seizures from multi-channel EEG signals employing dilated depthwise separable convolution (DDS Conv) for feature extraction and using pointwise convolution followed by global average pooling for classification. The proposed approach substantially reduces the model size, number of parameters, and computational complexity, which are crucial for real-time detection and clinical diagnosis of neonatal epileptic seizures. LMA-EEGNet integrates temporal and spectral features through distinct temporal and spectral branches. The temporal branch uses DDS Conv to extract temporal features, enhanced by a channel attention mechanism. The spectral branch utilizes similar convolutions alongside a spatial attention mechanism to highlight key frequency components. Outputs from both branches are merged and processed through a pointwise convolution layer and a global average pooling layer for efficient neonatal seizure detection. Experimental results show that our model, with only 2471 parameters and a size of 23 KB, achieves an accuracy of 95.71% and an AUC of 0.9862, demonstrating its potential for practical deployment. This study provides an effective deep learning solution for the early detection of neonatal epileptic seizures, improving diagnostic accuracy and timeliness.

CMNet: Cascaded context fusion and multi-attention network for multiple lesion segmentation of diabetic retinopathy images

Diabetic retinopathy (DR) is a leading cause of blindness in humans, and regular screening is helpful to early detection and containment of DR. Therefore, automated and accurate lesion segmentation is crucial for DR grading and diagnosis. However, it is full of challenges due to complex structures, inconsistent scales and blurry edges of different kinds of lesions. To address the above issues, this paper proposes a cascaded context fusion and multi-attention network (CMNet) for multiple lesion segmentation of DR images. The CMNet includes triple attention module (TAM), cascaded context fusion module (CFM) and balanced attention module (BAM). Firstly, TAM is proposed to extract channel attention, spatial attention and pixel-point attention features for addition fusion to ensure selectivity and consistency of information representation. Moreover, CFM applies adaptive average pooling and non-local operation to capture local and global contextual features for concatenation fusion to expand the receptive field of lesions. Finally, BAM computes foreground, background and boundary attention maps of lesions, as well as uses Squeeze-and-Excitation block to weigh the feature channels and rebalance attention of three regions to make the network focus on the edge details for fine segmentation. We performed comprehensive comparison experiments and ablation studies on three public datasets of DDR, IDRiD and E-Ophtha, and mAUC of four lesions segmentation reached 0.6765, 0.7466 and 0.6710, respectively. The experimental results show that the proposed model outperforms current state-of-the-arts, which overcomes the adverse interference of background noise, and preserves contour details for realizing precise segmentation of multi-scale lesion.

Correction: Enhancing real-time fire detection: an effective multi-attention network and a fire benchmark

Correction: Enhancing real-time fire detection: an effective multi-attention network and a fire benchmark

Enhancing Stock Price Prediction with Deep Cross-Modal Information Fusion Network

Stock price prediction is considered a classic and challenging task, with the potential to aid traders in making more profitable trading decisions. Significant improvements in stock price prediction methods based on deep learning have been observed in recent years. However, most existing methods are reliant solely on historical stock price data for predictions, resulting in the inability to capture market dynamics beyond price indicators, thus limiting their performance to some extent. Therefore, combining social media text with historical stock price information has proposed a novel stock price prediction method, known as the Deep Cross-Modal Information Fusion Network (DCIFNet). The process is initiated by DCIFNet, which employs temporal convolution processes to encode stock prices and Twitter content. This ensures that each element has sufficient information about its surrounding components. Following this, the outcomes are inputted into a cross-modal fusion structure based on transformers to enhance the integration of crucial information from stock prices and Twitter content. Lastly, a multi-graph convolution attention network is introduced to depict the relationships between different stocks from diverse perspectives. This facilitates the more effective capturing of industry affiliations, Wikipedia references, and associated relationships among linked stocks, ultimately leading to an enhancement in stock price prediction accuracy. Trend prediction and simulated trading experiments are conducted on high-frequency trading datasets spanning nine different industries. Comparative assessments with the Multi-Attention Network for Stock Prediction (MANGSF) method, as well as ablation experiments, confirm the effectiveness of the DCIFNet approach, resulting in an accuracy rate of 0.6309, a marked improvement compared to representative methods in the field.

Sonar Digital Twin Layer via Multiattention Networks With Feature Transfer

Sonar Digital Twin Layer via Multiattention Networks With Feature Transfer

Comparing Deep Learning Performance for Chronic Lymphocytic Leukaemia Cell Segmentation in Brightfield Microscopy Images.

This article focuses on the detection of cells in low-contrast brightfield microscopy images; in our case, it is chronic lymphocytic leukaemia cells. The automatic detection of cells from brightfield time-lapse microscopic images brings new opportunities in cell morphology and migration studies; to achieve the desired results, it is advisable to use state-of-the-art image segmentation methods that not only detect the cell but also detect its boundaries with the highest possible accuracy, thus defining its shape and dimensions. We compared eight state-of-the-art neural network architectures with different backbone encoders for image data segmentation, namely U-net, U-net++, the Pyramid Attention Network, the Multi-Attention Network, LinkNet, the Feature Pyramid Network, DeepLabV3, and DeepLabV3+. The training process involved training each of these networks for 1000 epochs using the PyTorch and PyTorch Lightning libraries. For instance segmentation, the watershed algorithm and three-class image semantic segmentation were used. We also used StarDist, a deep learning-based tool for object detection with star-convex shapes. The optimal combination for semantic segmentation was the U-net++ architecture with a ResNeSt-269 background with a data set intersection over a union score of 0.8902. For the cell characteristics examined (area, circularity, solidity, perimeter, radius, and shape index), the difference in mean value using different chronic lymphocytic leukaemia cell segmentation approaches appeared to be statistically significant (Mann-Whitney U test, P < .0001). We found that overall, the algorithms demonstrate equal agreement with ground truth, but with the comparison, it can be seen that the different approaches prefer different morphological features of the cells. Consequently, choosing the most suitable method for instance-based cell segmentation depends on the particular application, namely, the specific cellular traits being investigated.

A Multiattention ResUNet and Modified U‐Net Architecture for Liver Tumor Segmentation

Liver cancer is one of the leading causes of cancer death in the world, and early diagnosis is important. However, the similarity in shape, texture, and intensity values between the liver, tumors, and other neighboring organs such as the heart, spleen, stomach, and kidneys often complicates visual differentiation. Manual identification of tumors in the liver is time‐consuming, intricate, and susceptible to errors with potential repercussions for patient care. While machine learning–based approaches have emerged for liver organ recognition and segmenting the tumor, they continue to face challenges related to recognition accuracy and the inability to distinguish tumors of varied sizes. To solve the problems, a multiattention network made up of cascaded ResUNet and U‐Net with attention mechanisms was proposed in this study. We investigated liver tumor segmentation with various configurations of U‐Net, ResUNet, U‐Net with attention mechanisms, and ResUNet with attention mechanisms on augmented and nonaugmented data. We used the 3Dircadb dataset for training and validation purposes, and the proposed method was evaluated on dice score, intersection of union (IoU), recall, and precision. The performance metrics achieved with this method on the dataset are as follows: approximately 0.89 for the dice coefficient, 0.90 for IoU, 0.93 for recall, and 0.96 for precision in the case of liver segmentation without data augmentation and 0.92, 0.90, 0.92, and 0.94, respectively, for dice score, IoU, recall, and precision with data augmentation. For tumor segmentation, the metrics include 0.70 for dice coefficient, 0.61 for IoU, 0.91 for recall, and 0.94 for precision when the data were augmented but 0.83 for dice score, 0.78 for IoU, and 0.89 and 0.90, respectively, for recall and precision.

Single Stage Adaptive Multi-Attention Network for Image Restoration.

Recently attention-based networks have been successful for image restoration tasks. However, existing methods are either computationally expensive or have limited receptive fields, adding constraints to the model. They are also less resilient in spatial and contextual aspects and lack pixel-to-pixel correspondence, which may degrade feature representations. In this paper, we propose a novel and computationally efficient architecture Single Stage Adaptive Multi-Attention Network (SSAMAN) for image restoration tasks, particularly for image denoising and image deblurring. SSAMAN efficiently addresses computational challenges and expands receptive fields, enhancing robustness in spatial and contextual feature representation. Its Adaptive Multi-Attention Module (AMAM), which consists of Adaptive Pixel Attention Branch (APAB) and an Adaptive Channel Attention Branch (ACAB), uniquely integrates channel and pixel-wise dimensions, significantly improving sensitivity to edges, shapes, and textures. We perform extensive experiments and ablation studies to validate the performance of SSAMAN. Our model shows state-of-the-art results on various benchmarks, for example, on image denoising tasks, SSAMAN achieves a notable 40.08 dB PSNR on SIDD dataset, outperforming Restormer by 0.06 dB PSNR, with 41.02% less computational cost, and achieves a 40.05 dB PSNR on the DND dataset. For image deblurring, SSAMAN achieves 33.53 dB PSNR on GoPro dataset. Code and models are available at Github.

Multi-scale multi-attention network for diabetic retinopathy grading

Objective. Diabetic retinopathy (DR) grading plays an important role in clinical diagnosis. However, automatic grading of DR is challenging due to the presence of intra-class variation and small lesions. On the one hand, deep features learned by convolutional neural networks often lose valid information about these small lesions. On the other hand, the great variability of lesion features, including differences in type and quantity, can exhibit considerable divergence even among fundus images of the same grade. To address these issues, we propose a novel multi-scale multi-attention network (MMNet). Approach. Firstly, to focus on different lesion features of fundus images, we propose a lesion attention module, which aims to encode multiple different lesion attention feature maps by combining channel attention and spatial attention, thus extracting global feature information and preserving diverse lesion features. Secondly, we propose a multi-scale feature fusion module to learn more feature information for small lesion regions, which combines complementary relationships between different convolutional layers to capture more detailed feature information. Furthermore, we introduce a Cross-layer Consistency Constraint Loss to overcome semantic differences between multi-scale features. Main results. The proposed MMNet obtains a high accuracy of 86.4% and a high kappa score of 88.4% for multi-class DR grading tasks on the EyePACS dataset, while 98.6% AUC, 95.3% accuracy, 92.7% recall, 95.0% precision, and 93.3% F1-score for referral and non-referral classification on the Messidor-1 dataset. Extensive experiments on two challenging benchmarks demonstrate that our MMNet achieves significant improvements and outperforms other state-of-the-art DR grading methods. Significance. MMNet has improved the diagnostic efficiency and accuracy of diabetes retinopathy and promoted the application of computer-aided medical diagnosis in DR screening.

A graph multi-attention network for predicting airport delays

Predicting airport delays is of great importance for aviation operations, from the development of effective air traffic management strategies to the reallocation of airline resources. In this paper, the long-term prediction of the next 24 h of network-wide delays is investigated. The sensitivity to error propagation over long time periods as well as dynamic spatial correlations and non-linear temporal correlations of the aviation network are considered. An external impact modeling module is introduced to account for the influence of weather on flight delay patterns. A Spatial-Temporal Gated Multi-Attention Graph Network (STGMAGNet) considering external impact to predict airport delays is then developed. We validate our model on a flight delays dataset collected from the Bureau of Transportation Statistics of US, covering January 1, 2019, to December 31, 2019. In long-term (input-24-predict-24 setting) forecasting, STGMAGNet provides state-of-the-art accuracy, with a MAE reduction of at least 21% averaged in arrival delay prediction, 18% averaged MAE reduction in departure delay prediction compared to MLP, LSTM, Seq2Seq and Transformer. Our model can enable aviation management to shift from a reactive to proactive approach, thus enhancing its operational efficiency and overall performance.

Enhancing real-time fire detection: an effective multi-attention network and a fire benchmark

Enhancing real-time fire detection: an effective multi-attention network and a fire benchmark

MANet: a multi-attention network for automatic liver tumor segmentation in computed tomography (CT) imaging

Automatic liver tumor segmentation is a paramount important application for liver tumor diagnosis and treatment planning. However, it has become a highly challenging task due to the heterogeneity of the tumor shape and intensity variation. Automatic liver tumor segmentation is capable to establish the diagnostic standard to provide relevant radiological information to all levels of expertise. Recently, deep convolutional neural networks have demonstrated superiority in feature extraction and learning in medical image segmentation. However, multi-layer dense feature stacks make the model quite inconsistent in imitating visual attention and awareness of radiological expertise for tumor recognition and segmentation task. To bridge that visual attention capability, attention mechanisms have developed for better feature selection. In this paper, we propose a novel network named Multi Attention Network (MANet) as a fusion of attention mechanisms to learn highlighting important features while suppressing irrelevant features for the tumor segmentation task. The proposed deep learning network has followed U-Net as the basic architecture. Moreover, residual mechanism is implemented in the encoder. Convolutional block attention module has split into channel attention and spatial attention modules to implement in encoder and decoder of the proposed architecture. The attention mechanism in Attention U-Net is integrated to extract low-level features to combine with high-level ones. The developed deep learning architecture is trained and evaluated on the publicly available MICCAI 2017 Liver Tumor Segmentation dataset and 3DIRCADb dataset under various evaluation metrics. MANet demonstrated promising results compared to state-of-the-art methods with comparatively small parameter overhead.

A cascaded framework with cross-modality transfer learning for whole heart segmentation

Automatic and accurate segmentation of the whole heart structure from 3D cardiac images plays an important role in helping physicians diagnose and treat cardiovascular disease. However, the time-consuming and laborious manual labeling of the heart images results in the inefficiency of utilizing the existing CT or MRI for training the deep learning network, which decrease the accuracy of whole heart segmentation. However, multi-modality data contains multi-level information of cardiac images due to different imaging mechanisms, which is beneficial to improve the segmentation accuracy. Therefore, this paper proposes a cascaded framework with cross-modality transfer learning for whole heart segmentation (CM-TranCaF), which consists of three key modules: modality transfer network (MTN), U-shaped multi-attention network (MAUNet) and spatial configuration network (SCN). In MTN, MRI images are transferred from MRI domain to CT domain, to increase the data volume by adopting the idea of adversarial training. The MAUNet is designed based on UNet, while the attention gates (AGs) are integrated into the skip connection to reduce the weight of background pixels. Moreover, to solve the problem of boundary blur, the position attention block (PAB) is also integrated into the bottom layer to aggregate similar features. Finally, the SCN is used to finetune the segmentation results by utilizing the anatomical information between different cardiac substructures. By evaluating the proposed method on the dataset of the MM-WHS challenge, CM-TranCaF achieves a Dice score of 91.1% on the testing dataset. The extensive experimental results prove the effectiveness of the proposed method compared to other state-of-the-art methods.

Modeling of Digital Twin Workshop in Planning via a Graph Neural Network: The Case of an Ocean Engineering Manufacturing Intelligent Workshop

In the era of Industry 4.0 to 5.0, the manufacturing industry is dedicated to improving its production efficiency, control capability and competitiveness with intelligent enhancement. As a typical discrete manufacturing industry, it is difficult for ocean engineering (OE) manufacturers to accurately control the entire production process, and the establishment of an integrated system supported by digital twin (DT) technology is a better solution. This paper proposes a comprehensive set of system architectures for the DT workshop. It focuses on planning, which is the main line of control, to establish a model based on graph neural networks (GNNs) and suggests five decision-support approaches associated with the model from a practical application perspective. The utilization of complete twin data for prediction and visual simulation effectively eliminates the problem of unexpected factors interfering with scheduling in enterprise production planning and achieves the goals of rapid processing and just-in-time completion. The planning model is based on the attention mechanism, which characterizes the disjunctive graph, extracts the input GNN, and outputs the scheduling decision by constructing the multi-attention network of operations and machines to deal with the complicated “operation–machine” combination relationship. The proposed method has been verified in the case of structural assembly and welding workshops, has validity and reliability, and is superior to the traditional priority scheduling rules and heuristics in terms of precision rate and rapidity. Furthermore, the DT system completes the production line application, and its proven reliability supports its full-scale application in future smart factories.

Multi-attention network for pedestrian intention prediction based on spatio-temporal feature fusion

An essential prerequisite for autonomous vehicles deploying in urban scenarios is the ability to accurately recognize the behavioral intentions of pedestrians and other vulnerable road users and take measures to ensure their safety. In this paper, a spatial-temporal feature fusion-based multi-attention network (STFF-MANet) is designed to predict pedestrian crossing intention. Pedestrian information, vehicle information, scene context, and optical flow are extracted from continuous image sequences as feature sources. A lightweight 3D convolutional network is designed to extract temporal features from optical flow. Construct a spatial encoding module to extract the spatial features from the context. Pedestrian motion information are re-encoded using a collection of gated recurrent units. The final network structure is created through ablation research, which introduces attention mechanisms into the network to merge pedestrian motion features and spatio-temporal features. The efficiency of the suggested strategy is demonstrated by comparison experiments on the datasets JAAD and PIE. On the JAAD dataset, the intent recognition accuracy is 9% more accurate than the existing techniques.

MFMANet: Multi-feature Multi-attention Network for efficient subtype classification on non-small cell lung cancer CT images

Lung cancer is one of the most prevalent diseases worldwide, being the most common type of cancer. Non-small cell lung cancer (NSCLC) with a five-year survival rate of less than 20% and widely varying treatment modalities is diagnosed in more than 85% of cases. In this study, a multi-feature multi-attention network (MFMANet) was proposed for NSCLC subtype classification to address the problem of low classification accuracy between subtypes due to small lesion size and similar background. In MFMANet, the multi-scale spatial channel attention module (MSAM) and multi-feature fusion global local attention module (MFGLA), are proposed. MSAM is able to preserve spatial features during channel feature fusion to capture high-order statistical information. MFGLA performs effective fusion of multiple features to avoid interference caused by scale differences. The global and local information is extracted by global and local attention branches to enhance the perception of small lesion regions. The performance of the proposed MFMANet was validated on two public datasets and compared with other classification networks including ResNet18, ShuffleNetv2, MobileNetv3, and MnasNet. MFMANet achieved 99.06% and 91.67% accuracy in the binary classification of CT images of lung adenocarcinoma (ADC) and lung squamous cell carcinoma (SCC), outperforming other methods. This study confirms that the proposed MFMANet provides effective solution for the subtypes classification of NSCLC.

Multi-Attention Segmentation Networks Combined with the Sobel Operator for Medical Images

Medical images are used as an important basis for diagnosing diseases, among which CT images are seen as an important tool for diagnosing lung lesions. However, manual segmentation of infected areas in CT images is time-consuming and laborious. With its excellent feature extraction capabilities, a deep learning-based method has been widely used for automatic lesion segmentation of COVID-19 CT images. However, the segmentation accuracy of these methods is still limited. To effectively quantify the severity of lung infections, we propose a Sobel operator combined with multi-attention networks for COVID-19 lesion segmentation (SMA-Net). In our SMA-Net method, an edge feature fusion module uses the Sobel operator to add edge detail information to the input image. To guide the network to focus on key regions, SMA-Net introduces a self-attentive channel attention mechanism and a spatial linear attention mechanism. In addition, the Tversky loss function is adopted for the segmentation network for small lesions. Comparative experiments on COVID-19 public datasets show that the average Dice similarity coefficient (DSC) and joint intersection over union (IOU) of the proposed SMA-Net model are 86.1% and 77.8%, respectively, which are better than those in most existing segmentation networks.

Automatic vessel crossing and bifurcation detection based on multi-attention network vessel segmentation and directed graph search

Analysis of the vascular tree is the basic premise to automatically diagnose retinal biomarkers associated with ophthalmic and systemic diseases, among which accurate identification of intersection and bifurcation points is quite challenging but important for disentangling complex vascular network and tracking vessel morphology. In this paper, we present a novel directed graph search-based multi-attentive neural network approach to automatically segment the vascular network and separate intersections and bifurcations from color fundus images. Our approach uses multi-dimensional attention to adaptively integrate local features and their global dependencies while learning to focus on target structures at different scales to generate binary vascular maps. A directed graphical representation of the vascular network is constructed to represent the topology and spatial connectivity of the vascular structures. Using local geometric information including color difference, diameter, and angle, the complex vascular tree is decomposed into multiple sub-trees to finally classify and label vascular feature points. The proposed method has been tested on the DRIVE dataset and the IOSTAR dataset containing 40 images and 30 images, respectively, with 0.863 and 0.764 F1-score of detection points and average accuracy of 0.914 and 0.854 for classification points. These results demonstrate the superiority of our proposed method outperforming state-of-the-art methods in feature point detection and classification.