Dual Attention Network Research Articles

SDAN-MD: Supervised dual attention network for multi-stage motion deblurring in frontal-viewing vehicle-camera images

Motion blur in images usually distorts the information of objects, thus degrading the performance of semantic segmentation. However, there is no previous research on improving the segmentation performance by restoring the frontal-viewing vehicle-camera images taken under motion blur. Therefore, this study proposes a supervised dual attention network for multi-stage motion deblurring (SDAN-MD) for this task. In SDAN-MD, a supervised dual attention module (SDAM) is proposed, which adopts the supervised spatial and channel attention mechanisms to provide a supervisory signal of ground truth. In addition to Charbonnier loss and edge loss, we use perceptual loss utilizing Euclidean distance based on feature maps obtained from the segmentation network.Experiments were conducted with the motion blurred databases from the two open databases of road scene, Cambridge driving Labeled Video Database (CamVid) and Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago (KITTI).The results show that the proposed SDAN-MD achieves 92.89% and 87.27% pixel accuracies in semantic segmentation using these two databases, respectively, outperforming the state-of-the-art methods.

JAAL-Net: a joint attention and adversarial learning network for skin lesion segmentation

Objective. Skin lesion segmentation plays an important role in the diagnosis and treatment of melanoma. Existing skin lesion segmentation methods have trouble distinguishing hairs, air bubbles, and blood vessels around lesions, which affects the segmentation performance. Approach. To clarify the lesion boundary and raise the accuracy of skin lesion segmentation, a joint attention and adversarial learning network (JAAL-Net) is proposed that consists of a generator and a discriminator. In the JAAL-Net, the generator is a local fusion network (LF-Net) utilizing the encoder-decoder structure. The encoder contains a convolutional block attention module to increase the weight of lesion information. The decoder involves a contour attention to obtain edge information and locate the lesion. To aid the LF-Net generate higher confidence predictions, a discriminant dual attention network is constructed with channel attention and position attention. Main results. The JAAL-Net is evaluated on three datasets ISBI2016, ISBI2017 and ISIC2018. The intersection over union of the JAAL-Net on the three datasets are 90.27%, 89.56% and 80.76%, respectively. Experimental results show that the JAAL-Net obtains rich lesion and boundary information, enhances the confidence of the predictions, and improves the accuracy of skin lesion segmentation. Significance. The proposed approach effectively improves the performance of the model for skin lesion segmentation, which can assist physicians in accurate diagnosis well.

A Rumor Detection Model Incorporating Propagation Path Contextual Semantics and User Information.

Currently, social media is full of rumors. To stop rumors from spreading further, rumor detection has received increasing attention. Recent rumor detection methods treat all propagation paths and all nodes on the paths as equally important, resulting in models that fail to extract the key features. In addition, most methods ignore user features, leading to limitations in the performance improvement of rumor detection. To address these problems, we propose a Dual-Attention Network model on propagation Tree structures named DAN-Tree, where a node-and-path dual-attention mechanism is designed to organically fuse deep structure and semantic information on the propagation structures of rumors, and path oversampling and structural embedding are employed to enhance the learning of deep structures. Finally, we deeply integrate user profiles into the propagation trees in DAN-Tree, thus proposing the DAN-Tree++ model to further improve performance. Empirical studies on four rumor datasets have shown that DAN-Tree outperforms the state-of-the-art rumor detection models learning on propagation structures, and the results on two datasets with user information validate the superior performance of DAN-Tree++ over other models using both user profiles and propagation structures. What's more, DAN-Tree, especially DAN-Tree++, has achieved the best performance on early detection tasks.

Dual graph attention networks for multi-behavior recommendation

Dual graph attention networks for multi-behavior recommendation

CKDAN: Content and keystroke dual attention networks with pre-trained models for continuous authentication

With the rapid development of digitalization, information security becomes more and more important. User authentication is a significant line of defense. During the past few years, many research works concentrating on user authentication have been published. Traditional authentication techniques (password, ID card-based authentication, or other biometrics like fingerprint or iris) basically fall into the category of one-time authentication. The disadvantage of these methods is that once an unknown user completes an authentication through forgery, the system would continue to operate without any resistance, thus putting the entire system at risk. Recently, due to the prevalence of deep learning techniques, some researchers have applied learning-based models to user authentication. However, almost all existing methods only focus on keystroke dynamics while ignoring the ”text” entered by users during keystrokes. In this paper, we propose contents and keystroke dual attention networks with pre-trained models for continuous authentication. To the best of our knowledge, our paper is the first to address user-inputted ”text” during keystrokes as an important asset beyond traditional characteristic keystroke dynamics. Specifically, we use the well-known pre-trained RoBERTa model to capture textual features. Then, we pass textual features and conventional features through our proposed dual attention networks. Our networks fuse these features and acquire final representations. Experiments show that the CKDAN model achieves state-of-the-art performance on two datasets, Clarkson II keystroke dataset and Buffalo dataset, outperforming all baseline methods.

A Novel Hybridoma Cell Segmentation Method Based on Multi-Scale Feature Fusion and Dual Attention Network

The hybridoma cell screening method is usually done manually by human eyes during the production process for monoclonal antibody drugs. This traditional screening method has certain limitations, such as low efficiency and subjectivity bias. Furthermore, most of the existing deep learning-based image segmentation methods have certain drawbacks, due to different shapes of hybridoma cells and uneven location distribution. In this paper, we propose a deep hybridoma cell image segmentation method based on residual and attention U-Net (RA-UNet). Firstly, the feature maps of the five modules in the network encoder are used for multi-scale feature fusion in a feature pyramid form and then spliced into the network decoder to enrich the semantic level of the feature maps in the decoder. Secondly, a dual attention mechanism module based on global and channel attention mechanisms is presented. The global attention mechanism (non-local neural network) is connected to the network decoder to expand the receptive field of the feature map and bring more rich information to the network. Then, the channel attention mechanism SENet (the squeeze-and-excitation network) is connected to the non-local attention mechanism. Consequently, the important features are enhanced by the learning of the feature channel weights, and the secondary features are suppressed, hence improving the cell segmentation performance and accuracy. Finally, the focal loss function is used to guide the network to learn the hard-to-classify cell categories. Furthermore, we evaluate the performance of the proposed RA-UNet method on a newly established hybridoma cell image dataset. Experimental results show that the proposed method has good reliability and improves the efficiency of hybridoma cell segmentation compared with state-of-the-art networks such as FCN, UNet, and UNet++. The results show that the proposed RA-UNet model has improvements of 0.8937%, 0.9926%, 0.9512%, and 0.9007% in terms of the dice coefficients, PA, MPA, and MIoU, respectively.

DapNet-HLA: Adaptive dual-attention mechanism network based on deep learning to predict non-classical HLA binding sites

Human leukocyte antigen (HLA) plays a vital role in immunomodulatory function. Studies have shown that immunotherapy based on non-classical HLA has essential applications in cancer, COVID-19, and allergic diseases. However, there are few deep learning methods to predict non-classical HLA alleles. In this work, an adaptive dual-attention network named DapNet-HLA is established based on existing datasets. Firstly, amino acid sequences are transformed into digital vectors by looking up the table. To overcome the feature sparsity problem caused by unique one-hot encoding, the fused word embedding method is used to map each amino acid to a low-dimensional word vector optimized with the training of the classifier. Then, we use the GCB (group convolution block), SENet attention (squeeze-and-excitation networks), BiLSTM (bidirectional long short-term memory network), and Bahdanau attention mechanism to construct the classifier. The use of SENet can make the weight of the effective feature map high, so that the model can be trained to achieve better results. Attention mechanism is an Encoder-Decoder model used to improve the effectiveness of RNN, LSTM or GRU (gated recurrent neural network). The ablation experiment shows that DapNet-HLA has the best adaptability for five datasets. On the five test datasets, the ACC index and MCC index of DapNet-HLA are 4.89% and 0.0933 higher than the comparison method, respectively. According to the ROC curve and PR curve verified by the 5-fold cross-validation, the AUC value of each fold has a slight fluctuation, which proves the robustness of the DapNet-HLA. The codes and datasets are accessible at https://github.com/JYY625/DapNet-HLA.

DA-Net: Dual Attention Network for Flood Forecasting

DA-Net: Dual Attention Network for Flood Forecasting

Dual Attention and Patient Similarity Network for drug recommendation.

Artificially making clinical decisions for patients with multi-morbidity has long been considered a thorny problem due to the complexity of the disease. Drug recommendations can assist doctors in automatically providing effective and safe drug combinations conducive to treatment and reducing adverse reactions. However, the existing drug recommendation works ignored two critical information. (i) Different types of medical information and their interrelationships in the patient's visit history can be used to construct a comprehensive patient representation. (ii) Patients with similar disease characteristics and their corresponding medication information can be used as a reference for predicting drug combinations. To address these limitations, we propose DAPSNet, which encodes multi-type medical codes into patient representations through code- and visit-level attention mechanisms, while integrating drug information corresponding to similar patient states to improve the performance of drug recommendation. Specifically, our DAPSNet is enlightened by the decision-making process of human doctors. Given a patient, DAPSNet first learns the importance of patient history records between diagnosis, procedure and drug in different visits, then retrieves the drug information corresponding to similar patient disease states for assisting drug combination prediction. Moreover, in the training stage, we introduce a novel information constraint loss function based on the information bottleneck principle to constrain the learned representation and enhance the robustness of DAPSNet. We evaluate the proposed DAPSNet on the public MIMIC-III dataset, our model achieves relative improvements of 1.33%, 1.20% and 2.03% in Jaccard, F1 and PR-AUC scores, respectively, compared to state-of-the-art methods. The source code is available at the github repository: https://github.com/andylun96/DAPSNet.

Unsupervised Deep Exemplar Colorization via Pyramid Dual Non-local Attention.

Exemplar-based colorization is a challenging task, which attempts to add colors to the target grayscale image with the aid of a reference color image, so as to keep the target semantic content while with the reference color style. In order to achieve visually plausible chromatic results, it is important to sufficiently exploit the global color style and the semantic color information of the reference color image. However, existing methods are either clumsy in exploiting the semantic color information, or lack of the dedicated fusion mechanism to decorate the target grayscale image with the reference semantic color information. Besides, these methods usually use a single-stage encoder-decoder architecture, which results in the loss of spatial details. To remedy these problems, we propose an effective exemplar colorization strategy based on pyramid dual non-local attention network to exploit the long-range dependency as well as multi-scale correlation. Specifically, two symmetrical branches of pyramid non-local attention block are tailored to achieve alignments from the target feature to the reference feature and from the reference feature to the target feature respectively. The bidirectional non-local fusion strategy is further applied to get a sufficient fusion feature that achieves full semantic consistency between multi-modal information. To train the network, we propose an unsupervised learning manner, which employs the hybrid supervision including the pseudo paired supervision from the reference color images and unpaired supervision from both the target grayscale and reference color images. Extensive experimental results are provided to demonstrate that our method achieves better photo-realistic colorization performance than the state-of-the-art methods. The source code is available at https://github.com/wd1511/PDNLA-Net.

Dual Attention Network for Unsupervised Domain Adaptive Person Re-Identification

Dual Attention Network for Unsupervised Domain Adaptive Person Re-Identification

Recurrent Residual Dual Attention Network for Airborne Laser Scanning Point Cloud Semantic Segmentation

Kernel point convolution (KPConv) can effectively represent the point features of point cloud data. However, KPConv-based methods just consider the local information of each point, which is very difficult to characterize the intrinsic properties of ALS point clouds for complex laser scanning conditions. Therefore, we rethink KPConv and propose a recurrent residual dual attention network (RRDAN) based on the encoder-decoder structure for the semantic segmentation of ALS point cloud data. In the encoder stage, we design an attention kernel point convolution (AKPConv) block by using a scaling factor of batch normalization to highlight the significant channel information. Then, we use the AKPConv block to develop a recurrent residual kernel attention (RRKA) module to iteratively aggregate the local neighborhood features. In the decoder stage, we design a global and local channel attention (GLCA) module with global connection and local 1D convolution to interact the global and local information after fusing the upsampled high-level representations and the skip-connected low-level features. In addition, to reduce the influence of the long-tail distribution of reflection intensity, we apply gamma transformation to correct the data as normal distribution. The proposed RRDAN can achieve diversified feature aggregation to implement the refined semantic segmentation of ALS point clouds. We evaluate our method on two ALS datasets (i.e., ISPRS and DCF2019) to demonstrate its performance compared to a few advanced methods. Code: https://github.com/SC-shendazt/RRDAN.

Brea-Net: An Interpretable Dual-Attention Network for Imbalanced Breast Cancer Classification

Brea-Net: An Interpretable Dual-Attention Network for Imbalanced Breast Cancer Classification

Built-Up Area Extraction Combing Densely Connected Dual-Attention Network and Multiscale Context

Built-Up Area Extraction Combing Densely Connected Dual-Attention Network and Multiscale Context

Apple detection and instance segmentation in natural environments using an improved Mask Scoring R-CNN Model.

The accurate detection and segmentation of apples during growth stage is essential for yield estimation, timely harvesting, and retrieving growth information. However, factors such as the uncertain illumination, overlaps and occlusions of apples, homochromatic background and the gradual change in the ground color of apples from green to red, bring great challenges to the detection and segmentation of apples. To solve these problems, this study proposed an improved Mask Scoring region-based convolutional neural network (Mask Scoring R-CNN), known as MS-ADS, for accurate apple detection and instance segmentation in a natural environment. First, the ResNeSt, a variant of ResNet, combined with a feature pyramid network was used as backbone network to improve the feature extraction ability. Second, high-level architectures including R-CNN head and mask head were modified to improve the utilization of high-level features. Convolutional layers were added to the original R-CNN head to improve the accuracy of bounding box detection (bbox_mAP), and the Dual Attention Network was added to the original mask head to improve the accuracy of instance segmentation (mask_mAP). The experimental results showed that the proposed MS-ADS model effectively detected and segmented apples under various conditions, such as apples occluded by branches, leaves and other apples, apples with different ground colors and shadows, and apples divided into parts by branches and petioles. The recall, precision, false detection rate, and F1 score were 97.4%, 96.5%, 3.5%, and 96.9%, respectively. A bbox_mAP and mask_mAP of 0.932 and 0.920, respectively, were achieved on the test set, and the average run-time was 0.27 s per image. The experimental results indicated that the MS-ADS method detected and segmented apples in the orchard robustly and accurately with real-time performance. This study lays a foundation for follow-up work, such as yield estimation, harvesting, and automatic and long-term acquisition of apple growth information.

HD2A-Net: A novel dual gated attention network using comprehensive hybrid dilated convolutions for medical image segmentation

The convolutional neural networks (CNNs) have been widely proposed in the medical image analysis tasks, especially in the image segmentations. In recent years, the encoder–decoder structures, such as the U-Net, were rendered. However, the multi-scale information transmission and effective modeling for long-range feature dependencies in these structures were not sufficiently considered. To improve the performance of the existing methods, we propose a novel hybrid dual dilated attention network (HD2A-Net) to conduct the lesion region segmentations. In the proposed network, we innovatively present the comprehensive hybrid dilated convolution (CHDC) module, which facilitates the transmission of the multi-scale information. Based on the CHDC module and the attention mechanisms, we design a novel dual dilated gated attention (DDGA) block to enhance the saliency of related regions from the multi-scale aspect. Besides, a dilated dense (DD) block is designed to expand the receptive fields. The ablation studies were performed to verify our proposed blocks. Besides, the interpretability of the HD2A-Net was analyzed through the visualization of the attention weight maps from the key blocks. Compared to the state-of-the-art methods including CA-Net, DeepLabV3+, and Attention U-Net, the HD2A-Net outperforms significantly, with the metrics of Dice, Average Symmetric Surface Distance (ASSD), and mean Intersection-over-Union (mIoU) reaching 93.16%, 93.63%, and 94.72%, 0.36 pix, 0.69 pix, and 0.52 pix, and 88.03%, 88.67%, and 90.33% on three publicly available medical image datasets: MAEDE-MAFTOUNI (COVID-19 CT), ISIC-2018 (Melanoma Dermoscopy), and Kvasir-SEG (Gastrointestinal Disease Polyp), respectively.

Exploiting Dual-Attention Networks for Explainable Recommendation in Heterogeneous Information Networks.

The aim of explainable recommendation is not only to provide recommended items to users, but also to make users aware of why these items are recommended. Traditional recommendation methods infer user preferences for items using user-item rating information. However, the expressive power of latent representations of users and items is relatively limited due to the sparseness of the user-item rating matrix. Heterogeneous information networks (HIN) provide contextual information for improving recommendation performance and interpreting the interactions between users and items. However, due to the heterogeneity and complexity of context information in HIN, it is still a challenge to integrate this contextual information into explainable recommendation systems effectively. In this paper, we propose a novel framework-the dual-attention networks for explainable recommendation (DANER) in HINs. We first used multiple meta-paths to capture high-order semantic relations between users and items in HIN for generating similarity matrices, and then utilized matrix decomposition on similarity matrices to obtain low-dimensional sparse representations of users and items. Secondly, we introduced two-level attention networks, namely a local attention network and a global attention network, to integrate the representations of users and items from different meta-paths for obtaining high-quality representations. Finally, we use a standard multi-layer perceptron to model the interactions between users and items, which predict users' ratings of items. Furthermore, the dual-attention mechanism also contributes to identifying critical meta-paths to generate relevant explanations for users. Comprehensive experiments on two real-world datasets demonstrate the effectiveness of DANER on recommendation performance as compared with the state-of-the-art methods. A case study illustrates the interpretability of DANER.

Coastline Recognition Algorithm Based on Multi-Feature Network Fusion of Multi-Spectral Remote Sensing Images

Remote sensing images can obtain broad geomorphic features and provide a strong basis for analysis and decision making. As 71% of the earth is covered by water, shipping has become an efficient means of international trade and transportation, and the development level of coastal cities will directly reflect the development level of a country. The coastline is the boundary line between seawater and land, so it is of great significance to accurately identify it to assist shipping traffic and docking, and this identification will also play a certain auxiliary role in environmental analysis. Currently, the main problems of coastline recognition conducted by remote sensing images include: (1) in the process of remote sensing, image transmission inevitably brings noise causing poor image quality and difficult image quality enhancement; (2) s single scale does not allow for the identification of coastlines at different scales; and (3) features are under-utilized, false detection is high and intuitive measurement is difficult. To address these issues, we used the following multispectral methods: (1) a PCA-based image enhancement algorithm was proposed to improve image quality; (2) a dual attention network and HRnet network were proposed to extract suspected coastlines from different levels; and (3) a decision set fusion approach was proposed to transform the coastline identification problem into a probabilistic problem for coastline extraction. Finally, we constructed a coastline straightening model to visualize and analyze the recognition effect. Experiments showed that the algorithm has an AOM greater than 0.88 and can achieve coastline extraction.

MA-Xnet: Mobile-Attention X-Network for Crack Detection

Modern crack detection algorithms based on deep learning have unsolved issues, such as an abundance of parameters in the resulting models and lack of context information. Such issues may lower the efficiency of feature extraction and lead to unexpected task performance. Based on two semantic segmentation models, U-Net and the dual-attention network (DANet), an efficient mobile-attention X-network (MA-Xnet) is proposed for crack detection. For performance evaluation, segmentation experiments were performed on concrete crack images from an internationally recognized dataset, which were collected from various campus buildings of Middle East Technical University. The experimental results demonstrated that, compared with U-Net, the proposed method parameters were reduced by 82.33%, and improved by 11.32% and 12.37% in the key indices of the F1-Score and the mean intersection of union (mIoU), respectively, providing a reference for subsequent related lightweight crack-segmentation research.

Metallic product recognition with dual attention and multi-branch residual blocks-based convolutional neural networks

Visual recognition technologies based on deep learning have been gradually playing an important role in various resource recovery fields. However, in the field of metal resource recycling, there is still a lack of intelligent and accurate recognition of metallic products, which seriously hinders the operation of the metal resource recycling industry chain. In this article, a convolutional neural network with dual attention mechanism and multi-branch residual blocks is proposed to realize the recognition of metallic products with a high accuracy. First, a channel-spatial dual attention mechanism is introduced to enhance the model sensitivity on key features. The model can focus on key features even when extracting features of metallic products with too much confusing information. Second, a deep convolutional network with multi-branch residual blocks as the backbone while embedding a dual-attention mechanism module is designed to satisfy deeper and more effective feature extraction for metallic products with complex characteristic features. To evaluate the proposed model, a waste electrical and electronic equipment (WEEE) dataset containing 9266 images in 18 categories and a waste household metal appliance (WHMA) dataset containing 11,757 images in 23 categories are built. The experimental results show that the accuracy reaches 94.31% and 95.88% in WEEE and WHMA, respectively, achieving high accuracy and high quality recycling.