Attentional Gating Research Articles

A framework for computing angle of progression from transperineal ultrasound images for evaluating fetal head descent using a novel double branch network.

Background: Accurate assessment of fetal descent by monitoring the fetal head (FH) station remains a clinical challenge in guiding obstetric management. Angle of progression (AoP) has been suggested to be a reliable and reproducible parameter for the assessment of FH descent. Methods: A novel framework, including image segmentation, target fitting and AoP calculation, is proposed for evaluating fetal descent. For image segmentation, this study presents a novel double branch segmentation network (DBSN), which consists of two parts: an encoding part receives image input, and a decoding part composed of deformable convolutional blocks and ordinary convolutional blocks. The decoding part includes the lower and upper branches, and the feature map of the lower branch is used as the input of the upper branch to assist the upper branch in decoding after being constrained by the attention gate (AG). Given an original transperineal ultrasound (TPU) image, areas of the pubic symphysis (PS) and FH are firstly segmented using the proposed DBSN, the ellipse contours of segmented regions are secondly fitted with the least square method, and three endpoints are finally determined for calculating AoP. Results: Our private dataset with 313 transperineal ultrasound (TPU) images was used for model evaluation with 5-fold cross-validation. The proposed method achieves the highest Dice coefficient (93.4%), the smallest Average Surface Distance (6.268 pixels) and the lowest AoP difference (5.993°) by comparing four state-of-the-art methods. Similar results (Dice coefficient: 91.7%, Average Surface Distance: 7.729 pixels: AoP difference: 5.110°) were obtained on a public dataset with >3,700 TPU images for evaluating its generalization performance. Conclusion: The proposed framework may be used for the automatic measurement of AoP with high accuracy and generalization performance. However, its clinical availability needs to be further evaluated.

Learning Discriminative Feature Representation for Estimating Smoke Density of Smoky Vehicle Rear

Estimating the smoke density from a single image of the smoky vehicle rear is significant for smoke level estimation and smoky vehicle recognition. However, this is a highly ill-posed problem. To solve it, this paper presents a novel smoke density estimation network (SDENet). First, to reduce the susceptibility of deeper convolutions towards the smoke scale variant and enhance feature diversity, we propose the attention multi-scale encoding blocks based on multi-scale blocks and the convolutional block attention. The multi-scale block exploits the multi-scale features at a granular level within a single basic block. The attention makes our model pay more attention to specific regions which are beneficial to smoke density estimation. Second, the extracted features for smoke recognition may contain background interference information due to the smoke translucency, so we propose semantic-guided feature selection blocks to progressively select smoke-relevant features and suppress background interference features from the encoded feature via global high-level semantic information for learning more discriminative features. Finally, to make the learned features more adaptive to smoke feature resolution and visual appearance, we design attention gate decoding blocks to fuse different features via gate blocks, which enhance the features at spatial locations and channel locations where the features are essential for smoke density estimation. Extensive experiments on smoke density estimation show that our model achieves the best performance among existing methods.

RAG-TCGCN: Aspect Sentiment Analysis Based on Residual Attention Gating and Three-Channel Graph Convolutional Networks

Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment analysis task that mainly judges the polarity of a given aspect word in a review. Current methods mainly use graph networks to do aspect-level sentiment classification tasks, most of which use syntactic or semantic graphs, and utilize attention mechanisms to interact and correlate aspect terms and contexts to obtain more useful feature representations. However, these methods may ignore some insignificant syntactic structures and some implicit information in some sentences. The attention mechanism then easily loses the original information, which eventually leads to inaccurate sentiment analysis. In order to solve this problem, this paper proposes a model based on residual attention gating and three-channel graph convolutional network (RAG-TCGCN). Firstly, the model uses a three-channel network composed of syntactic information, semantic information, and public information to simultaneously optimize and fuse through the multi-head attention mechanism to solve the problem of sentences without significant syntactic structure and with implicit information. Through the residual attention gating mechanism the problem of loss of original information is solved. Experimental verification shows that the accuracy and F1 value of the model are improved on the three public datasets.

Soybean Seedling Root Segmentation Using Improved U-Net Network.

Soybean seedling root morphology is important to genetic breeding. Root segmentation is a key technique for identifying root morphological characteristics. This paper proposed a semantic segmentation model of soybean seedling root images based on an improved U-Net network to address the problems of the over-segmentation phenomenon, unsmooth root edges and root disconnection, which are easily caused by background interference such as water stains and noise, as well as inconspicuous contrast in soybean seedling images. Soybean seedling root images in the hydroponic environment were collected for annotation and augmentation. A double attention mechanism was introduced in the downsampling process, and an Attention Gate mechanism was added in the skip connection part to enhance the weight of the root region and suppress the interference of background and noise. Then, the model prediction process was visually interpreted using feature maps and class activation mapping maps. The remaining background noise was removed by connected component analysis. The experimental results showed that the Accuracy, Precision, Recall, F1-Score and Intersection over Union of the model were 0.9962, 0.9883, 0.9794, 0.9837 and 0.9683, respectively. The processing time of an individual image was 0.153 s. A segmentation experiment on soybean root images was performed in the soil-culturing environment. The results showed that this proposed model could extract more complete detail information and had strong generalization ability. It can achieve accurate root segmentation in soybean seedlings and provide a theoretical basis and technical support for the quantitative evaluation of the root morphological characteristics in soybean seedlings.

Foreground-Aware Refinement Network for Building Extraction from Remote Sensing Images

To extract buildings accurately, we propose a foreground-aware refinement network for building extraction. In particular, in order to reduce the false positive of buildings, we design the foreground-aware module using the attention gate block, which effectively suppresses the features of nonbuilding and enhances the sensitivity of the model to buildings. In addition, we introduce the reverse attention mechanism in the detail refinement module. Specifically, this module guides the network to learn to supplement the missing details of the buildings by erasing the currently predicted regions of buildings and achieves more accurate and complete building extraction. To further optimize the network, we design hybrid loss, which combines BCE loss and SSIM loss, to supervise network learning from both pixel and structure layers. Experimental results demonstrate the superiority of our network over state-of-the-art methods in terms of both quantitative metrics and visual quality.

Recurrent attention unit: A new gated recurrent unit for long-term memory of important parts in sequential data

Gated recurrent unit (GRU) is a variant of the recurrent neural network (RNN). It has been widely used in many applications, such as handwriting recognition and natural language processing. However, GRU can only memorize the sequential information, but lacks the capability of adaptively paying attention to important parts in the sequences. In this paper, we propose a novel RNN model, called recurrent attention unit (RAU), which can seamlessly integrate the attention mechanism into the interior of the GRU cell by adding an attention gate. The attention gate enhances the ability of RAU to remember long-term information and pay attention to important parts in the sequential data. Extensive experiments on adding problem, image classification, sentiment classification and language modeling show that RAU consistently outperforms GRU and other related models.

Structure-Preserving Random Noise Attenuation Method for Seismic Data Based on a Flexible Attention CNN

The noise attenuation of seismic data is an indispensable part of seismic data processing, directly impacting the following inversion and imaging. This paper focuses on two bottlenecks in the AI-based denoising method of seismic data: the destruction of structural information of seismic data and the inferior generalizability. We propose a flexible attention-CNN (FACNN) and realized the denoising work of seismic data. This paper’s main work and advantages were concentrated on the following three aspects: (i) We propose attention gates (AGs), which progressively suppressed features in irrelevant background parts and improved the denoising performance. (ii) We added a noise level map M as an additional channel, making a single CNN model expected to inherit the flexibility of handling noise models with different parameters, even spatially variant noises. (iii) We propose a mixed loss function based on MS_SSIM to improve the performance of FACNN further. Adding the noise level map can improve the network’s generalization ability, and adding the attention structure with the mixed loss function can better protect the structural information of the seismic data. The numerical tests showed that our method has better generalization and can better protect the details of seismic events.

Segmentation of kidney mass using AgDenseU-Net 2.5D model

The Kidney and Kidney Tumor Segmentation Challenge 2021 (KiTS21) released a kidney CT dataset with 300 patients. Unlike KiTS19, KiTS21 provided a cyst category. Therefore, the segmentation of kidneys, tumors, and cysts will be able to assess the complexity and aggressiveness of kidney mass. Deep learning models can save medical resources, but 3D models still have some disadvantages, such as the high cost of computing resources. This paper proposes a scheme that saves computing resources and achieves the segmentation of kidney mass in two steps. First, we preprocess the kidney volume data using the automatic down-sampling method of 3D images, reducing the volume while preserving the feature information. Second, we finely segment kidneys, tumors, and cysts using the AgDenseU-Net (Attention gate DenseU-Net) 2.5D model. KiTS21 proposed using Hierarchical Evaluation Classes (HECs) to compute a metric for the superset: the HEC of kidney considers kidneys, tumors, and cysts as the foreground to compute segmentation performance; the HEC of kidney mass considers both tumor and cyst as the foreground classes; the HEC of tumor considers tumor as the foreground only. For KiTS21, our model achieved a dice score of 0.971 for the kidney, 0.883 for the mass, and 0.815 for the tumor. In addition, we also tested segmentation results without HECs, and our model achieved a dice score of 0.950 for the kidney, 0.878 for the tumor, and 0.746 for the cyst. The results demonstrate that the method proposed in this paper can be used as a reference for kidney tumor segmentation.

Deep learning-based computed tomography image segmentation and volume measurement of intracerebral hemorrhage.

The study aims to enhance the accuracy and practicability of CT image segmentation and volume measurement of ICH by using deep learning technology. A dataset including the brain CT images and clinical data of 1,027 patients with spontaneous ICHs treated from January 2010 to December 2020 were retrospectively analyzed, and a deep segmentation network (AttFocusNet) integrating the focus structure and the attention gate (AG) mechanism is proposed to enable automatic, accurate CT image segmentation and volume measurement of ICHs. In internal validation set, experimental results showed that AttFocusNet achieved a Dice coefficient of 0.908, an intersection-over-union (IoU) of 0.874, a sensitivity of 0.913, a positive predictive value (PPV) of 0.957, and a 95% Hausdorff distance (HD95) (mm) of 5.960. The intraclass correlation coefficient (ICC) of the ICH volume measurement between AttFocusNet and the ground truth was 0.997. The average time of per case achieved by AttFocusNet, Coniglobus formula and manual segmentation is 5.6, 47.7, and 170.1 s. In the two external validation sets, AttFocusNet achieved a Dice coefficient of 0.889 and 0.911, respectively, an IoU of 0.800 and 0.836, respectively, a sensitivity of 0.817 and 0.849, respectively, a PPV of 0.976 and 0.981, respectively, and a HD95 of 5.331 and 4.220, respectively. The ICC of the ICH volume measurement between AttFocusNet and the ground truth were 0.939 and 0.956, respectively. The proposed segmentation network AttFocusNet significantly outperforms the Coniglobus formula in terms of ICH segmentation and volume measurement by acquiring measurement results closer to the true ICH volume and significantly reducing the clinical workload.

Multi-class motion-based semantic segmentation for ureteroscopy and laser lithotripsy.

Ureteroscopy with laser lithotripsy has evolved as the most commonly used technique for the treatment of kidney stones. Automated segmentation of kidney stones and the laser fiber is an essential initial step to performing any automated quantitative analysis, particularly stone-size estimation, that can be used by the surgeon to decide if the stone requires further fragmentation. However, factors such as turbid fluid inside the cavity, specularities, motion blur due to kidney movements and camera motion, bleeding, and stone debris impact the quality of vision within the kidney, leading to extended operative times. To the best of our knowledge, this is the first attempt made towards multi-class segmentation in ureteroscopy and laser lithotripsy data. We propose an end-to-end convolution neural network (CNN) based learning framework for the segmentation of stones and laser fiber. The proposed approach utilizes two sub-networks: (I) HybResUNet, a hybrid version of residual U-Net, that uses residual connections in the encoder path of the U-Net to improve semantic predictions, and (II) a DVFNet that generates deformation vector field (DVF) predictions by leveraging motion differences between the adjacent video frames which is then used to prune the prediction maps. We also present ablation studies that combine different dilated convolutions, recurrent and residual connections, atrous spatial pyramid pooling, and attention gate models. Further, we propose a compound loss function that significantly boosts the segmentation performance in our data. We have also provided an ablation study to determine the optimal data augmentation strategy for our dataset. Our qualitative and quantitative results illustrate that our proposed method outperforms state-of-the-art methods such as UNet and DeepLabv3+ showing a DSC improvement of 4.15% and 13.34%, respectively, in our in vivo test dataset. We further show that our proposed model outperforms state-of-the-art methods on an unseen out-of-sample clinical dataset with a DSC improvement of 9.61%, 11%, and 5.24% over UNet, HybResUNet, and DeepLabv3+, respectively in the case of the stone class and an improvement of 31.79%, 22.15%, and 10.42% over UNet, HybResUNet, and DeepLabv3+, respectively, in case of the laser class.

Federated learning-based vertebral body segmentation

To ensure the safety of spinal surgery, sufficiently labeled Magnetic Resonance Imaging (MRI) images are essential for training an accurate vertebral segmentation model, but the number of labeled MRI images owned by the independent medical institution such as the hospital is generally limited. Besides, in consideration of patients’ privacy, annotated images are difficult to share directly as the medical data to train vertebral body segment models. To address these challenges, a Federated Learning-based Vertebral Body Segment Framework (FLVBSF) is proposed in this work, which includes a novel local Dual Attention Gates (DAGs)-based attention mechanism and a global federated learning framework. The model sensitivity to vertebral body pixels and segmentation accuracy can be improved by using the DAGs. The performance of vertebral body segmentation models is boosted by the global federated learning framework via collaboratively exploiting the labeled spine image data from different institutions. The centralized training-based experimental results show that 98.29% in pixel-level accuracy is achieved by the U-Net with DAGs, 88.04% in dice similarity coefficient, 88.25% in sensitivity, 99.16% in specificity, and 79.09% in Jaccard similarity coefficient and the mean segmentation time per case is 0.14 s. Meanwhile, the federated learning-based experimental results show that the proposed FLVBSF can enhance the performance of the vertebral segmentation model by a statistically significant margin.

Marine Oil Spill Detection from SAR Images Based on Attention U-Net Model Using Polarimetric and Wind Speed Information.

With the rapid development of marine trade, marine oil pollution is becoming increasingly severe, which can exert damage to the health of the marine environment. Therefore, detection of marine oil spills is important for effectively starting the oil-spill cleaning process and the protection of the marine environment. The polarimetric synthetic aperture radar (PolSAR) technique has been applied to the detection of marine oil spills in recent years. However, most current studies still focus on using the simple intensity or amplitude information of SAR data and the detection results are not reliable enough. This paper presents a deep-learning-based method to detect oil spills on the marine surface from Sentinel-1 PolSAR satellite images. Specifically, attention gates are added to the U-Net network architecture, which ensures that the model focuses more on feature extraction. In the training process of the model, sufficient Sentinel-1 PolSAR images are selected as sample data. The polarimetric information from the PolSAR dataset and the wind-speed information of the marine surface are both taken into account when training the model and detecting oil spills. The experimental results show that the proposed method achieves better performance than the traditional methods, and taking into account both the polarimetric and wind-speed information, can indeed improve the oil-spill detection results. In addition, the model shows pleasing performance in capturing the fine details of the boundaries of the oil-spill patches.

Multi-target segmentation of pancreas and pancreatic tumor based on fusion of attention mechanism

Existing neural network segmentation schemes perform well in the task of segmenting images of organs with large areas and clear morphology, such as the liver and lungs. However, it is difficult to segment organs with variable morphology and small target area, such as pancreas and tumors. In order to achieve accurate segmentation of pancreas and its cysts, MDAG-Net (Multi-dimensional Attention Gate Network) is proposed in this paper. Combining three attention mechanisms: spatial, channel and multi-dimensional feature map input, MDAG(Multi-dimensional Attention Gate) obtains the global distribution of semantic information in spatial and channel dimensions, filters redundant information in shallow feature maps, realizes feature response, and recalibrates convolution kernel parameters. In addition, the WML(Weighted cross entropy and MIoU loss function) loss can adaptively assign the weight of category loss and count the classification error of global pixels, which can increase the error attention of the target area and improving the segmentation accuracy of the network. The algorithm is experimented on the Task07_Pancreas dataset, compared with U-Net under the same conditions, the Dice coefficient, Precision, Recall rate and MIoU (Mean Intersection over Union) of MDAG-Net are improved by 5.3%, 1.5%, 12.7% and 7.6% respectively. The results show that MDAG-Net can accurately segment the region of pancreas and its cyst in CT(Computed Tomography) images, which proves that MDAG has better segmentation efficiency for such small target regions.

Deep learning approaches based improved light weight U-Net with attention module for optic disc segmentation.

Glaucoma is a major cause of blindness worldwide, and its early detection is essential for the timely management of the condition. Glaucoma-induced anomalies of the optic nerve head may cause variation in the Optic Disc (OD) size. Therefore, robust OD segmentation techniques are necessary for the screening for glaucoma. Computer-aided segmentation has become a promising diagnostic tool for the early detection of glaucoma, and there has been much interest in recent years in using neural networks for medical image segmentation. This study proposed an enhanced lightweight U-Net model with an Attention Gate (AG) to segment OD images. We also used a transfer learning strategy to extract relevant features using a pre-trained EfficientNet-B0 CNN, which preserved the receptive field size and AG, which reduced the impact of gradient vanishing and overfitting. Additionally, the neural network trained using the binary focal loss function improved segmentation accuracy. The pre-trained Attention U-Net was validated using publicly available datasets, such as DRIONS-DB, DRISHTI-GS, and MESSIDOR. The model significantly reduced parameter quantity by around 0.53 M and had inference times of 40.3 ms, 44.2 ms, and 60.6 ms, respectively.

Automatic localization of target point for subthalamic nucleus-deep brain stimulation via hierarchical attention-UNet based MRI segmentation.

Deep brain stimulation of the subthalamic nucleus (STN-DBS) is an effective treatment for patients with advanced Parkinson's disease, the outcome of this surgery is highly dependent on the accurate placement of the electrode in the optimal target of STN. In this study, we aim to develop a target localization pipeline for DBS surgery, considering that the heart of this matter is to achieve the STN and red nucleus segmentation, a deep learning-based automatic segmentation approach is proposed to tackle this issue. To address the problems of ambiguous boundaries and variable shape of the segmentation targets, the hierarchical attention mechanism with two different attention strategies is integrated into an encoder-decoder network for mining both semantics and fine-grained details for segmentation. The hierarchical attention mechanism is utilized to suppress irrelevant regions in magnetic resonance (MR) images while build long-range dependency among segmentation targets. Specifically, the attention gate (AG) is integrated into low-level features to suppress irrelevant regions in an input image while highlighting the salient features useful for segmentation. Besides, the self-attention involved in the transformer block is integrated into high-level features to model the global context. Ninety-nine brain magnetic resonance imaging (MRI) studies were collected from 99 patients with Parkinson's disease undergoing STN-DBS surgery, among which 80 samples were randomly selected as the training datasets for deep learning training, and ground truths (segmentation masks) were manually generated by radiologists. We applied five-fold cross-validation on these data to train our model, the mean results on 19 test samples are used to conduct the comparison experiments, the Dice similarity coefficient (DSC), Jaccard (JA), sensitivity (SEN), and HD95 of the segmentation for STN are 88.20%, 80.32%, 90.13%, and 1.14mm, respectively, outperforming the state-of-the-art STN segmentation method with 2.82%, 4.52%, 2.56%, and 0.02mm respectively. The source code and trained models of this work have been released in the URL below: https://github.com/liuruiqiang/HAUNet/tree/master. In this study, we demonstrate the effectiveness of the hierarchical attention mechanism for building global dependency on high-level semantic features and enhancing the fine-grained details on low-level features, the experimental results show that our method has considerable superiority for STN and red nucleus segmentation, which can provide accurate target localization for STN-DBS.

Gated Adaptive Hierarchical Attention Unit Neural Networks for the Life Prediction of Servo Motors

The remaining useful life (RUL) prediction of servo motors plays a greatly indispensable role in preventing abnormal and inefficient operation of servo drive system and even major safety accident. In this article, a novel variant of recurrent neural networks, called gated adaptive hierarchical attention unit (GAHAU), is constructed to efficiently predict the RULs of servo motors, which can not only pay attention to the outputs of reset and update gates but also adaptively divide the hierarchy of the sequence information in attention gate. The updating formulas of GAHAU are derived. With the raw vibration signals of servo motors, a quadratic function-based deep convolutional autoencoder is used for automatically generating the health indicator vector. Based on the constructed HI vector, the RULs of servo motors are accurately predicted by GAHAU. The experiments on servo motors demonstrate that the presented GAHAU has remarkable prediction performance especially for the long-term prediction. Compared to the existing typical RUL prediction methods, the proposed methodology has stronger predictive ability according to the comparative results.

Nested U-Net with Enhanced Attention Gate and Compound Loss for Semantic Segmentation of Brain Tumor from Multimodal MRI

Nested U-Net with Enhanced Attention Gate and Compound Loss for Semantic Segmentation of Brain Tumor from Multimodal MRI

Learning to detect boundary information for brain image segmentation

MRI brain images are always of low contrast, which makes it difficult to identify to which area the information at the boundary of brain images belongs. This can make the extraction of features at the boundary more challenging, since those features can be misleading as they might mix properties of different brain regions. Hence, to alleviate such a problem, image boundary detection plays a vital role in medical image segmentation, and brain segmentation in particular, as unclear boundaries can worsen brain segmentation results. Yet, given the low quality of brain images, boundary detection in the context of brain image segmentation remains challenging. Despite the research invested to improve boundary detection and brain segmentation, these two problems were addressed independently, i.e., little attention was paid to applying boundary detection to brain segmentation tasks. Therefore, in this paper, we propose a boundary detection-based model for brain image segmentation. To this end, we first design a boundary segmentation network for detecting and segmenting images brain tissues. Then, we design a boundary information module (BIM) to distinguish boundaries from the three different brain tissues. After that, we add a boundary attention gate (BAG) to the encoder output layers of our transformer to capture more informative local details. We evaluate our proposed model on two datasets of brain tissue images, including infant and adult brains. The extensive evaluation experiments of our model show better performance (a Dice Coefficient (DC) accuracy of up to 5.3% compared to the state-of-the-art models) in detecting and segmenting brain tissue images.

MSAA-Net: a multi-scale attention-aware U-Net is used to segment the liver

Automatic segmentation of the liver from CT images is a very challenging task because the shape of the liver in the abdominal cavity varies from person to person and it also often fits closely with other organs. In recent years, with the continuous development of deep learning and the proposal of CNN, the neural network-based segmentation models have shown good performance in the field of image segmentation. Among the many network models, U-Net stands out in the task of medical image segmentation. In this paper, we propose a segmentation network MSAA-Net combining multi-scale features and an improved attention-aware U-Net. We extracted features of different scales on a single feature layer and performed attention perception in the channel dimension. We demonstrate that this architecture improves the performance of U-Net, while significantly reducing computational costs. To address the problem that U-Net’s skip connection is difficult to optimize for merging objects of different sizes, we designed a multi-scale attention gate structure (MAG), which allows the model to automatically learn to focus on targets of different sizes. In addition, MAG can be extended to all structures which contain skip connections, such as U-Net and FCN variants. Our structure was extensively evaluated on the 3Dircadb dataset, and the DICE similarity coefficient of the method for the liver segmentation task was 94.42%, with a much smaller number of model parameters than other attentional models. The experimental results show that MSAA-Net achieves very competitive performance in liver segmentation.

Temperature compensation to guided wave-based monitoring of bolt loosening using an attention-based multi-task network

Online monitoring of bolt torque is critical to ensure the safe operation of bolted structures. Guided waves have been intensively explored for bolt loosening monitoring. Nevertheless, guided waves are excessively sensitive to fluctuation of ambient temperature. As a result of the complexity of wave transmitting across a bolted joint, it is highly challenging to compensate for the effect of temperature. To this end, an attention-based multi-task network is developed towards accurate detection of bolt loosening in multi-bolt connections over a wide range of temperature variation. By integrating improved attention gate modules in a modified U-Net architecture, an attention U-Net is configured for temperature compensation. A two-layer convolutional subnetwork is connected in series behind the attention U-Net to identify bolt loosening. Experimental validation is carried out on a bolt jointed lap plate simulating a real aircraft structure. The results have proved that the developed multi-task network achieves temperature compensation and accurate bolt loosening identification. To further understand the multi-task network, the Integrated Gradients method and a simplified structure of the bolt lap plate are used to interpret the developed network. It is proved that the A0 mode is sensitive to bolt loosening, while the S0 mode is not.