Global Attention Mechanism Research Articles

A deep learning approach for automated segmentation of magnetic bright points in the solar photosphere

Context. Magnetic bright points (MBPs) are small, bright, and conspicuous magnetic structures observed in the solar photosphere and are widely recognized as tracers of magnetic flux tubes. Previous studies have underscored the significance of MBPs in elucidating the mechanisms of coronal heating. The continuous advancement of solar telescopes and observation techniques has significantly enhanced the resolution of solar images, enabling a more detailed examination of MBP structures. In light of the growing availability of MBP observation images, the implementation of large-scale automated and precise MBP segmentation methods holds tremendous potential to facilitate significant progress in solar physics research. Aims. The objective of this study is to propose a deep learning network called MBP-TransCNN that enables the automatic and precise pixel-level segmentation of MBPs in large quantities, even with limited annotated data. This network is designed to effectively handle MBPs of various shapes and backgrounds, including those with complex features. Methods. First, we normalized our sample of MBP images. We then followed this with elastic deformation and rotation translation to enhance the images and expand the dataset. Next, a dual-branch encoder was used to extract the features of the MBPs, and a Transformer-based global attention mechanism was used to extract global contextual information, while a convolutional neural network (CNN) was used to extract detailed local information. Afterwards, an edge aware module was proposed to extract detailed edge features of MBPs, which were used to optimize the segmentation results. Focal loss was used during the training process to address the problem of the small number of MBP samples. Results. The average values of precision, recall, F1, pixel accuracy, and intersection over union of the MBP-TransCNN are 0.976, 0.827, 0.893, 0.999, and 0.808, respectively. Experimental results show that the proposed MBP-TransCNN deep learning network can quickly and accurately segment the fine structure of MBPs.

Feature Alignment in Anchor-Free Object Detection

Most anchor-free methods perform object detection using dense recommendation, which assumes that one point can simultaneously conduct accurate category prediction and regression estimation. However, due to different task drivers, valid features for classification and regression may locate at distinct areas in the training phase. This problem is called feature misalignment. To solve it, we propose a new feature alignment method based on anchor-free object detector. Firstly, a global receptive field adaptor (G-RFA) is designed by incorporating the feature pyramid networks (FPN) with the global attention mechanism, and forward features are further fine-tuned with a deformable-subnet (De-Subnet) to remove the influence of redundant contextual information. Then, a new feature filter strategy with a misalignment score is proposed to guide the network to focus on sampling points with aligned features. In addition, we establish mutually independent multi-layer quality distributions to model the priori information of an object on different FPN levels. Equipped with our method, the classification and regression features are aligned, and the generated foreground weight map converges to the centers of classification and regression heatmaps. Experimental results show that without bells and whistles, our method achieves 49.3% AP on MS COCO test-dev under the default 2x training schedule, outperforming related methods. Besides, experiments on PASCAL VOC demonstrate the generalization ability of our method. Code is available at https://github.com/GFENGG/featurealign.

Images Classification Integrating Transformer with Convolutional Neural Network

Convolutional neural networks (CNN) are one of the most widely used deep learning methods in computer vision, which can effectively extract local spatial information from images, but lack global understanding and dependency modelling of image features. As a result, contextual information cannot be fully utilized by the network. For example, on coordinate modelling tasks (such as object detection, image generation, etc.), CNN may not be able to accurately locate or reconstruct the position and shape of objects. In contrast to traditional CNN models such as ResNet, Transformers rely on their global attention mechanism to capture long-distance dependencies between patches. The thesis presents an enhanced lightweight method which integrates Transformer with five convolutional neural layers. Model based on CNN and Transformer is tested on the two benchmark datasets MNIST and CIFAR-10. After a few epochs, the model is convergent and reaches high accuracy of 99.34% in MNIST and 92.04% in CIFAR-10. This model outperforms the single CNN and some state-of-the-art models in classifying both datasets, especially in distinguishing similar images like ‘6’ and ‘9’, ‘bird’ and ‘plane’. These results indicate the model's good robustness and generality.

Application of an Encoder–Decoder Model with Attention Mechanism for Trajectory Prediction Based on AIS Data: Case Studies from the Yangtze River of China and the Eastern Coast of the U.S

With the rapid growth of shipping volumes, ship navigation and path planning have attracted increased attention. To design navigation routes and avoid ship collisions, accurate ship trajectory prediction based on automatic identification system data is required. Therefore, this study developed an encoder–decoder learning model for ship trajectory prediction, to avoid ship collisions. The proposed model includes long short-term memory units and an attention mechanism. Long short-term memory can extract relationships between the historical trajectory of a ship and the current state of encountered ships. Simultaneously, the global attention mechanism in the proposed model can identify interactions between the output and input trajectory sequences, and a multi-head self-attention mechanism in the proposed model is used to learn the feature fusion representation between the input trajectory sequences. Six case studies of trajectory prediction for ship collision avoidance from the Yangtze River of China and the eastern coast of the U.S. were investigated and compared. The results showed that the average mean absolute errors of our model were much lower than those of the classical neural networks and other state-of-the-art models that included attention mechanisms.

Tassel-YOLO: A New High-Precision and Real-Time Method for Maize Tassel Detection and Counting Based on UAV Aerial Images

Tassel is an important part of the maize plant. The automatic detection and counting of tassels using unmanned aerial vehicle (UAV) imagery can promote the development of intelligent maize planting. However, the actual maize field situation is complex, and the speed and accuracy of the existing algorithms are difficult to meet the needs of real-time detection. To solve this problem, this study constructed a large high-quality maize tassel dataset, which contains information from more than 40,000 tassel images at the tasseling stage. Using YOLOv7 as the original model, a Tassel-YOLO model for the task of maize tassel detection is proposed. Our model adds a global attention mechanism, adopts GSConv convolution and a VoVGSCSP module in the neck part, and improves the loss function to a SIoU loss function. For the tassel detection task, the mAP@0.5 of Tassel-YOLO reaches 96.14%, with an average prediction time of 13.5 ms. Compared with YOLOv7, the model parameters and computation cost are reduced by 4.11 M and 11.4 G, respectively. The counting accuracy has been improved to 97.55%. Experimental results show that the overall performance of Tassel-YOLO is better than other mainstream object detection algorithms. Therefore, Tassel-YOLO represents an effective exploration of the YOLO network architecture, as it satisfactorily meets the requirements of real-time detection and presents a novel solution for maize tassel detection based on UAV aerial images.

Segmentation and Angle Calculation of Rice Lodging during Harvesting by a Combine Harvester

Rice lodging not only brings trouble to harvesting but also reduces yield. Therefore, the effective identification of rice lodging is of great significance. In this paper, we have designed a bilinear interpolation upsampling feature fusion module (BIFF) to decompose the quadruple upsampling of the connected part of encoder and decoder into two double upsampling processes and insert the intermediate feature layer in the backbone network for feature fusion in this process. The global attention mechanism(GAM) attention module is added to the feature extraction network, allowing the network to effectively focus on the lodging regions, thus effectively improving the segmentation effect. The average accuracy of the improved network is 93.55%, mrecall is 93.65%, and mIoU is 88.10%, and the feasibility of the improvement is demonstrated by ablation experiments and comparison with other algorithms. In addition, the angle calculation method is designed by combining the detection algorithm, adding a detection head branch to the output results for reading the distance information from the depth camera, and combining the distance information with mechanical analysis to determine the relationship between the angle of the stalk and the vertical direction when the rice is upright, tilted and fallen. A comparison of the calculated angle with the actual measured angle gives a final average error of approximately 5.364%, indicating that the harvest boundary extraction algorithm in this paper is highly accurate and has value for application in real-time harvesting scenarios.

Feature interaction network based on hierarchical decoupled convolution for 3D medical image segmentation.

Manual image segmentation consumes time. An automatic and accurate method to segment multimodal brain tumors using context information rich three-dimensional medical images that can be used for clinical treatment decisions and surgical planning is required. However, it is a challenge to use deep learning to achieve accurate segmentation of medical images due to the diversity of tumors and the complex boundary interactions between sub-regions while limited computing resources hinder the construction of efficient neural networks. We propose a feature fusion module based on a hierarchical decoupling convolution network and an attention mechanism to improve the performance of network segmentation. We replaced the skip connections of U-shaped networks with a feature fusion module to solve the category imbalance problem, thus contributing to the segmentation of more complicated medical images. We introduced a global attention mechanism to further integrate the features learned by the encoder and explore the context information. The proposed method was evaluated for enhance tumor, whole tumor, and tumor core, achieving Dice similarity coefficient metrics of 0.775, 0.900, and 0.827, respectively, on the BraTS 2019 dataset and 0.800, 0.902, and 0.841, respectively on the BraTS 2018 dataset. The results show that our proposed method is inherently general and is a powerful tool for brain tumor image studies. Our code is available at: https://github.com/WSake/Feature-interaction-network-based-on-Hierarchical-Decoupled-Convolution.

Saliency Prediction in 360º Videos with Transformers

We present a model for saliency prediction in 360º videos based on the Transformer architecture. Our model leverages the global attention mechanism in order to represent the temporal dependencies that drive human attention. We compare our model with a current state-of-the-art model and outperform it for all metrics measured.

TGC-YOLOv5: An Enhanced YOLOv5 Drone Detection Model Based on Transformer, GAM & CA Attention Mechanism

Drone detection is a significant research topic due to the potential security threats posed by the misuse of drones in both civilian and military domains. However, traditional drone detection methods are challenged by the drastic scale changes and complex ambiguity during drone flight, and it is difficult to detect small target drones quickly and efficiently. We propose an information-enhanced model based on improved YOLOv5 (TGC-YOLOv5) for fast and accurate detection of small target drones in complex environments. The main contributions of this paper are as follows: First, the Transformer encoder module is incorporated into YOLOv5 to augment attention toward the regions of interest. Second, the Global Attention Mechanism (GAM) is embraced to mitigate information diffusion among distinct layers and amplify the global cross-dimensional interaction features. Finally, the Coordinate Attention Mechanism (CA) is incorporated into the bottleneck part of C3, enhancing the extraction capability of local information for small targets. To enhance and verify the robustness and generalization of the model, a small target drone dataset (SUAV-DATA) is constructed in all-weather, multi-scenario, and complex environments. The experimental results show that based on the SUAV-DATA dataset, the AP value of TGC-YOLOv5 reaches 0.848, which is 2.5% higher than the original YOLOv5, and the Recall value of TGC-YOLOv5 reaches 0.823, which is a 3.8% improvement over the original YOLOv5. The robustness of our proposed model is also verified on the Real-World open-source image dataset, achieving the best accuracy in light, fog, stain, and saturation pollution images. The findings and methods of this paper have important significance and value for improving the efficiency and precision of drone detection.

Adaptive learning Unet‐based adversarial network with CNN and transformer for segmentation of hard exudates in diabetes retinopathy

AbstractAccurate segmentation of hard exudates in early non‐proliferative diabetic retinopathy can assist physicians in taking appropriate treatment in a more targeted manner, in order to avoid more serious damage to vision caused by the deterioration of the disease in the later stages. Here, an Adaptive Learning Unet‐based adversarial network with Convolutional neural network and Transformer (CT‐ALUnet) is proposed for automatic segmentation of hard exudates, combining the excellent local modelling ability of Unet with the global attention mechanism of transformer. Firstly, multi‐scale features are extracted through a CNN dual‐branch encoder. Then, the information fusion of features at adjacent scale is realized and the fused features are selected adaptively to maintain the overall consistency of features by attention‐guided multi‐scale fusion blocks (AGMFB). After that, the high‐level encoded features are input to transformer blocks to extract global contexts. Finally, these features are fused layer‐by‐layer to achieve accurate segmentation of hard exudates. In addition, adversarial training is incorporated into the above segmentation model, which improves Dice scores and MIoU scores by 7.5% and 3%, respectively. Experiments demonstrate that CT‐ALUnet shows more reliable segmentation and stronger generalization ability than other SOTA methods, which lays a good foundation for computer‐assisted diagnosis and assessment of efficacy.

SWSA transformer: A forecasting method of ultra-short-term wind speed from an offshore wind farm using global attention mechanism

Accurate ultra-short-term wind speed forecasting is great significance to ensure large scale integration of wind power into the power grid, but the randomness, instability, and non-linear nature of wind speed make it very difficult to be predicted accurately. To solve this problem, shifted window stationary attention transformer (SWSA transformer) is proposed based on a global attention mechanism for ultra-short-term forecasting of wind speed. SWSA transformer can sufficiently extract these complicated features of wind speed to improve the prediction accuracy of wind speed. First, positional embedding and temporal embedding are added at the bottom of the proposed method structure to mark wind speed series, which enables complicated global features of wind speed to be more effectively extracted by attention. Second, a shifted window is utilized to enhance the ability of attention to capture features from the edge sequences. Third, a stationary attention mechanism is applied to not only extract features of wind speed but also optimize the encoder-decoder network for smoothing wind speed sequences. Finally, the predicted values of wind speed are obtained using the calculation in the decoder network. To verify the proposed method, tests are performed utilizing data from an real offshore wind farm. The results show that the proposed method outperforms many popular models evaluated by many indexes including gated recurrent unit, Gaussian process regression, long-short term memory, shared weight long short-term memory network, and shared weight long short-term memory network -Gaussian process regression, in terms of mean absolute error, mean square error (MSE), root mean square error, mean absolute percentage error, mean square percentage error, and coefficient of determination (R2).

Research on Remaining Useful Life Prediction of Bearings Based on MBCNN-BiLSTM

For safe maintenance and to reduce the risk of mechanical faults, the remaining useful life (RUL) estimate of bearings is significant. The typical methods of bearings’ RUL prediction suffer from low prediction accuracy because of the difficulty in extracting features. With the aim of improving the accuracy of RUL prediction, an approach based on multi-branch improved convolutional network (MBCNN) with global attention mechanism combined with bi-directional long- and short-term memory (BiLSTM) network is proposed for bearings’ RUL prediction. Firstly, the original vibration signal is fast Fourier transformed to obtain the frequency domain signal and then normalized. Secondly, the original signal and the frequency domain signal are input into the designed MBCNN network as two branches to extract the spatial features, and then input into the BiLSTM network to further extract the timing features, and the RUL of bearings is mapped by the fully connected network to achieve the purpose of prediction. Finally, an example validation was performed on a publicly available bearing degradation dataset. Compared with some existing prediction methods, the mean absolute and root mean square errors of the predictions were reduced by “22.2%” to “50.0%” and “26.1%” to “52.8%”, respectively, which proved the effectiveness and feasibility of the proposed method.

Multi-spatio-temporal Fusion Graph Recurrent Network for Traffic Forecasting

Accurate traffic forecasting is crucial for smart city development in the new era. However, the intricate spatial and temporal dependencies in traffic data present significant challenges for prediction accuracy. Existing methods often rely on predefined adjacency matrices to capture Spatio-temporal dependencies, which may not adapt well to the dynamic nature of road traffic. To address these challenges, we propose the Multi-Spatio-temporal Fusion Graph Recurrent Network (MSTFGRN). This innovative approach introduces a data-driven method for generating a weighted adjacency matrix, effectively capturing real-time spatial dependencies that are not adequately captured by predefined matrices. The MSTFGRN also incorporates a novel two-way Spatio-temporal fusion operation to learn hidden dependencies between parallel Spatio-temporal relations at different time points. Additionally, a global attention mechanism is integrated into the Spatio-temporal fusion module, enabling the simultaneous capture of global Spatio-temporal dependencies. Through extensive trials on publicly available highway traffic datasets, our method demonstrates state-of-the-art performance compared to alternative baselines.

Research on Safety Helmet Detection Algorithm Based on Improved YOLOv5s.

Safety helmets are essential in various indoor and outdoor workplaces, such as metallurgical high-temperature operations and high-rise building construction, to avoid injuries and ensure safety in production. However, manual supervision is costly and prone to lack of enforcement and interference from other human factors. Moreover, small target object detection frequently lacks precision. Improving safety helmets based on the helmet detection algorithm can address these issues and is a promising approach. In this study, we proposed a modified version of the YOLOv5s network, a lightweight deep learning-based object identification network model. The proposed model extends the YOLOv5s network model and enhances its performance by recalculating the prediction frames, utilizing the IoU metric for clustering, and modifying the anchor frames with the K-means++ method. The global attention mechanism (GAM) and the convolutional block attention module (CBAM) were added to the YOLOv5s network to improve its backbone and neck networks. By minimizing information feature loss and enhancing the representation of global interactions, these attention processes enhance deep learning neural networks' capacity for feature extraction. Furthermore, the CBAM is integrated into the CSP module to improve target feature extraction while minimizing computation for model operation. In order to significantly increase the efficiency and precision of the prediction box regression, the proposed model additionally makes use of the most recent SIoU (SCYLLA-IoU LOSS) as the bounding box loss function. Based on the improved YOLOv5s model, knowledge distillation technology is leveraged to realize the light weight of the network model, thereby reducing the computational workload of the model and improving the detection speed to meet the needs of real-time monitoring. The experimental results demonstrate that the proposed model outperforms the original YOLOv5s network model in terms of accuracy (Precision), recall rate (Recall), and mean average precision (mAP). The proposed model may more effectively identify helmet use in low-light situations and at a variety of distances.

Rail Surface Defect Detection Based on An Improved YOLOv5s

As the operational time of the railway increases, rail surfaces undergo irreversible defects. Once the defects occur, it is easy for them to develop rapidly, which seriously threatens the safe operation of trains. Therefore, the accurate and rapid detection of rail surface defects is very important. However, in the detection of rail surface defects, there are problems, such as low contrast between defects and the background, large scale differences, and insufficient training samples. Therefore, we propose a rail surface defect detection method based on an improved YOLOv5s in this paper. Firstly, the sample dataset of rail surface defect images was augmented with flip transformations, random cropping, and brightness transformations. Next, a Conv2D and Dilated Convolution(CDConv) module was designed to reduce the amount of network computation. In addition, the Swin Transformer was combined with the Backbone and Neck ends to improve the C3 module of the original network. Then, the global attention mechanism (GAM) was introduced into PANet to form a new prediction head, namely Swin transformer and GAM Prediction Head (SGPH). Finally, we used the Soft-SIoUNMS loss to replace the original CIoU loss, which accelerates the convergence speed of the algorithm and reduces regression errors. The experimental results show that the improved YOLOv5s detection algorithm reaches 96.9% in the average precision of rail surface defect detection, offering the accurate and rapid detection of rail surface defects, which has certain engineering application value.

TCN-GAT multivariate load forecasting model based on SHAP value selection strategy in integrated energy system

Load forecasting is an important prerequisite and foundation for ensuring the rational planning and safe operation of integrated energy systems. In view of the interactive coupling problem among multivariate loads, this paper constructs a TCN-GAT multivariate load forecasting model based on SHAP (Shapley Additive Explanation) value selection strategy. The model uses temporal convolutional networks (TCN) to model the multivariate load time series of the integrated energy system, and applies the global attention mechanism (GAT) to process the output of the network hidden layer state, thereby increasing the weight of key features that affect load changes. The input variables are filtered by calculating the SHAP values of each feature, and then returned to the TCN-GAT model for training to obtain multivariate load forecasting results. This can remove the interference of features with low correlation to the model and improve the forecasting effect. The analysis results of practical examples show that compared with other models, the TCN-GAT multivariate load forecasting model based on SHAP value selection strategy proposed in this paper can further reduce the forecasting error and has better forecasting accuracy and application value.

Pests Identification of IP102 by YOLOv5 Embedded with the Novel Lightweight Module

The development of the agricultural economy is hindered by various pest-related problems. Most pest detection studies only focus on a single pest category, which is not suitable for practical application scenarios. This paper presents a deep learning algorithm based on YOLOv5, which aims to assist agricultural workers in efficiently diagnosing information related to 102 types of pests. To achieve this, we propose a new lightweight convolutional module called C3M, which is inspired by the MobileNetV3 network. Compared to the original convolution module C3, C3M occupies less computing memory and results in a faster inference speed, with the detection precision improved by 4.6%. In addition, the GAM (Global Attention Mechanism) is introduced into the neck of YOLO5, which further improves the detection capability of the model. The experimental results indicate that the C3M-YOLO algorithm performs better than YOLOv5 on IP102, a public dataset consisting of 102 pests. Specifically, the detection precision P is 2.4% higher than that of the original model, and mAP0.75 increased by 1.7%, while the F1-score improved by 1.8%. Furthermore, the mAP0.5 and mAP0.75 of the C3M-YOLO algorithm are higher than those of the YOLOX detection model by 5.1% and 6.2%, respectively.

An Improved YOLOv5 Underwater Detector Based on an Attention Mechanism and Multi-Branch Reparameterization Module

Underwater target detection is a critical task in various applications, including environmental monitoring, underwater exploration, and marine resource management. As the demand for underwater observation and exploitation continues to grow, there is a greater need for reliable and efficient methods of detecting underwater targets. However, the unique underwater environment often leads to significant degradation of the image quality, which results in reduced detection accuracy. This paper proposes an improved YOLOv5 underwater-target-detection network to enhance accuracy and reduce missed detection. First, we added the global attention mechanism (GAM) to the backbone network, which could retain the channel and spatial information to a greater extent and strengthen cross-dimensional interaction so as to improve the ability of the backbone network to extract features. Then, we introduced the fusion block based on DAMO-YOLO for the neck, which enhanced the system’s ability to extract features at different scales. Finally, we used the SIoU loss to measure the degree of matching between the target box and the regression box, which accelerated the convergence and improved the accuracy. The results obtained from experiments on the URPC2019 dataset revealed that our model achieved an mAP@0.5 score of 80.2%, representing a 1.8% and 2.3% increase in performance compared to YOLOv7 and YOLOv8, respectively, which means our method achieved state-of-the-art (SOTA) performance. Moreover, additional evaluations on the MS COCO dataset indicated that our model’s mAP@0.5:0.95 reached 51.0%, surpassing advanced methods such as ViDT and RF-Next, demonstrating the versatility of our enhanced model architecture.

A Survey on Medical Health Records and AI

As the population increases, so do the number of patients getting admitted in hospitals. This generates an overwhelming amount of data within the electronic health records (EHRs) that is impossible to manage manually. This is where machine learning concepts come in handy. The ML algorithms for regression-classification have become increasingly popular within the healthcare sector. Especially for cardiovascular diseases (CVD). Estimation states that by 2030, over 23 million people will die from CVD each year. But, it is estimated that 90% of CVD is preventable. The on time recognition and diagnosis of heart failure from the pre-existing medical records is a way to do so. However, the EHRs are not particularly reliable when it comes to the comparison of structured and unstructured data. This is due to the use of colloquial language and possible existence of sparse content. To tackle this issue, the proposed system uses the KTI-RNN model for recognition of unstructured data and removal of sparse content, TF-IWF model to extract the keyword set, the LDA model to extract the topic word set, and finally, the GA-BiRNN model to identify heart failure from extensive medical texts. The GA-BiRNN model is made up of a bidirectional recurrent neural network model and its output layer embedded with global attention mechanism and gating mechanism.

A CNNA-Based Lightweight Multi-Scale Tomato Pest and Disease Classification Method

Tomato is generally cultivated by transplanting seedlings in ridges and furrows. During growth, there are various types of tomato pests and diseases, making it challenging to identify them simultaneously. To address this issue, conventional convolutional neural networks have been investigated, but they have a large number of parameters and are time-consuming. In this paper, we proposed a lightweight multi-scale tomato pest and disease classification network, called CNNA. Firstly, we constructed a dataset of tomato diseases and pests consisting of 27,193 images with 18 categories. Then, we compressed and optimized the ConvNeXt-Tiny network structure to maintain accuracy while significantly reducing the number of parameters. In addition, we proposed a multi-scale feature fusion module to improve the feature extraction ability of the model for different spot sizes and pests, and we proposed a global channel attention mechanism to enhance the sensitivity of the network model to spot and pest features. Finally, the model was trained and deployed to the Jetson TX2 NX for inference of tomato pests and diseases in video stream data. The experimental results showed that the proposed CNNA model outperformed the pre-trained lightweight models such as MobileNetV3, MobileVit, and ShuffleNetV2 in terms of accuracy and all parameters, with a recognition accuracy of 98.96%. Meanwhile, the error rate, inference time for a single image, network parameters, FLOPs, and model size were only 1%, 47.35 ms, 0.37 M, 237.61 M, and 1.47 MB, respectively.