Multi-scale Attention Network Research Articles

Supersonic combustion field evolution prediction in scramjet engine using a deblurring multi-scale attention network

Traditional deep learning technologies often overlook crucial spatiotemporal information when reconstructing supersonic combustion flow fields. These models can only reconstruct the current flow field, resulting in poor visual quality due to the atomization effect. This research introduces experimental studies on scramjet combustion flows with variable injection pressures at the inlet of a scramjet Mach 2.5, creating image datasets of flame evolution processes over different predicted time spans (5, 10, and 15 ms). It develops a multi-scale attention algorithm with deblurring (DB-MSAN) capabilities to predict the flame image after a certain period using the wall pressure signal from the previous moment. The algorithm employs two multi-head attention channels at different scales to extract timing information, encouraging the model to focus on the anisotropy of the input data and to identify more significant features. The DB module utilizes the adapted atmospheric scattering model to jointly estimate the transmission map and atmospheric light, achieving an exceptional defogging effect with a minimal computational expense. The model’s overall performance is assessed regarding efficiency and accuracy, comparing DB-MSAN’s performance differences with those of Chen’s and the ResNet16 models across multi-span, large-span, various test sets, and sparse wall measurement points. Experimental results indicated that, compared to other methods, DB-MSAN achieves a maximum peak signal-to-noise index improvement of 76.10 % and an SSIM index improvement of 155.26 % when predicting average-quality images. The prediction accuracy remains stable, even with sparse pressure input. In addition, DB-MSAN strikes an optimal balance between the number of parameters and model accuracy; it represents only 16.97 % of the volume compared to Chen’s model, which is 585 MB.

SRMA: a dual-branch parallel multi-scale attention network for remote sensing images sea-land segmentation

ABSTRACT The use of deep learning-based high resolution remote sensing image sea and land segmentation method has become prevalent in various fields such as environmental monitoring, resource assessment, and urban planning. However, the segmentation process faces challenges due to the complex coastline and the similarity in spectral colour difference between the sea and land in certain areas. The remote sensing images obtained from satellites often exhibit unclear boundaries between the sea and land, as well as intricate coastlines. Consequently, the segmentation of sea and land becomes a difficult task. The existing sea-land segmentation methods have limitations in accurately segmenting long and narrow waters, as well as areas with low differentiation between sea and land. In this paper, we propose a new sea-land segmentation method called SRMA (Swin-Res-Multi-Attention). SRMA performs parallel feature extraction using a dual-branch parallel: Swin-Transformer branch and ResNet branch. The combination of Swin-Transformer’s self-attention and ResNet’s residual design helps reduce the loss of feature information, thereby avoiding incorrect segmentation in regions with low distinctness. To improve overall stability, we introduce the MCA (Multiscale channel attention module) module, which combines feature information from different scales. The multi-scale pyramid design of MCA module ensures stability and improves segmentation accuracy in narrow waters. Experimental results on a public dataset demonstrate that our model outperforms existing methods. Additionally, we evaluate the model’s performance in complex scenarios by creating a new dataset that includes a complex urban water body and cloud interference. The results showcase the robustness and superiority of our model.

SCPMan: Shape context and prior constrained multi-scale attention network for pancreatic segmentation

Due to the poor prognosis of Pancreatic cancer, accurate early detection and segmentation are critical for improving treatment outcomes. However, pancreatic segmentation is challenged by blurred boundaries, high shape variability, and class imbalance. To tackle these problems, we propose a multiscale attention network with shape context and prior constraint for robust pancreas segmentation. Specifically, we proposed a Multi-scale Feature Extraction Module (MFE) and a Mixed-scale Attention Integration Module (MAI) to address unclear pancreas boundaries. Furthermore, a Shape Context Memory (SCM) module is introduced to jointly model semantics across scales and pancreatic shape. Active Shape Model (ASM) is further used to model the shape priors. Experiments on NIH and MSD datasets demonstrate the efficiency of our model, which improves the state-of-the-art Dice Score for 1.01% and 1.03% respectively. Our architecture provides robust segmentation performance, against the blurry boundaries, and variations in scale and shape of pancreas.

Multi-scale attention network (MSAN) for track circuits fault diagnosis

As one of the three major outdoor components of the railroad signal system, the track circuit plays an important role in ensuring the safety and efficiency of train operation. Therefore, when a fault occurs, the cause of the fault needs to be found quickly and accurately and dealt with in a timely manner to avoid affecting the efficiency of train operation and the occurrence of safety accidents. This article proposes a fault diagnosis method based on multi-scale attention network, which uses Gramian Angular Field (GAF) to transform one-dimensional time series into two-dimensional images, making full use of the advantages of convolutional networks in processing image data. A new feature fusion training structure is designed to effectively train the model, fully extract features at different scales, and fusing spatial feature information through spatial attention mechanisms. Finally, experiments are conducted using real track circuit fault datasets, and the accuracy of fault diagnosis reaches 99.36%, and our model demonstrates better performance compared to classical and state-of-the-art models. And the ablation experiments verified that each module in the designed model plays a key role.

Multiscale attention network via topology learning for cerebral vessel segmentation in angiography images

Cerebrovascular diseases are a widespread threat to human health. The accurate extraction of cerebral vessel structures is of paramount importance in the diagnosis and treatment of cerebrovascular diseases. However, the complexity of cerebral vessel structures and the low imaging contrast present significant challenges for vessel segmentation. Therefore, we propose a Multiscale Attention Network based on topological learning to extract vessel structures from angiographic images. This method employs a Multiscale Squeeze Attention (MSA) module for channel-wise attention learning, extracting multiscale attention feature maps from angiographic images. To maintain the topological connectivity of vessel segmentation, we introduced the clDice loss function to enforce skeleton connectivity of vessel segmentation. We conducted an experimental analysis of the proposed method using a publicly available cerebral vessel dataset. The results demonstrated that the proposed method achieved a sensitivity score of 0.8507 and a dice score of 0.8669 for cerebrovascular segmentation, enabling accurate and complete extraction of vascular structures. The proposed method was extended to coronary angiography images. The results show that the proposed method can accurately extract coronary structures, proving its broad applicability to other vascular segmentation tasks.

FMA-Net: Fusion of Multi-Scale Attention for Grading Cervical Precancerous Lesions

Cervical cancer, as the fourth most common cancer in women, poses a significant threat to women’s health. Vaginal colposcopy examination, as the most cost-effective step in cervical cancer screening, can effectively detect precancerous lesions and prevent their progression into cancer. The size of the lesion areas in the colposcopic images varies, and the characteristics of the lesions are complex and difficult to discern, thus heavily relying on the expertise of the medical professionals. To address these issues, this paper constructs a vaginal colposcopy image dataset, ACIN-3, and proposes a Fusion Multi-scale Attention Network for the detection of cervical precancerous lesions. First, we propose a heterogeneous receptive field convolution module to construct the backbone network, which utilizes combinations of convolutions with different structures to extract multi-scale features from multiple receptive fields and capture features from different-sized regions of the cervix at different levels. Second, we propose an attention fusion module to construct a branch network, which integrates multi-scale features and establishes connections in both the spatial and channel dimensions. Finally, we design a dual-threshold loss function and introduce positive and negative thresholds to improve sample weights and address the issue of data imbalance in the dataset. Multiple experiments are conducted on the ACIN-3 dataset to demonstrate the superior performance of our approach compared to some classical and recent advanced methods. Our method achieves an accuracy of 92.2% in grading and 94.7% in detection, with average AUCs of 0.9862 and 0.9878. Our heatmap illustrates the accuracy of our approach in focusing on the locations of lesions.

Multiscale attention for few‐shot image classification

AbstractIn recent years, the application of traditional deep learning methods in the agricultural field using remote sensing techniques, such as crop area and growth monitoring, crop classification, and agricultural disaster monitoring, has been greatly facilitated by advancements in deep learning. The accuracy of image classification plays a crucial role in these applications. Although traditional deep learning methods have achieved significant success in remote sensing image classification, they often involve convolutional neural networks with a large number of parameters that require extensive optimization using numerous remote sensing images for training purposes. To address these challenges, we propose a novel approach called multiscale attention network (MAN) for sample‐based remote sensing image classification. This method consists primarily of feature extractors and attention modules to effectively utilize different scale features through multiscale feature training during the training phase. We evaluate our proposed method on three datasets comprising agricultural remote sensing images and observe superior performance compared to existing approaches. Furthermore, we validate its generalizability by testing it on an oil well indicator diagram specifically designed for classification tasks.

Cloud and snow neural network segmentation using the Electro-L No. 2 satellite low-resolution data

This paper is devoted to cloud and snow semantic segmentation using multispectral satellite imagery, received from a multizone scanning instrument (MSU-GS) installed on the Russian satellite Electro-L No. 2. For segmentation, some additional geographic information is used such as latitude, longitude, and altitude. The main difficulty of evaluating snow and cloud masks from MSU-GS data is absence of shortwave infrared (SWIR) spectral channels in the 1.3-1.6μm range, which are necessary for accurate snow and cloud distinction. This fact makes it impossible to apply any of existing algorithms for cloud and snow destinction used for other multispectral satellite imagery. The results of the presented work include a self-collected dataset, consisting of cloud masks from the meteorological satellites GOES-16 and Meteosat-10 L2 products, snow masks from Terra/MODIS products, considered to be the ground truth for multispectral imagery from Electro-L No. 2 with geographical information for each sample, as well as a trained Multi-Scale Attention Network (MANet), able to perform snow and cloud segmentation for multispectral data from Electro-L No. 2. The purpose of this study is to create a stable algorithm for MSU-GS imagery to conduct an operational snow and cloud coverage monitoring of the full Earth disk, that can not be evaluated by using multispectral data from other Russian satellites. The novelty of the work is the combining geographical and multispectral information with weighted sampling approach for hard samples, as well as using other satellite products for creating ground truth masks for needed satellite imagery. The developed algorithm was tested on various scenes during all seasons with F1 and IoU metrics. We reach segmentation quality equal to F1=0.7454,F1=0.8773 and IoU=0.7398,IoU=0.7976 for snow and clouds respectively. The proposed neural network can be used at any time of year during daytime for processing Electro-L No. 2 multispectral data in real time.

Spider-Net: High-resolution multi-scale attention network with full-attention decoder for tumor segmentation in kidney, liver and pancreas

The abdominal tumor is a general term for tumors in kidney, liver and pancreas. Accurate segmentation of abdominal tumors is essential for their treatment. However, the varying shapes and sizes of abdominal organs result in significant differences in tumor regions. Existing convolution neural networks (CNNs) can only accurately segment individual abdominal tumors, lacking sufficient generalizability. We aim to design a network that can achieve good segmentation results for different abdominal tumors. To this end, a Spider-net to segment tumors is presented in this paper, which consists of a high-resolution multi-scale attention encoder and a full-attention decoder. Additionally, scale attention that integrates channel attention and spatial attention is designed for generating output. We have also designed a classification branch to distinguish whether the segmented region is a real tumor area or another benign lesion. We train and evaluate the Spider-net on three different organs: the kidney, pancreas, and liver. Spider-net achieves state-of-the-art results compared to methods that only use CNNs or transformers. Code are available at https://github.com/h2440222798/HRMA.

Multi-Scale Attention Network for Building Extraction from High-Resolution Remote Sensing Images

The precise building extraction from high-resolution remote sensing images holds significant application for urban planning, resource management, and environmental conservation. In recent years, deep neural networks (DNNs) have garnered substantial attention for their adeptness in learning and extracting features, becoming integral to building extraction methodologies and yielding noteworthy performance outcomes. Nonetheless, prevailing DNN-based models for building extraction often overlook spatial information during the feature extraction phase. Additionally, many existing models employ a simplistic and direct approach in the feature fusion stage, potentially leading to spurious target detection and the amplification of internal noise. To address these concerns, we present a multi-scale attention network (MSANet) tailored for building extraction from high-resolution remote sensing images. In our approach, we initially extracted multi-scale building feature information, leveraging the multi-scale channel attention mechanism and multi-scale spatial attention mechanism. Subsequently, we employed adaptive hierarchical weighting processes on the extracted building features. Concurrently, we introduced a gating mechanism to facilitate the effective fusion of multi-scale features. The efficacy of the proposed MSANet was evaluated using the WHU aerial image dataset and the WHU satellite image dataset. The experimental results demonstrate compelling performance metrics, with the F1 scores registering at 93.76% and 77.64% on the WHU aerial imagery dataset and WHU satellite dataset II, respectively. Furthermore, the intersection over union (IoU) values stood at 88.25% and 63.46%, surpassing benchmarks set by DeepLabV3 and GSMC.

Tibia-YOLO: An AssistedDetection System Combined with Industrial CT Equipment for Leg Diseases in Broilers

With the continuous improvement of broiler production performance, the frequent occurrence of leg problems has caused serious economic losses in many factories. In order to more efficiently detect and prevent broiler leg diseases, we propose an auxiliary detection system for broiler leg diseases based on deep learning. The system is divided into two parts. First, a digital radiography (DR) image of a broiler is taken through industrial computed tomography (CT), and then the improved deep-learning network Tibia-YOLO is used to detect the tibia; the detected results are then extracted and measured. Our improved Tibia-YOLO network uses the Content-Aware ReAssembly of Features (CARAFE) upsampling operator to avoid checkerboard artifacts and increase the generalization capabilities. Efficient multi-scale attention (EMA) and parallel network attention (ParNet) were added to the Tibia dataset at multiple scales (COCO2016), and there were improvements when testing on the three VOC2012 datasets. The mean average precision of tibia detection reached 90.8%, and the root mean square error (RMSE) for the tibia length was 3.37 mm.

Multi-scale Attention and Feature Decomposition Network for Surveillance Vehicle Detection

Multi-scale Attention and Feature Decomposition Network for Surveillance Vehicle Detection

Multipath Multiscale Attention Network for Cloud and Cloud Shadow Segmentation

Multipath Multiscale Attention Network for Cloud and Cloud Shadow Segmentation

Reflections on Rhythm Control in the Stage Performance Process of Dancers

Abstract The article proposes a dance score generation model based on the graph attention model and Transformer, which first preprocesses the stage motion capture data of dancers, utilizes a multi-scale attention network to realize motion feature aggregation, and introduces the gated loop unit in the Transformer model to enhance the recognition accuracy of dance movements. Based on the recognition of the dancer’s stage movements, the skeleton size normalization is performed on the joints of the dancer’s gestures, and the skeleton replacement method is combined to realize the correction of the dance movements. Next, we test the model’s dance score generation effect, the matching accuracy of dance movements and music rhythm, and evaluate and analyze the quality of dance movement correction. The results show that the Former model in this paper scores 8.33% higher in beat consistency than the traditional dance generation model Bailando, and the matching coefficient of the rhythmic intensity of the generated dance is basically the same as that of the original dance, with a difference of only 0.02-0.06. It shows that the rhythmic control of the stage performance of dancers carried out by the Transformer model can enhance the stage performance power of dancers and strengthen their stage performance. It shows that using the Transformer model to control the rhythm of dancers’ stage performances can improve the stage performance of dancers and enhance the viewability of stage performances.

Multi-Scale Attention Network for Vertical Seed Distribution in Soybean Breeding Fields.

The increase in the global population is leading to a doubling of the demand for protein. Soybean (Glycine max), a key contributor to global plant-based protein supplies, requires ongoing yield enhancements to keep pace with increasing demand. Precise, on-plant seed counting and localization may catalyze breeding selection of shoot architectures and seed localization patterns related to superior performance in high planting density and contribute to increased yield. Traditional manual counting and localization methods are labor-intensive and prone to error, necessitating more efficient approaches for yield prediction and seed distribution analysis. To solve this, we propose MSANet: a novel deep learning framework tailored for counting and localization of soybean seeds on mature field-grown soy plants. A multi-scale attention map mechanism was applied to maximize model performance in seed counting and localization in soybean breeding fields. We compared our model with a previous state-of-the-art model using the benchmark dataset and an enlarged dataset, including various soybean genotypes. Our model outperforms previous state-of-the-art methods on all datasets across various soybean genotypes on both counting and localization tasks. Furthermore, our model also performed well on in-canopy 360° video, dramatically increasing data collection efficiency. We also propose a technique that enables previously inaccessible insights into the phenotypic and genetic diversity of single plant vertical seed distribution, which may accelerate the breeding process. To accelerate further research in this domain, we have made our dataset and software publicly available: https://github.com/UTokyo-FieldPhenomics-Lab/MSANet.

STMA-Net: A Spatial Transformation-Based Multiscale Attention Network for Complex Defect Detection With X-Ray Images

STMA-Net: A Spatial Transformation-Based Multiscale Attention Network for Complex Defect Detection With X-Ray Images

Player detection method based on scale attention and scale equalization algorithm.

Object detection methods for team ball games players often struggle due to their reliance on dataset scale statistics, resulting in missed detections for players with smaller bounding boxes and reduced accuracy for larger bounding boxes. This study introduces a two-fold approach to address these challenges. Firstly, a novel multi-scale attention mechanism is proposed, aiming to reduce reliance on scale statistics by utilizing a specially created SIoU (Similar to Intersection over Union) label that explicitly represents multi-scale features. This label guides the training of multi-scale attention network modules at two granularity levels. Secondly, an integrated scale equalization algorithm within SIoU labels enhances the detection ability of multi-scale targets in imbalanced samples. Comparative experiments conducted on basketball, volleyball, and ice hockey datasets validate the proposed method. The relative optimal approach demonstrated improvements in the detection accuracy of players with smaller and larger scale bounding boxes by 11%, 7%, 15%, 8%, 9%, and 4%, respectively.

GRFB-UNet: A new multi-scale attention network with group receptive field block for tactile paving segmentation

Tactile paving plays a crucial role in the travel of visually impaired, and assists them to find the way forward. Therefore, it is quite meaningful to identify the regions and trends of tactile paving to support the independent walking of the visually impaired. Visual segmentation technology shows potential to segment the regions of tactile paving, and the shapes of these regions can be used to further check the road trends. To effectively improve the accuracy and robustness of tactile paving segmentation, a novel tactile paving segmentation method that combines UNet network and multi-scale feature extraction is proposed in this work. The structure of group receptive field block (GRFB) has been embedded into the basic UNet network to obtain multi-scale receptive fields of the tactile paving. Aiming to enhance the computational efficiency, the strategy of group convolution is adopted to combine with GRFB module. Meanwhile, small-scale convolution is used after each group convolution to achieve cross-channel information interaction and integration, aiming to extract more abundant high-level features. In this paper, we have constructed the dataset of tactile paving in various scenarios, and labeled them for experimental evaluation. Furthermore, a comparative analysis with the typical networks and structure modules has been demonstrated in details. The experimental results show that the proposed network achieves the best overall performance among those compared networks on tactile paving segmentation, and provides a valuable reference for the segmentation of tactile paving.

Rubber Tree Recognition Based on UAV RGB Multi-Angle Imagery and Deep Learning

The rubber tree (Hevea brasiliensis) is an important tree species for the production of natural latex, which is an essential raw material for varieties of industrial and non-industrial products. Rapid and accurate identification of the number of rubber trees not only plays an important role in predicting biomass and yield but also is beneficial to estimating carbon sinks and promoting the sustainable development of rubber plantations. However, the existing recognition methods based on canopy characteristic segmentation are not suitable for detecting individual rubber trees due to their high canopy coverage and similar crown structure. Fortunately, rubber trees have a defoliation period of about 40 days, which makes their trunks clearly visible in high-resolution RGB images. Therefore, this study employed an unmanned aerial vehicle (UAV) equipped with an RGB camera to acquire high-resolution images of rubber plantations from three observation angles (−90°, −60°, 45°) and two flight directions (SN: perpendicular to the rubber planting row, and WE: parallel to rubber planting rows) during the deciduous period. Four convolutional neural networks (multi-scale attention network, MAnet; Unet++; Unet; pyramid scene parsing network, PSPnet) were utilized to explore observation angles and directions beneficial for rubber tree trunk identification and counting. The results indicate that Unet++ achieved the best recognition accuracy (precision = 0.979, recall = 0.919, F-measure = 94.7%) with an observation angle of −60° and flight mode of SN among the four deep learning algorithms. This research provides a new idea for tree trunk identification by multi-angle observation of forests in specific phenological periods.

MSA-Net: Multi-scale attention network for image splicing localization

MSA-Net: Multi-scale attention network for image splicing localization