Multi-scale Attention Research Articles

Lesion-aware cross-phase attention network for renal tumor subtype classification on multi-phase CT scans

Multi-phase computed tomography (CT) has been widely used for the preoperative diagnosis of kidney cancer due to its non-invasive nature and ability to characterize renal lesions. However, since enhancement patterns of renal lesions across CT phases are different even for the same lesion type, the visual assessment by radiologists suffers from inter-observer variability in clinical practice. Although deep learning-based approaches have been recently explored for differential diagnosis of kidney cancer, they do not explicitly model the relationships between CT phases in the network design, limiting the diagnostic performance. In this paper, we propose a novel lesion-aware cross-phase attention network (LACPANet) that can effectively capture temporal dependencies of renal lesions across CT phases to accurately classify the lesions into five major pathological subtypes from time-series multi-phase CT images. We introduce a 3D inter-phase lesion-aware attention mechanism to learn effective 3D lesion features that are used to estimate attention weights describing the inter-phase relations of the enhancement patterns. We also present a multi-scale attention scheme to capture and aggregate temporal patterns of lesion features at different spatial scales for further improvement. Extensive experiments on multi-phase CT scans of kidney cancer patients from the collected dataset demonstrate that our LACPANet outperforms state-of-the-art approaches in diagnostic accuracy.

FAGD-Net: Feature-Augmented Grasp Detection Network Based on Efficient Multi-Scale Attention and Fusion Mechanisms

Grasping robots always confront challenges such as uncertainties in object size, orientation, and type, necessitating effective feature augmentation to improve grasping detection performance. However, many prior studies inadequately emphasize grasp-related features, resulting in suboptimal grasping performance. To address this limitation, this paper proposes a new grasping approach termed the Feature-Augmented Grasp Detection Network (FAGD-Net). The proposed network incorporates two modules designed to enhance spatial information features and multi-scale features. Firstly, we introduce the Residual Efficient Multi-Scale Attention (Res-EMA) module, which effectively adjusts the importance of feature channels while preserving precise spatial information within those channels. Additionally, we present a Feature Fusion Pyramidal Module (FFPM) that serves as an intermediary between the encoder and decoder, effectively addressing potential oversights or losses of grasp-related features as the encoder network deepens. As a result, FAGD-Net achieved advanced levels of grasping accuracy, with 98.9% and 96.5% on the Cornell and Jacquard datasets, respectively. The grasp detection model was deployed on a physical robot for real-world grasping experiments, where we conducted a series of trials in diverse scenarios. In these experiments, we randomly selected various unknown household items and adversarial objects. Remarkably, we achieved high success rates, with a 95.0% success rate for single-object household items, 93.3% for multi-object scenarios, and 91.0% for cluttered scenes.

EMR-HRNet: A Multi-Scale Feature Fusion Network for Landslide Segmentation from Remote Sensing Images.

Landslides constitute a significant hazard to human life, safety and natural resources. Traditional landslide investigation methods demand considerable human effort and expertise. To address this issue, this study introduces an innovative landslide segmentation framework, EMR-HRNet, aimed at enhancing accuracy. Initially, a novel data augmentation technique, CenterRep, is proposed, not only augmenting the training dataset but also enabling the model to more effectively capture the intricate features of landslides. Furthermore, this paper integrates a RefConv and Multi-Dconv Head Transposed Attention (RMA) feature pyramid structure into the HRNet model, augmenting the model's capacity for semantic recognition and expression at various levels. Last, the incorporation of the Dilated Efficient Multi-Scale Attention (DEMA) block substantially widens the model's receptive field, bolstering its capability to discern local features. Rigorous evaluations on the Bijie dataset and the Sichuan and surrounding area dataset demonstrate that EMR-HRNet outperforms other advanced semantic segmentation models, achieving mIoU scores of 81.70% and 71.68%, respectively. Additionally, ablation studies conducted across the comprehensive dataset further corroborate the enhancements' efficacy. The results indicate that EMR-HRNet excels in processing satellite and UAV remote sensing imagery, showcasing its significant potential in multi-source optical remote sensing for landslide segmentation.

Efficient multi-scale network for semantic segmentation of fine-resolution remotely sensed images

Abstract Semantic segmentation of remote sensing urban scene images has diverse practical applications, including land cover mapping, urban change detection, environmental protection, and economic evaluation. However, classical semantic segmentation networks encounter challenges such as inadequate utilization of multi-scale semantic information and imprecise edge target segmentation in high-resolution remote sensing images. In response, this article introduces an efficient multi-scale network (EMNet) tailored for semantic segmentation of common features in remote sensing images. To address these challenges, EMNet integrates several key components. Firstly, the efficient atrous spatial pyramid pooling module is employed to enhance the relevance of multi-scale targets, facilitating improved extraction and processing of context information across different scales. Secondly, the efficient multi-scale attention mechanism and multi-scale jump connections are utilized to fuse semantic features from various levels, thereby achieving precise segmentation boundaries and accurate position information. Finally, an encoder-decoder structure is incorporated to refine the segmentation results. The effectiveness of the proposed network is validated through experiments conducted on the publicly available DroneDeploy image dataset and Potsdam dataset. Results indicate that EMNet achieves impressive performance metrics, with mean intersection over union (MIoU), mean precision (MPrecision), and mean recall (MRecall) reaching 75.99%, 86.76%, and 85.07%, respectively. Comparative analysis demonstrates that the network proposed in this article outperforms current mainstream semantic segmentation networks on both the DroneDeploy and Potsdam dataset.

Research on image classification based on residual group multi-scale enhanced attention network

To address the challenges associated with insufficient feature extraction and gradient degradation encountered when dealing with deepening network structures in image classification tasks, this paper presents a ResGMEANet(Residual Group Multi-scale Enhanced Attention Network). The model introduces a multi-scale attention enhancement module. This design draws inspiration from the original model's capability to independently capture feature correlations in channels and spaces. By implementing shuffle operations and feature transformations within the group, our method expands the receptive field through the utilization of multiple convolution kernels. Additionally, we incorporate an improved tensor synthesis attention, building upon the traditional convolution attention, to derive attention feature maps after feature enhancement. Evaluation on the CIFAR-10 and CIFAR-100 datasets shows that ResGMEANet outperforms both the original backbone model and several existing mainstream methods in classification accuracy. This work aims to provide a new perspective for the future by combining residual neural networks with different attention mechanisms.

Multi-Scale Attention Based Channel Estimation for RIS-Aided Massive MIMO Systems

Multi-Scale Attention Based Channel Estimation for RIS-Aided Massive MIMO Systems

A Video Action Recognition Method via Dual-Stream Feature Fusion Neural Network with Attention

Video action recognition is a technique for automatically determining the category of a video action. It is necessary to design an efficient video action recognition algorithm to predict video labels. This work proposes a video action recognition model based on dual-stream information fusion with attention mechanisms (DSIFAM), which consists of three different sub-modules. First, this proposes an improved keyframe extraction method (IKFE). Based on K-means clustering, this uses convolutional features to calculate the similarity between video frames instead of pixel points. After obtaining preliminary clustering results, the method performs secondary optimization to obtain more representative keyframes. Second, this proposes a video action recognition model based on dual-stream information fusion (DSIF). The method introduces ConvLSTM in the spatial stream and uses P3D instead of the original convolutional network in the temporal stream, which can better extract spatial-temporal information and improve the classification performance. Third, this designs a multi-scale attention mechanism (MSAM) to enhance the feature extraction stage and obtain higher quality classification features. The resulting features are more prominent and have stronger representation capabilities. Finally, this work conducts systematic experiments on different datasets, the results verify the superiority of DSIFAM for video action recognition.

RSDNet: A New Multiscale Rail Surface Defect Detection Model.

The rapid and accurate identification of rail surface defects is critical to the maintenance and operational safety of the rail. For the problems of large-scale differences in rail surface defects and many small-scale defects, this paper proposes a rail surface defect detection algorithm, RSDNet (Rail Surface Defect Detection Net), with YOLOv8n as the baseline model. Firstly, the CDConv (Cascade Dilated Convolution) module is designed to realize multi-scale convolution by cascading the cavity convolution with different cavity rates. The CDConv is embedded into the backbone network to gather earlier defect local characteristics and contextual data. Secondly, the feature fusion method of Head is optimized based on BiFPN (Bi-directional Feature Pyramids Network) to fuse more layers of feature information and improve the utilization of original information. Finally, the EMA (Efficient Multi-Scale Attention) attention module is introduced to enhance the network's attention to defect information. The experiments are conducted on the RSDDs dataset, and the experimental results show that the RSDNet algorithm achieves a mAP of 95.4% for rail surface defect detection, which is 4.6% higher than the original YOLOv8n. This study provides an effective technical means for rail surface defect detection that has certain engineering applications.

Construction of multi-scale feature fusion segmentation model of MRI knee images based on dual attention mechanism weighted aggregation.

Early diagnosis of knee osteoarthritis is an important area of research in the field of clinical medicine. Due to the complexity in the MRI imaging sequences and the diverse structure of cartilage, there are many challenges in the segmentation of knee bone and cartilage. Relevant studies have conducted semantic fusion processing through splicing or summing forms, which results in reduced resolution and the accumulation of redundant information. This study was envisaged to construct an MRI image segmentation model to improve the diagnostic efficiency and accuracy of different grade knee osteoarthritis by adopting the Dual Attention and Multi-scale Feature Fusion Segmentation network (DA-MFFSnet). The feature information of different scales was fused through the Multi-scale Attention Downsample module to extract more accurate feature information, and the Global Attention Upsample module weighted lower-level feature information to reduce the loss of key information. The collected MRI knee images were screened and labeled, and the study results showed that the segmentation effect of DA-MFFSNet model was closer to that of the manually labeled images. The mean intersection over union, the dice similarity coefficient and the volumetric overlap error was 92.74%, 91.08% and 7.44%, respectively, and the accuracy of the differential diagnosis of knee osteoarthritis was 84.42%. The model exhibited better stability and classification effect. Our results indicated that the Dual Attention and Multi-scale Feature Fusion Segmentation model can improve the segmentation effect of MRI knee images in mild and medium knee osteoarthritis, thereby offering an important clinical value and improving the accuracy of the clinical diagnosis.

Multi-scale Attention GAN with Adaptive Feature Fusion for Unsupervised Animation Face Generation

Multi-scale Attention GAN with Adaptive Feature Fusion for Unsupervised Animation Face Generation

A non-uniform low-light image enhancement method with multi-scale attention transformer and luminance consistency loss

A non-uniform low-light image enhancement method with multi-scale attention transformer and luminance consistency loss

Multi-Scale Target Detection in Autonomous Driving Scenarios Based on YOLOv5-AFAM

Multi-scale object detection is critically important in complex driving environments within the field of autonomous driving. To enhance the detection accuracy of both small-scale and large-scale targets in complex autonomous driving environments, this paper proposes an improved YOLOv5-AFAM algorithm. Firstly, the Adaptive Fusion Attention Module (AFAM) and Down-sampling Module (DownC) are introduced to increase the detection precision of small targets. Secondly, the Efficient Multi-scale Attention Module (EMA) is incorporated, enabling the model to simultaneously recognize small-scale and large-scale targets. Finally, a Minimum Point Distance IoU-based Loss Function (MPDIou-LOSS) is introduced to improve the accuracy and efficiency of object detection. Experimental validation on the KITTI dataset shows that, compared to the baseline model, the improved algorithm increased precision by 2.4%, recall by 2.6%, mAP50 by 1.5%, and mAP50-90 by an impressive 4.8%.

Supersonic combustion field evolution prediction in scramjet engine using a deblurring multi-scale attention network

Traditional deep learning technologies often overlook crucial spatiotemporal information when reconstructing supersonic combustion flow fields. These models can only reconstruct the current flow field, resulting in poor visual quality due to the atomization effect. This research introduces experimental studies on scramjet combustion flows with variable injection pressures at the inlet of a scramjet Mach 2.5, creating image datasets of flame evolution processes over different predicted time spans (5, 10, and 15 ms). It develops a multi-scale attention algorithm with deblurring (DB-MSAN) capabilities to predict the flame image after a certain period using the wall pressure signal from the previous moment. The algorithm employs two multi-head attention channels at different scales to extract timing information, encouraging the model to focus on the anisotropy of the input data and to identify more significant features. The DB module utilizes the adapted atmospheric scattering model to jointly estimate the transmission map and atmospheric light, achieving an exceptional defogging effect with a minimal computational expense. The model’s overall performance is assessed regarding efficiency and accuracy, comparing DB-MSAN’s performance differences with those of Chen’s and the ResNet16 models across multi-span, large-span, various test sets, and sparse wall measurement points. Experimental results indicated that, compared to other methods, DB-MSAN achieves a maximum peak signal-to-noise index improvement of 76.10 % and an SSIM index improvement of 155.26 % when predicting average-quality images. The prediction accuracy remains stable, even with sparse pressure input. In addition, DB-MSAN strikes an optimal balance between the number of parameters and model accuracy; it represents only 16.97 % of the volume compared to Chen’s model, which is 585 MB.

Intelligent Detection of Muskmelon Ripeness in Greenhouse Environment Based on YOLO-RFEW

Accurate detection of muskmelon fruit ripeness is crucial to ensure fruit quality, optimize picking time, and enhance economic benefits. This study proposes an improved lightweight YOLO-RFEW model based on YOLOv8n, aiming to address the challenges of low efficiency in muskmelon fruit ripeness detection and the complexity of deploying a target detection model to a muskmelon picking robot. Firstly, the RFAConv replaces the Conv in the backbone part of YOLOv8n, allowing the network to focus more on regions with significant contributions in feature extraction. Secondly, the feature extraction and fusion capability are enhanced by improving the C2f module into a C2f-FE module based on FasterNet and an Efficient Multi-Scale attention (EMA) mechanism within the lightweight model. Finally, Weighted Intersection over Union (WIoU) is optimized as the loss function to improve target frame prediction capability and enhance target detection accuracy. The experimental results demonstrate that the YOLO-RFEW model achieves high accuracy, with precision, recall, F1 score, and mean Average Precision (mAP) values of 93.16%, 83.22%, 87.91%, and 90.82%, respectively. Moreover, it maintains a lightweight design and high efficiency with a model size of 4.75 MB and an inference time of 1.5 ms. Additionally, in the two types of maturity tests (M-u and M-r), APs of 87.70% and 93.94% are obtained, respectively, by the YOLO-RFEW model. Compared to YOLOv8n, significant improvements in detection accuracy have been achieved while reducing both model size and computational complexity using the proposed approach for muskmelon picking robots’ real-time detection requirements. Furthermore, when compared to lightweight models such as YOLOv3-Tiny, YOLOv4-Tiny, YOLOv5s, YOLOv7-Tiny, YOLOv8s, and YOLOv8n, the YOLO-RFEW model demonstrates superior performance with only 28.55%, 22.42%, 24.50%, 40.56%, 22.12%, and 79.83% of their respective model sizes, respectively, while achieving the highest F1 score and mAP values among these seven models. The feasibility and effectiveness of our improved scheme are verified through comparisons between thermograms generated by YOLOv8n and YOLO-RFEW as well as detection images. In summary, the YOLO-RFEW model not only improves the accuracy rate of muskmelon ripeness detection but also successfully realizes the lightweight and efficient performance, which has important theoretical support and application value in the field of muskmelon picking robot development.

Unsupervised image-to-image translation with multiscale attention generative adversarial network

Unsupervised image-to-image translation with multiscale attention generative adversarial network

MACCNet: Multiscale Attention and Cross- Convolutional Network for Infrared and Visible Image Fusion

MACCNet: Multiscale Attention and Cross- Convolutional Network for Infrared and Visible Image Fusion

Extraction of laser stripe centerlines from translucent optical components using a multi-scale attention deep neural network

The precise extraction of laser stripe centerlines is critical for line-laser 3D scanning systems. However, conventional methods relying on threshold segmentation and morphological operations face significant challenges when confronted with pervasive optical phenomena, including specular reflection, scattering, and bleeding, which are commonly observed in translucent optical components. These methods typically require complex preprocessing procedures and often yield poor precision in centerline extraction. In this paper, we introduce a novel learning-based approach, complemented by a meticulously curated dataset, explicitly designed to address these challenges. Our proposed method leverages a multi-scale attention U-Net-like architecture, initially tasked with the segmentation of laser stripes from the complex background environment. Subsequently, it employs the Steger algorithm for the precise extraction of laser stripe centerlines. The experimental results, obtained by comprehensively evaluating real-world captured images, clearly demonstrate the effectiveness of our deep neural network combined with the Steger algorithm. This combined approach exhibits exceptional accuracy even when challenged by the interferences from specular reflection, scattering, and bleeding artifacts. Specifically, our method achieves a mean intersection over union (mIoU) of 84.71% for the laser stripe detection task, accompanied by a mean square error (MSE) of 10.371 pixels. Also, the average execution time for the centerline extraction task is notably efficient at 0.125 s.

DMAF-NET: Deep Multi-Scale Attention Fusion Network for Hyperspectral Image Classification with Limited Samples

In recent years, deep learning methods have achieved remarkable success in hyperspectral image classification (HSIC), and the utilization of convolutional neural networks (CNNs) has proven to be highly effective. However, there are still several critical issues that need to be addressed in the HSIC task, such as the lack of labeled training samples, which constrains the classification accuracy and generalization ability of CNNs. To address this problem, a deep multi-scale attention fusion network (DMAF-NET) is proposed in this paper. This network is based on multi-scale features and fully exploits the deep features of samples from multiple levels and different perspectives with an aim to enhance HSIC results using limited samples. The innovation of this article is mainly reflected in three aspects: Firstly, a novel baseline network for multi-scale feature extraction is designed with a pyramid structure and densely connected 3D octave convolutional network enabling the extraction of deep-level information from features at different granularities. Secondly, a multi-scale spatial–spectral attention module and a pyramidal multi-scale channel attention module are designed, respectively. This allows modeling of the comprehensive dependencies of coordinates and directions, local and global, in four dimensions. Finally, a multi-attention fusion module is designed to effectively combine feature mappings extracted from multiple branches. Extensive experiments on four popular datasets demonstrate that the proposed method can achieve high classification accuracy even with fewer labeled samples.

Self-support matching networks with multiscale attention for few-shot semantic segmentation

Recent advancements in Few-shot segmentation (FSS) have displayed remarkable capabilities in predicting segmentation masks for unseen class images, using only a limited number of annotated images. However, existing methods have overlooked the influence of contextual information on segmentation and primarily rely on supporting prototypes, with limited research on query prototyping. Effectively utilizing multi-scale features and query information poses a challenging problem in this domain. To address these challenges, this paper proposes a novel approach called the multi-scale and attention-based self-support prototype few-shot semantic segmentation network (MASNet). First, a multi-scale feature enhancement module is designed to obtain features at different scales to enrich global context information. Then, simple and efficient channel attention is utilized to guide the query features related to the target class. Finally, the query prototype is matched with the query features using a self-supporting matching module. This strategy efficiently captures class-based features and addresses the issue of intra-class variance in few-shot segmentation. The experimental results on Pascal-5i, COCO-20i and Abdominal MRI datasets demonstrate that the proposed method achieves remarkable robustness and improved accuracy performance.

Cyto R-CNN and CytoNuke Dataset: Towards reliable whole-cell segmentation in bright-field histological images

Background and objective:Cell segmentation in bright-field histological slides is a crucial topic in medical image analysis. Having access to accurate segmentation allows researchers to examine the relationship between cellular morphology and clinical observations. Unfortunately, most segmentation methods known today are limited to nuclei and cannot segment the cytoplasm. Methods:We present a new network architecture Cyto R-CNN that is able to accurately segment whole cells (with both the nucleus and the cytoplasm) in bright-field images. We also present a new dataset CytoNuke, consisting of multiple thousand manual annotations of head and neck squamous cell carcinoma cells. Utilizing this dataset, we compared the performance of Cyto R-CNN to other popular cell segmentation algorithms, including QuPath’s built-in algorithm, StarDist, Cellpose and a multi-scale Attention Deeplabv3+. To evaluate segmentation performance, we calculated AP50, AP75 and measured 17 morphological and staining-related features for all detected cells. We compared these measurements to the gold standard of manual segmentation using the Kolmogorov–Smirnov test. Results:Cyto R-CNN achieved an AP50 of 58.65% and an AP75 of 11.56% in whole-cell segmentation, outperforming all other methods (QuPath 19.46/0.91%; StarDist 45.33/2.32%; Cellpose 31.85/5.61%, Deeplabv3+ 3.97/1.01%). Cell features derived from Cyto R-CNN showed the best agreement to the gold standard (D¯=0.15) outperforming QuPath (D¯=0.22), StarDist (D¯=0.25), Cellpose (D¯=0.23) and Deeplabv3+ (D¯=0.33). Conclusion:Our newly proposed Cyto R-CNN architecture outperforms current algorithms in whole-cell segmentation while providing more reliable cell measurements than any other model. This could improve digital pathology workflows, potentially leading to improved diagnosis. Moreover, our published dataset can be used to develop further models in the future.