Multi-scale Channel Attention Research Articles

Iterative optimization annotation pipeline and ALSS-YOLO-Seg for efficient banana plantation segmentation in UAV imagery.

Precise segmentation of unmanned aerial vehicle (UAV)-captured images plays a vital role in tasks such as crop yield estimation and plant health assessment in banana plantations. By identifying and classifying planted areas, crop areas can be calculated, which is indispensable for accurate yield predictions. However, segmenting banana plantation scenes requires a substantial amount of annotated data, and manual labeling of these images is both timeconsuming and labor-intensive, limiting the development of large-scale datasets. Furthermore, challenges such as changing target sizes, complex ground backgrounds, limited computational resources, and correct identification of crop categories make segmentation even more difficult. To address these issues, we propose a comprehensive solution. First, we designed an iterative optimization annotation pipeline leveraging SAM2's zero-shot capabilities to generate high-quality segmentation annotations, thereby reducing the cost and time associated with data annotation significantly. Second, we developed ALSS-YOLO-Seg, an efficient lightweight segmentation model optimized for UAV imagery. The model's backbone includes an Adaptive Lightweight Channel Splitting and Shuffling (ALSS) module to improve information exchange between channels and optimize feature extraction, aiding accurate crop identification. Additionally, a Multi-Scale Channel Attention (MSCA) module combines multi-scale feature extraction with channel attention to tackle challenges of varying target sizes and complex ground backgrounds. We evaluated the zero-shot segmentation performance of SAM2 on the ADE20K and Javeri datasets. Our iterative optimization annotation pipeline demonstrated a significant reduction in manual annotation effort while achieving high-quality segmentation labeling. Extensive experiments on our custom Banana Plantation segmentation dataset show that ALSS-YOLO-Seg achieves state-of-the-art performance. Our code is openly available at https://github.com/helloworlder8/computer_vision.

Integrating spatial and channel attention mechanisms with domain knowledge in convolutional neural networks for friction coefficient prediction

AbstractThe pavement skid resistance is crucial for ensuring driving safety. However, the reproducibility and comparability of field measurements are constrained by various influencing factors. One solution to these constraints is utilizing laser‐based 3D pavement data, which are notably stable and can be employed to estimate pavement skid resistance indirectly. However, the integration of tire–road friction mechanisms and deep neural networks has not been fully studied. This study employed spatial‐channel attention mechanisms to integrate frictional domain knowledge and convolutional neural networks (CNNs) that predict the friction coefficient as the output. The models’ inputs include 3D texture data, corresponding finite element (FE) simulation outcomes, and 2D wavelet decomposition outcomes. An additional spatial attention (ASA) mechanism guided the CNNs to focus on the tire–road contact region, using tire–road contact stress from FE simulation as domain knowledge. Multi‐scale channel attention (MSCA) mechanisms enabled the CNNs to learn the channel weights of 2D‐wavelet‐based multi‐scale inputs, thereby assessing the contribution of different texture scales to tire–road friction. A multi‐attention and feature fusion mechanism was designed, and the performances of various combinations were compared. The results showed that the fusion of ASA and MSCA achieved the best performance, with a regression R2 of 0.8470, which was a 20.25% improvement over the baseline model.

A Multi-Scale and Multi-Stage Human Pose Recognition Method Based on Convolutional Neural Networks for Non-Wearable Ergonomic Evaluation

In the context of industrial robot maintenance and assembly, workers often suffer from work-related musculoskeletal disorders (WRMSDs). This paper proposes a multi-scale, multi-stage pose recognition method (MMARM-CNN) based on convolutional neural networks to provide ergonomic intervention. The method leverages computer vision technology to enable non-contact data acquisition, reducing the interference of physiological and psychological factors on assessment results. Built upon the baseline yolov8-pose framework, the method addresses complex maintenance environments, which are prone to occlusion, by introducing the Lightweight Shared Convolutional Detection Head-pose (LSCD-pose) module, Multi-Scale Channel Attention (MSCA) mechanism, and Efficient Multi-Scale Patch Convolution (EMSPC) module, enhancing the model’s feature extraction capabilities. The MMARM-CNN model was validated using the MS COCO 2017 dataset and robot assembly data collected under laboratory conditions. The experimental results show that the MMARM-CNN achieved an accuracy improvement, reaching 0.875 in the mAP@0.5 evaluation. Overall, this method demonstrates significant potential in advancing the automation and intelligence of ergonomic interventions.

INA-Net: An integrated noise-adaptive attention neural network for enhanced medical image segmentation

Skin cancer, a growing health concern, poses a significant threat. Automated segmentation of skin lesions is crucial for precise clinical diagnosis and treatment. However, challenges such as hair interference, vascular occlusion, and ambiguous lesion boundaries in dermoscopic images complicate accurate segmentation. Recent research has focused on capturing multiscale-enhanced global information across spatial and channel dimensions to improve the segmentation of ambiguous boundaries. Despite these advancements, performance remains suboptimal in the presence of complex disturbances. To address this, we propose INA-Net, an Integrated Noise-Adaptive Attention Neural Network for enhanced medical image segmentation. INA-Net proposes a lightweight noise-range attention module that integrates local and simulated noise features with global information to improve robustness against complex mixed noises. Additionally, our proposed Edge-aware Spatial Attention (ESA) and Multi-scale Channel Attention (MCA) enhance feature information and boundary perception. The proposed Dynamic Noise Encoding module and Fourier Wavelet Analysis (FW-Parser) further improve robustness by handling high-frequency noise components. INA-Net effectively manages hair interference, vascular occlusion, and ambiguous lesion boundaries, thereby providing critical support for accurate lesion identification. Evaluations on the ISIC2016, ISIC2018, and PH2 datasets demonstrate outstanding performance, with IOU scores of 87.53%, 84.74%, and 85.30%, respectively. These results surpass those of other models with comparable complexity and size.

(HTBNet)Arbitrary Shape Scene Text Detection with Binarization of Hyperbolic Tangent and Cross-Entropy.

The existing segmentation-based scene text detection methods mostly need complicated post-processing, and the post-processing operation is separated from the training process, which greatly reduces the detection performance. The previous method, DBNet, successfully simplified post-processing and integrated post-processing into a segmentation network. However, the training process of the model took a long time for 1200 epochs and the sensitivity to texts of various scales was lacking, leading to some text instances being missed. Considering the above two problems, we design the text detection Network with Binarization of Hyperbolic Tangent (HTBNet). First of all, we propose the Binarization of Hyperbolic Tangent (HTB), optimized along with which the segmentation network can expedite the initial convergent speed by reducing the number of epochs from 1200 to 600. Because features of different channels in the same scale feature map focus on the information of different regions in the image, to better represent the important features of all objects in the image, we devise the Multi-Scale Channel Attention (MSCA). Meanwhile, considering that multi-scale objects in the image cannot be simultaneously detected, we propose a novel module named Fused Module with Channel and Spatial (FMCS), which can fuse the multi-scale feature maps from channel and spatial dimensions. Finally, we adopt cross-entropy as the loss function, which measures the difference between predicted values and ground truths. The experimental results show that HTBNet, compared with lightweight models, has achieved competitive performance and speed on Total-Text (F-measure:86.0%, FPS:30) and MSRA-TD500 (F-measure:87.5%, FPS:30).

Environmentally adaptive fast object detection in UAV images

Detecting objects in aerial images poses a challenging task due to the presence of numerous small objects and complex environmental information. To address these problems, we propose an efficient detector specifically designed for aerial images, named EAF-YOLOv8, based on YOLOv8-S. In this paper, we introduce a novel backbone network called EAFNet, specifically designed for small object detection. EAFNet consists of the Rapidly Merging Receptive Fields Aggregation Module (RMRFAM) and Multi-Scale Channel Attention (MSCA). The RMRFAM utilizes dilated convolution (DConv) and partial convolution (PConv) to acquire richer receptive fields, capturing more extensive contextual information at higher levels while reducing redundancy in channel information, thereby accelerating inference speed. Furthermore, inspired by denoising tasks, we focus on the feature information surrounding the target background and propose MSCA. MSCA integrates channel attention with an embedded self-attention feature pyramid, extending the feature learning scope to the surrounding environment of the target, beyond the target itself. This approach utilizes enhanced background features to elicit a higher response for small targets, reducing false positives. Experimental results demonstrate that in UAVDT and VisDrone2019, the proposed EAF-YOLOv8 achieves mAP50 scores of 34.3% and 49.7%, respectively. Additionally, EAF-YOLOv8 exhibits high real-time inference speeds of 77.60 FPS and 55.56 FPS, showcasing competitive detection performance.

MEAG-YOLO: A Novel Approach for the Accurate Detection of Personal Protective Equipment in Substations

Timely and accurately detecting personal protective equipment (PPE) usage among workers is essential for substation safety management. However, traditional algorithms encounter difficulties in substations due to issues such as varying target scales, intricate backgrounds, and many model parameters. Therefore, this paper proposes MEAG-YOLO, an enhanced PPE detection model for substations built upon YOLOv8n. First, the model incorporates the Multi-Scale Channel Attention (MSCA) module to improve feature extraction. Second, it newly designs the EC2f structure with one-dimensional convolution to enhance feature fusion efficiency. Additionally, the study optimizes the Path Aggregation Network (PANet) structure to improve feature learning and the fusion of multi-scale targets. Finally, the GhostConv module is integrated to optimize convolution operations and reduce computational complexity. The experimental results show that MEAG-YOLO achieves a 2.4% increase in precision compared to YOLOv8n, with a 7.3% reduction in FLOPs. These findings suggest that MEAG-YOLO is effective in identifying PPE in complex substation scenarios, contributing to the development of smart grid systems.

SRMA: a dual-branch parallel multi-scale attention network for remote sensing images sea-land segmentation

ABSTRACT The use of deep learning-based high resolution remote sensing image sea and land segmentation method has become prevalent in various fields such as environmental monitoring, resource assessment, and urban planning. However, the segmentation process faces challenges due to the complex coastline and the similarity in spectral colour difference between the sea and land in certain areas. The remote sensing images obtained from satellites often exhibit unclear boundaries between the sea and land, as well as intricate coastlines. Consequently, the segmentation of sea and land becomes a difficult task. The existing sea-land segmentation methods have limitations in accurately segmenting long and narrow waters, as well as areas with low differentiation between sea and land. In this paper, we propose a new sea-land segmentation method called SRMA (Swin-Res-Multi-Attention). SRMA performs parallel feature extraction using a dual-branch parallel: Swin-Transformer branch and ResNet branch. The combination of Swin-Transformer’s self-attention and ResNet’s residual design helps reduce the loss of feature information, thereby avoiding incorrect segmentation in regions with low distinctness. To improve overall stability, we introduce the MCA (Multiscale channel attention module) module, which combines feature information from different scales. The multi-scale pyramid design of MCA module ensures stability and improves segmentation accuracy in narrow waters. Experimental results on a public dataset demonstrate that our model outperforms existing methods. Additionally, we evaluate the model’s performance in complex scenarios by creating a new dataset that includes a complex urban water body and cloud interference. The results showcase the robustness and superiority of our model.

A multiscale feature fusion enhanced CNN with the multiscale channel attention mechanism for efficient landslide detection (MS2LandsNet) using medium-resolution remote sensing data

ABSTRACT Deep learning (DL) models have been widely used for remote sensing-based landslide mapping due to their impressive capabilities for automatic information extraction. However, the large volumes of parameters and calculations have compromised the efficiency of DL models in extracting landslides from a large set of RS images. Lightweight convolutional neural networks (CNNs) exhibit promising feature representation abilities with fewer parameters. This study aims to introduce a new lightweight CNN called MS2LandsNet, designed to detect landslides with both high efficiency and accuracy. The MS2LandsNet consists of three down-sampling stages embedded with multi-scale feature fusion (MFF), aiming to decrease parameters while aggregating contextual features. Additionally, we incorporate multi-scale channel attention (MSCA) into MFF to improve performance. According to experimental results on three landslip datasets, MS2LandsNet obtains the highest F1 score of 85.90% and the highest IoU of 75.28%. Notably, MS2LandsNet accomplishes the resuts with the fewest parameters and the fastest inference speed, outperforming seven classical semantic segmentation models and three lightweight CNNs. The proposed lightweight model holds potential for application on a cloud computing platform for larger-scale landslide mapping tasks in future work.

GAN for Semantic Image Synthesis With Laplacian Pyramid and Multi-Scale Channel Attention

GAN for Semantic Image Synthesis With Laplacian Pyramid and Multi-Scale Channel Attention

H2Former: An Efficient Hierarchical Hybrid Transformer for Medical Image Segmentation.

Accurate medical image segmentation is of great significance for computer aided diagnosis. Although methods based on convolutional neural networks (CNNs) have achieved good results, it is weak to model the long-range dependencies, which is very important for segmentation task to build global context dependencies. The Transformers can establish long-range dependencies among pixels by self-attention, providing a supplement to the local convolution. In addition, multi-scale feature fusion and feature selection are crucial for medical image segmentation tasks, which is ignored by Transformers. However, it is challenging to directly apply self-attention to CNNs due to the quadratic computational complexity for high-resolution feature maps. Therefore, to integrate the merits of CNNs, multi-scale channel attention and Transformers, we propose an efficient hierarchical hybrid vision Transformer (H2Former) for medical image segmentation. With these merits, the model can be data-efficient for limited medical data regime. The experimental results show that our approach exceeds previous Transformer, CNNs and hybrid methods on three 2D and two 3D medical image segmentation tasks. Moreover, it keeps computational efficiency in model parameters, FLOPs and inference time. For example, H2Former outperforms TransUNet by 2.29% in IoU score on KVASIR-SEG dataset with 30.77% parameters and 59.23% FLOPs.

Multi-Supervised Feature Fusion Attention Network for Clouds and Shadows Detection

Cloud and cloud shadow detection are essential in remote sensing imagery applications. Few semantic segmentation models were designed specifically for clouds and their shadows. Based on the visual and distribution characteristics of clouds and their shadows in remote sensing imagery, this paper provides a multi-supervised feature fusion attention network. We design a multi-scale feature fusion block (FFB) for the problems caused by the complex distribution and irregular boundaries of clouds and shadows. The block consists of a fusion convolution block (FCB), a channel attention block (CAB), and a spatial attention block (SPA). By multi-scale convolution, FCB reduces excessive semantic differences between shallow and deep feature maps. CAB focuses on global and local features through multi-scale channel attention. Meanwhile, it fuses deep and shallow feature maps with non-linear weighting to optimize fusion performance. SPA focuses on task-relevant areas through spatial attention. With the three blocks above, FCB alleviates the difficulties of fusing multi-scale features. Additionally, it makes the network resistant to background interference while optimizing boundary detection. Our proposed model designs a class feature attention block (CFAB) to increase the robustness of cloud detection. The network achieves good performance on the self-made cloud and shadow dataset. This dataset is taken from Google Earth and contains remote sensing imagery from several satellites. The proposed model achieved a mean intersection over union (MIoU) of 94.10% on our dataset, which is 0.44% higher than the other models. Moreover, it shows high generalization capability due to its superior prediction results on HRC_WHU and SPARCS datasets.

HarDNet-CPS: Colorectal polyp segmentation based on Harmonic Densely United Network

Colorectal cancer (CRC) is one of the top three cancers in the world. The disease is initially caused by the deterioration of polyps that are regarded as the predecessor of CRC. Many people die from the disease yearly, so clinical specialists pay high attention to the polyp. It is significant to improve the accurate recognition of polyp areas for clinicians to find the lesion area. Although a set of existing methods have been used in segmentation of polyps, the segmentation results still can be improved further. This paper proposed a new model called Harmonic Densely United Network for Colorectal Polyp Segmentation (HarDNet-CPS). Harmonic Densely Connected Network-68 is used as the backbone, in which branches with multi-scale channel attention (MS-CAM) are cleverly embedded to extract features from channel attention; the pyramid feature fusion technique is used to adaptively spatial feature fusion (ASFF) in the decoding area, and then output is obtained through aggregation module. The mean Dice in Kvasir and CVC-ClinicDB data sets both have broken through 0.9 for the first time. HarDNet-CPS has advantages such as good robustness and high accuracy, revealing the effectiveness of automated polyp detection. It will bring great help to clinical diagnosis and treatment.

A cross-scale mixed attention network for smoke segmentation

Deep neural networks have achieved good progresses in smoke segmentation, but it is a challenging task to accurately segment smoke images due to smoke semi-transparency and large variance. To improve performance, we propose a Cross-scale Mixed-attention Network (CMNet) by designing multi-scale and mixed attention modules. We first concatenate the results of average and maximum pooling along each axis to learn powerful attention coefficients, which are used to weight the original input for producing a directional attention map along each axis. Then we use point-wise additions for combining three directional attention maps to propose a module of Fused 3D Attention (F3A). In addition, we adopt atrous convolutions to generate multi-scale feature maps, then point-wisely add the results of average and maximum pooling on each scale feature map, and design a bottleneck structure to produce an effective attention map for each scale and reduce learnable parameters simultaneously. All attention feature maps with different scales are concatenated to obtain Multi-scale Channel Attention (MCA). Finally, we cross-wisely stack the modules of F3A and MCA on both shallow and deep feature maps to propose Mixed Cross Enhancement (MCE) for fully fusing information across scales. Experiments show that our method surpasses most existing methods.

Computationally Lightweight Hyperspectral Image Classification Using a Multiscale Depthwise Convolutional Network With Channel Attention

Convolutional networks have been widely used for the classification of hyperspectral images; however, such networks are notorious for their large number of trainable parameters and high computational complexity. Additionally, traditional convolution-based methods are typically implemented as a simple cascade of a number of convolutions using a single-scale convolution kernel. In contrast, a lightweight multiscale convolutional network is proposed, capitalizing on feature extraction at multiple scales in parallel branches followed by feature fusion. In this approach, 2D depthwise convolution is used instead of conventional convolution in order to reduce network complexity without sacrificing classification accuracy. Furthermore, multiscale channel attention is also employed to selectively exploit discriminative capability across various channels. To do so, multiple 1D convolutions with varying kernel sizes provide channel attention at multiple scales, again with the goal of minimizing network complexity. Experimental results reveal that the proposed network not only outperforms other competing lightweight classifiers in terms of classification accuracy but also exhibits a lower number of parameters as well as significantly less computational cost.

Skin disease migration segmentation network based on multi-scale channel attention

ABSTRACT Skin is the first line of defence of the human body. Accurate skin lesions image segmentation is essential for skin disease diagnosis and treatment planning. In this work, we create a new multi-scale channel attention module – MS-CA, which can display more accurate and relevant feature channels on multiple scales. And a network segmentation model for multi-scale channel attention(MS-CA) is proposed. The network model embeds the MS-CA module into two different types of benchmark networks, and modifies the two types of benchmark network models to obtain an image segmentation model suitable for skin diseases. Specifically, by extending the encoder and decoder structures of the LCA-Net network respectively, the MS-CA segmentation model of LCA-Net is proposed, and experiments are carried out on the ISIC2017 and ISIC2018 datasets, and the Dice and accuracy evaluation indicators are improved from 85.33% to 86.71 %, 96.07% increased to 97.23% and the MS-CA segmentation model of TransUNet was proposed. The Jaccard and Dice coefficients were increased from 66.21% to 68.80%, respectively. 79.67% increased to 81.52%. Compared with the existing methods, the MS-CA attention network segmentation model achieves better segmentation results, and it can be used as a new skin disease segmentation network as a computer-aided diagnosis tool.