Channel-wise Attention Research Articles

A Novel Defect Segmentation Model via Integrating Enhanced U-Net with Convolutional Block Attention Mechanisms

Abstract. Defect segmentation in industrial quality control is pivotal for maintaining high product surface standards, but it remains a challenge due to complex defect patterns and varying backgrounds. Traditional segmentation methods often struggle with sensitivity and specificity, leading to either missed defects or false positives. In order to address the above issues this paper presents an advanced defect segmentation model that integrates a Convolutional Block Attention Module (CBAM) with an enhanced U-Net architecture. Our method leverages the spatial and channel-wise attention mechanisms to focus more accurately on subtle defects and remove background interference, enhancing the discriminative power of the feature maps generated by the U-Net. Experimental results on SD-saliency-900 industrial datasets demonstrate significant improvements in segmentation accuracy and robustness compared to existing state-of-the-art methods, confirming the effectiveness of integrating attention mechanisms in deep learning models for complex defect segmentation tasks. Specifically, our model achieved a mean Intersection over Union (mIoU) of 0.87, which is an increase of 5% over the traditional U-Net model and a 4% improvement compared to the DeepLabv3+ model. These results indicate a substantial enhancement in defect detection performance, validating the superiority of our approach.

Feature Enhancement Based Oriented Object Detection in Remote Sensing Images

Since objects in remote sensing imagery often have arbitrary orientations and high densities, the features of small objects are inclined to be contaminated by the background and other instances. To address the issues, we propose a new oriented object detection framework where a series of feature enhancement schemes are implemented so as to improve robustness and accuracy of the detector. Firstly, we design a weighted bidirectional feature pyramid network, which can be used to fuse both high-level semantic features and low-level detail features for effectively handling with multi-scale objects. Accordingly, we apply the convolutional block attention module that exploits both spatial- and channel-wise attention in our detector, and study how to effectively integrate it into the framework for adaptive feature refinement. In the meanwhile, we present a semantic segmentation guided module to generate naive mask, which is used to multiple with pyramid features to filter out background noise and improve feature representation for small objects. The experimental results on two public datasets, i.e., UCAS-AOD and DOTA, validate the effective performance of the proposed method for oriented object detection in remote sensing images.

Selective-Hop Graph Neural Networks for semi-supervised node classification

Message-passing Graph Neural Networks (GNNs) are remarkably successful in graph representation learning. Most of them propagate and aggregate information within fixed-size receptive-fields (RFs) for all nodes. In this paper, our empirical studies discover that different-sized RFs are often desired for distinct graphs and target nodes. To this end, we propose a dynamic selection mechanism by allowing each node to adaptively adjust the size of its RFs according to the graph properties. In particular, we design an architectural unit called Selective-Hop (SH) block, in which information from neighborhoods within different hops are merged using dual attention mechanisms: (i) node-wise attention on each network branch to capture mutual information between the subgraphs and the entire graph; and (ii) channel-wise attention on different network branches to detect the significance of different hops at the node-level granularity. As such, the SH block determines neighbors flexibly without the constraints of hops. By stacking several SH blocks in GNN style, we instantiate two new GNN variants, SHGCN and SHSGC. Empirical results on eight benchmark datasets for semi-supervised node classification reveal that our new models achieve a considerable improvement over the state-of-the-art methods under low-homophily or heterophily and maintain competitive performance under homophily.

SNN using color-opponent and attention mechanisms for object recognition

The current spiking neural network (SNN) relies on spike-timing-dependent plasticity (STDP) primarily for shape learning in object recognition tasks, overlooking the equally critical aspect of color information. To address this gap, our study introduces an unsupervised variant of STDP that incorporates principles from color-opponency mechanisms (COM) and classical receptive fields (CRF) found in the biological visual system, facilitating the integration of color information during parameter updates within the SNN architecture. Our approach initially preprocesses images into two distinct feature maps: one for shape and another for color. Then, signals derived from COM and intensity concurrently drive the STDP process, thereby updating parameters associated with both color and shape feature maps. Furthermore, we propose a channel-wise attention mechanism to enhance differentiation among objects sharing similar shapes or colors. Specifically, this mechanism utilizes convolution to generate an output spike-wave, identifying a winner based on earliest spike timing and maximal potential. The winning kernel computes attention, which is then applied via convolution to each input image feature map, generating post-feature maps. A STDP-like normalization rule compares firing times between pre- and post-feature maps, dynamically adjusting channel weights to optimize object recognition during the training phase.We assessed the proposed algorithm using SNN with both single-layer and multi-layer architectures across three datasets. Experimental findings highlight its efficacy and superiority in complex object recognition tasks compared to state-of-the-art (SOTA) algorithms. Notably, our approach achieved a significant 20% performance improvement over the SOTA on the Caltech-101 dataset. Moreover, the algorithm is well-suited for hardware implementation and energy efficiency, leveraging a winner-selection mechanism based on the earliest spike time.

Feature Interaction-Based Face De-Morphing Factor Prediction for Restoring Accomplice's Facial Image.

Face morphing attacks disrupt the essential correlation between a face image and its identity information, posing a significant challenge to face recognition systems. Despite advancements in face morphing attack detection methods, these techniques cannot reconstruct the face images of accomplices. Existing deep learning-based face de-morphing techniques have mainly focused on identity disentanglement, overlooking the morphing factors inherent in the morphed images. This paper introduces a novel face de-morphing method to restore the identity information of accomplices by predicting the corresponding de-morphing factor. To obtain reasonable de-morphing factors, a channel-wise attention mechanism is employed to perform feature interaction, and the correlation between the morphed image and the real-time captured reference image is integrated to promote the prediction of the de-morphing factor. Furthermore, the identity information of the accomplice is restored by mapping the morphed and reference images into the StyleGAN latent space and performing inverse linear interpolation using the predicted de-morphing factor. Experimental results demonstrate the superiority of this method in restoring accomplice facial images, achieving improved restoration accuracy and image quality compared to existing techniques.

Enhancing radio signal classification under low SNR conditions using deep residual networks with channel attention mechanisms

This paper delves into the advancement of deep residual networks (ResNets) integrated with channel attention mechanisms for the classification of radio signals under conditions of low Signal-to-Noise Ratio (SNR). Utilizing an expansive dataset of radio signals, this paper introduces a novel architecture, MyResNet1, that combines residual learning with channel-wise attention, allowing the model to concentrate on essential features for precise classification. My, investigations exhibit notable improvements in classification accuracy, especially in challenging low SNR scenarios, highlighting the potential of attention-augmented deep residual networks in radio signal processing. Furthermore, this studyexplores various optimization strategies, including data augmentation and regularization techniques, to enhance the models performance and robustness. My findings contribute significantly to cognitive radio technologies and illuminate the potential of deep learning in sophisticated signal classification tasks, aligned with recent explorations in automatic modulation recognition (AMR) through deep learning and autoencoder-based methodologies for enhancing I/Q channel interactions.

Surface Defect Detection for Aerospace Aluminum Profiles with Attention Mechanism and Multi-Scale Features

A YOLOv5 aluminum profile defect detection algorithm that integrates attention and multi-scale features is proposed in this paper to address the issues of the low detection accuracy, high false detection rates, and high missed detection rates that are caused by the large-scale variation of surface defects, inconspicuous small defect characteristics, and a lack of concentrated feature information in defect areas. Firstly, an improved CBAM (Channel-Wise Attention Module) convolutional attention module is employed, which effectively focuses on the feature information of defect areas in the aluminum defect dataset with only a small amount of spatial dimension. Secondly, a bidirectional weighted feature fusion network is utilized, incorporating a multi-scale feature fusion network with skip connections to aggregate various high-resolution features, thus enriching the semantic expression of features. Then, new size feature maps that have not been fused are introduced into the detection layer network to improve the detection effect of small target defects. Experimental results indicate that an average detection accuracy (mAP) of 82.6% was achieved by the improved YOLOv5 algorithm on the aluminum surface defect dataset. An improvement of 6.2% over the previous version was observed. The current defect detection requirements of aluminum profile production sites are met by this enhanced algorithm.

Source-free cross-domain fault diagnosis of rotating machinery using the Siamese framework

Despite deep learning based intelligent diagnosis has become an essential means of perceiving the health status of rotating machinery, existing diagnostic models struggle to efficiently extract domain-invariant representations for cross-domain tasks in the absence of source domain data due to the limitations of imbalanced machine fault data, the various scenarios, as well as privacy protection in industrial applications. In order to solve these issues, we innovatively propose a source-free cross-domain fault diagnosis algorithm for rotating machinery using the Siamese framework (SCSF). Specifically, a deep residual shrinkage network with parallel spatial and channel-wise attention mechanisms (DRSN-SCW) is used for feature extraction to suppress background noise. In addition, pre-trained feature extraction networks and classifiers use only source domains based on the Siamese framework for end-to-end representation learning to address the problem of fault diagnosis of category-imbalanced data at the feature representation level. Finally, for cross-domain scenarios where the target domain does not have access to source domain data during training, we use a prototype-based pseudo-labeling strategy as well as impose consistency and diversity constraints on the classifier outputs to improve the diagnostic performance of unlabeled target domains. Extensive experiments conducted on the Spectra Quest mechanical fault dataset and the Case Western Reserve University rolling bearing dataset validate the effectiveness of the SCSF. The code is available at .

Robust brain tumor classification by fusion of deep learning and channel-wise attention mode approach

Diagnosing brain tumors is a complex and time-consuming process that relies heavily on radiologists’ expertise and interpretive skills. However, the advent of deep learning methodologies has revolutionized the field, offering more accurate and efficient assessments. Attention-based models have emerged as promising tools, focusing on salient features within complex medical imaging data. However, the precise impact of different attention mechanisms, such as channel-wise, spatial, or combined attention within the Channel-wise Attention Mode (CWAM), for brain tumor classification remains relatively unexplored. This study aims to address this gap by leveraging the power of ResNet101 coupled with CWAM (ResNet101-CWAM) for brain tumor classification. The results show that ResNet101-CWAM surpassed conventional deep learning classification methods like ConvNet, achieving exceptional performance metrics of 99.83% accuracy, 99.21% recall, 99.01% precision, 99.27% F1-score and 99.16% AUC on the same dataset. This enhanced capability holds significant implications for clinical decision-making, as accurate and efficient brain tumor classification is crucial for guiding treatment strategies and improving patient outcomes. Integrating ResNet101-CWAM into existing brain classification software platforms is a crucial step towards enhancing diagnostic accuracy and streamlining clinical workflows for physicians.

Combining dual attention mechanism and efficient feature aggregation for road and vehicle segmentation from UAV imagery

Unmanned aerial vehicles (UAVs) have gained significant popularity in recent years due to their ability to capture high-resolution aerial imagery for various applications, including traffic monitoring, urban planning, and disaster management. Accurate road and vehicle segmentation from UAV imagery plays a crucial role in these applications. In this paper, we propose a novel approach combining dual attention mechanisms and efficient multi-layer feature aggregation to enhance the performance of road and vehicle segmentation from UAV imagery. Our approach integrates a spatial attention mechanism and a channel-wise attention mechanism to enable the model to selectively focus on relevant features for segmentation tasks. In conjunction with these attention mechanisms, we introduce an efficient multi-layer feature aggregation method that synthesizes and integrates multi-scale features at different levels of the network, resulting in a more robust and informative feature representation. Our proposed method is evaluated on the UAVid semantic segmentation dataset, showcasing its exceptional performance in comparison to renowned approaches such as U-Net, DeepLabv3+, and SegNet. The experimental results affirm that our approach surpasses these state-of-the-art methods in terms of segmentation accuracy.

Unsupervised Monocular Depth Estimation With Channel and Spatial Attention.

Understanding 3-D scene geometry from videos is a fundamental topic in visual perception. In this article, we propose an unsupervised monocular depth and camera motion estimation framework using unlabeled monocular videos to overcome the limitation of acquiring per-pixel ground-truth depth at scale. The photometric loss couples the depth network and pose network together and is essential to the unsupervised method, which is based on warping nearby views to target using the estimated depth and pose. We introduce the channelwise attention mechanism to dig into the relationship between channels and introduce the spatialwise attention mechanism to utilize the inner-spatial relationship of features. Both of them applied in depth networks can better activate the feature information between different convolutional layers and extract more discriminative features. In addition, we apply the Sobel boundary to our edge-aware smoothness for more reasonable accuracy, and clearer boundaries and structures. All of these help to close the gap with fully supervised methods and show high-quality state-of-the-art results on the KITTI benchmark and great generalization performance on the Make3D dataset.

Quality Prediction Modeling for Industrial Processes Using Multiscale Attention-Based Convolutional Neural Network.

Soft sensors have been increasingly applied for quality prediction in complex industrial processes, which often have different scales of topology and highly coupled spatiotemporal features. However, the existing soft sensing models usually face difficulties in extracting the multiscale local spatiotemporal features in multicoupled complex process data and harnessing them to their full potential to improve the prediction performance. Therefore, a multiscale attention-based CNN (MSACNN) is proposed in this article to alleviate such problems. In MSACNN, convolutional kernels of different sizes are first designed in parallel in the convolutional layers, which can generate feature maps containing local spatiotemporal features at different scales. Meanwhile, a channel-wise attention mechanism is designed on the feature maps in parallel to get their attention weights, representing the significance of the local spatiotemporal feature at different scales. The superiority of the proposed MSACNN over the other state-of-the-art methods is validated through the performance evaluation in two real industrial processes.

BAF-Net: bidirectional attention-aware fluid pyramid feature integrated multimodal fusion network for diagnosis and prognosis

Objective. To go beyond the deficiencies of the three conventional multimodal fusion strategies (i.e. input-, feature- and output-level fusion), we propose a bidirectional attention-aware fluid pyramid feature integrated fusion network (BAF-Net) with cross-modal interactions for multimodal medical image diagnosis and prognosis. Approach. BAF-Net is composed of two identical branches to preserve the unimodal features and one bidirectional attention-aware distillation stream to progressively assimilate cross-modal complements and to learn supplementary features in both bottom-up and top-down processes. Fluid pyramid connections were adopted to integrate the hierarchical features at different levels of the network, and channel-wise attention modules were exploited to mitigate cross-modal cross-level incompatibility. Furthermore, depth-wise separable convolution was introduced to fuse the cross-modal cross-level features to alleviate the increase in parameters to a great extent. The generalization abilities of BAF-Net were evaluated in terms of two clinical tasks: (1) an in-house PET-CT dataset with 174 patients for differentiation between lung cancer and pulmonary tuberculosis. (2) A public multicenter PET-CT head and neck cancer dataset with 800 patients from nine centers for overall survival prediction. Main results. On the LC-PTB dataset, improved performance was found in BAF-Net (AUC = 0.7342) compared with input-level fusion model (AUC = 0.6825; p < 0.05), feature-level fusion model (AUC = 0.6968; p = 0.0547), output-level fusion model (AUC = 0.7011; p < 0.05). On the H&N cancer dataset, BAF-Net (C-index = 0.7241) outperformed the input-, feature-, and output-level fusion model, with 2.95%, 3.77%, and 1.52% increments of C-index (p = 0.3336, 0.0479 and 0.2911, respectively). The ablation experiments demonstrated the effectiveness of all the designed modules regarding all the evaluated metrics in both datasets. Significance. Extensive experiments on two datasets demonstrated better performance and robustness of BAF-Net than three conventional fusion strategies and PET or CT unimodal network in terms of diagnosis and prognosis.

Multiscale attention network via topology learning for cerebral vessel segmentation in angiography images

Cerebrovascular diseases are a widespread threat to human health. The accurate extraction of cerebral vessel structures is of paramount importance in the diagnosis and treatment of cerebrovascular diseases. However, the complexity of cerebral vessel structures and the low imaging contrast present significant challenges for vessel segmentation. Therefore, we propose a Multiscale Attention Network based on topological learning to extract vessel structures from angiographic images. This method employs a Multiscale Squeeze Attention (MSA) module for channel-wise attention learning, extracting multiscale attention feature maps from angiographic images. To maintain the topological connectivity of vessel segmentation, we introduced the clDice loss function to enforce skeleton connectivity of vessel segmentation. We conducted an experimental analysis of the proposed method using a publicly available cerebral vessel dataset. The results demonstrated that the proposed method achieved a sensitivity score of 0.8507 and a dice score of 0.8669 for cerebrovascular segmentation, enabling accurate and complete extraction of vascular structures. The proposed method was extended to coronary angiography images. The results show that the proposed method can accurately extract coronary structures, proving its broad applicability to other vascular segmentation tasks.

DICAM: Deep Inception and Channel-wise Attention Modules for underwater image enhancement

In underwater environments, imaging devices suffer from water turbidity, attenuation of lights, scattering, and particles, leading to low quality, poor contrast, and biased color images. This has led to great challenges for underwater condition monitoring and inspection using conventional vision techniques. In recent years, underwater image enhancement has attracted increasing attention due to its critical role in improving the performance of current computer vision tasks in underwater object detection and segmentation. As existing methods, built mainly from natural scenes, have performance limitations in improving the color richness and distributions we propose a novel deep learning-based approach namely Deep Inception and Channel-wise Attention Modules (DICAM) to enhance the quality, contrast, and color cast of the hazy underwater images. The proposed DICAM model enhances the quality of underwater images, considering both the proportional degradations and non-uniform color cast. Extensive experiments on two publicly available underwater image enhancement datasets have verified the superiority of our proposed model compared with several state-of-the-art conventional and deep learning-based methods in terms of full-reference and reference-free image quality assessment metrics. The source code of our DICAM model is available at https://github.com/hfarhaditolie/DICAM.

SwiftPillars: High-Efficiency Pillar Encoder for Lidar-Based 3D Detection

Lidar-based 3D Detection is one of the significant components of Autonomous Driving. However, current methods over-focus on improving the performance of 3D Lidar perception, which causes the architecture of networks becoming complicated and hard to deploy. Thus, the methods are difficult to apply in Autonomous Driving for real-time processing. In this paper, we propose a high-efficiency network, SwiftPillars, which includes Swift Pillar Encoder (SPE) and Multi-scale Aggregation Decoder (MAD). The SPE is constructed by a concise Dual-attention Module with lightweight operators. The Dual-attention Module utilizes feature pooling, matrix multiplication, etc. to speed up point-wise and channel-wise attention extraction and fusion. The MAD interconnects multiple scale features extracted by SPE with minimal computational cost to leverage performance. In our experiments, our proposal accomplishes 61.3% NDS and 53.2% mAP in nuScenes dataset. In addition, we evaluate inference time on several platforms (P4, T4, A2, MLU370, RTX3080), where SwiftPillars achieves up to 13.3ms (75FPS) on NVIDIA Tesla T4. Compared with PointPillars, SwiftPillars is on average 26.58% faster in inference speed with equivalent GPUs and a higher mAP of approximately 3.2% in the nuScenes dataset.

Urban expansion simulation with an explainable ensemble deep learning framework

Urban expansion simulation is of significant importance to land management and policymaking. Advances in deep learning facilitate capturing and anticipating urban land dynamics with state-of-the-art accuracy properties. In this context, a novel deep learning-based ensemble framework was proposed for urban expansion simulation at an intra-urban granular level. The ensemble framework comprises i) multiple deep learning models as encoders, using transformers for encoding multi-temporal spatial features and convolutional layers for processing single-temporal spatial features, ii) a tailored channel-wise attention module to address the challenge of limited interpretability in deep learning methods. The channel attention module enables the examination of the rationality of feature importance, thereby establishing confidence in the simulated results. The proposed method accurately anticipated urban expansion in Shenzhen, China, and it outperformed all the baseline methods in terms of both spatial accuracy and temporal consistency.

Automatic Detection of COVID-19 from Chest X-Ray Images Using EfficientNet-B7 CNN Model with Channel-wise Attention

Automatic Detection of COVID-19 from Chest X-Ray Images Using EfficientNet-B7 CNN Model with Channel-wise Attention

Channel-wise attention enhanced and structural similarity constrained cycleGAN for effective synthetic CT generation from head and neck MRI images

BackgroundMagnetic resonance imaging (MRI) plays an increasingly important role in radiotherapy, enhancing the accuracy of target and organs at risk delineation, but the absence of electron density information limits its further clinical application. Therefore, the aim of this study is to develop and evaluate a novel unsupervised network (cycleSimulationGAN) for unpaired MR-to-CT synthesis.MethodsThe proposed cycleSimulationGAN in this work integrates contour consistency loss function and channel-wise attention mechanism to synthesize high-quality CT-like images. Specially, the proposed cycleSimulationGAN constrains the structural similarity between the synthetic and input images for better structural retention characteristics. Additionally, we propose to equip a novel channel-wise attention mechanism based on the traditional generator of GAN to enhance the feature representation capability of deep network and extract more effective features. The mean absolute error (MAE) of Hounsfield Units (HU), peak signal-to-noise ratio (PSNR), root-mean-square error (RMSE) and structural similarity index (SSIM) were calculated between synthetic CT (sCT) and ground truth (GT) CT images to quantify the overall sCT performance.ResultsOne hundred and sixty nasopharyngeal carcinoma (NPC) patients who underwent volumetric-modulated arc radiotherapy (VMAT) were enrolled in this study. The generated sCT of our method were more consistent with the GT compared with other methods in terms of visual inspection. The average MAE, RMSE, PSNR, and SSIM calculated over twenty patients were 61.88 ± 1.42, 116.85 ± 3.42, 36.23 ± 0.52 and 0.985 ± 0.002 for the proposed method. The four image quality assessment metrics were significantly improved by our approach compared to conventional cycleGAN, the proposed cycleSimulationGAN produces significantly better synthetic results except for SSIM in bone.ConclusionsWe developed a novel cycleSimulationGAN model that can effectively create sCT images, making them comparable to GT images, which could potentially benefit the MRI-based treatment planning.

A robust one-stage detector for SAR ship detection with sequential three-way decisions and multi-granularity

Synthetic Aperture Radar (SAR) images are widely used in ship detection because of their all-weather and all-day imaging characteristics. However, there are two challenges for SAR ship detection. One is coherent speckle noise, causing ship confusion with similar objects and raising false alarms. The other is multi-scale ship detection, particularly in small ships, which suffers from insufficient accuracy. To address these challenges, this paper proposes a robust one-stage detector, S3MDet, for SAR ship detection with sequential three-way decisions (S3WDs) and multi-granularity. First, to effectively eliminate the interference of coherent speckle noise, a noise classification and denoising module (NCDM) S3WD-based is designed. This module can accurately classify the noise level of the image and only perform denoising on the images identified as noisy, avoiding unnecessary operations on noise-free images. Then, to solve the problem of multi-scale ship detection, a multi-granularity group attention module (MGAM) is designed to obtain a richer representation of multi-granularity features. This module adopts a multi-granularity group convolution structure and channel-wise attention weights to efficiently extract ship features of different scales from SAR images. Extensive experiments on four SAR ship datasets, including SAR-Ship-Dataset, HRSID, SSDD, and LS-SSDD-v1.0, validate the robustness of S3MDet, demonstrating that it achieves state-of-the-art performance.