Spatial-channel Attention Module Research Articles

LERFNet: an enlarged effective receptive field backbone network for enhancing visual drone detection

AbstractRecently, the world has witnessed a great increase in drone applications and missions. Drones must be detected quickly, effectively, and precisely when they are being handled illegally. Vision-based anti-drone systems provide an efficient performance compared to radar- and acoustic-based systems. The effectiveness of drone detection is affected by a number of issues, including the drone’s small size, conflicts with other objects, and noisy backgrounds. This paper employs enlarging the effective receptive field (ERF) of feature maps generated from the YOLOv6 backbone. First, RepLKNet is used as the backbone of YOLOv6, which deploys large kernels with depth-wise convolution. Then, to get beyond RepLKNet’s large inference time, a novel LERFNet is implemented. LERFNet uses dilated convolution in addition to large kernels to enlarge the ERF and overcome each other’s problems. The linear spatial-channel attention module (LAM) is used to give more attention to the most informative pixels and high feature channels. LERFNet produces output feature maps with a large ERF and high shape bias to enhance the detection of various drone sizes in complex scenes. The RepLKNet and LERFNet backbones for Tiny-YOLOv6, Tiny-YOLOv6, YOLOv5s, and Tiny-YOLOv7 are compared. In comparison to the aforementioned techniques, the suggested model’s results show a greater balance between accuracy and speed. LERFNet increases the MAP by $$2.8\%$$ 2.8 % , while significantly reducing the GFLOPs and parameter numbers when compared to the original backbone of YOLOv6.

CMAN: Cascaded Multi-scale Spatial Channel Attention-guided Network for large 3D deformable registration of liver CT images

Deformable image registration is an essential component of medical image analysis and plays an irreplaceable role in clinical practice. In recent years, deep learning-based registration methods have demonstrated significant improvements in convenience, robustness and execution time compared to traditional algorithms. However, registering images with large displacements, such as those of the liver organ, remains underexplored and challenging. In this study, we present a novel convolutional neural network (CNN)-based unsupervised learning registration method, Cascaded Multi-scale Spatial-Channel Attention-guided Network (CMAN), which addresses the challenge of large deformation fields using a double coarse-to-fine registration approach. The main contributions of CMAN include: (i) local coarse-to-fine registration in the base network, which generates the displacement field for each resolution and progressively propagates these local deformations as auxiliary information for the final deformation field; (ii) global coarse-to-fine registration, which stacks multiple base networks for sequential warping, thereby incorporating richer multi-layer contextual details into the final deformation field; (iii) integration of the spatial-channel attention module in the decoder stage, which better highlights important features and improves the quality of feature maps. The proposed network was trained using two public datasets and evaluated on another public dataset as well as a private dataset across several experimental scenarios. We compared CMAN with four state-of-the-art CNN-based registration methods and two well-known traditional algorithms. The results show that the proposed double coarse-to-fine registration strategy outperforms other methods in most registration evaluation metrics. In conclusion, CMAN can effectively handle the large-deformation registration problem and show potential for application in clinical practice. The source code is made publicly available at https://github.com/LocPham263/CMAN.git.

Hybrid Spatial-Channel Attention Mechanism for Cross-Age Face Recognition

Face recognition techniques have been widely employed in real-world biomimetics applications. However, traditional approaches have limitations in recognizing faces correctly with large age differences because of significant changes over age in the same person, leading to unsatisfactory recognition performance. To address this, previous studies propose to decompose and identify age and identity features independently in facial images across diverse age groups when optimizing the discriminative model so as to improve the age-invariant face recognition accuracy. Nevertheless, the interrelationships between these features make it difficult for the decomposition to disentangle them properly, thus compromising the recognition accuracy due to the interactive impacts on both features. To this end, this paper proposes a novel deep framework that incorporates a novel Hybrid Spatial-Channel Attention Module to facilitate the cross-age face recognition task. Particularly, the proposed module enables better decomposition of the facial features in both spatial and channel dimensions with attention mechanisms simultaneously while mitigating the impact of age variation on the recognition performance. Beyond this, diverse pooling strategies are also combined when applying those spatial and channel attention mechanisms, which allows the module to generate discriminative face representations while preserving complete information within the original features, further yielding sounder recognition accuracy. The proposed model is extensively validated through experiments on public face datasets such as CACD-VS, AgeDB-30, and FGNET, where the results show significant performance improvements compared to competitive baselines.

Unified Convolutional Sparse Transformer for Disease Diagnosis, Monitoring, Drug Development, and Therapeutic Effect Prediction from EEG Raw Data.

Electroencephalogram (EEG) analysis plays an indispensable role across contemporary medical applications, which encompasses diagnosis, monitoring, drug discovery, and therapeutic assessment. This work puts forth an end-to-end deep learning framework that is uniquely tailored for versatile EEG analysis tasks by directly operating on raw waveform inputs. It aims to address the challenges of manual feature engineering and the neglect of spatial interrelationships in existing methodologies. Specifically, a spatial channel attention module is introduced to emphasize the critical inter-channel dependencies in EEG signals through channel statistics aggregation and multi-layer perceptron operations. Furthermore, a sparse transformer encoder is used to leverage selective sparse attention in order to efficiently process long EEG sequences while reducing computational complexity. Distilling convolutional layers further concatenates the temporal features and retains only the salient patterns. As it was rigorously evaluated on key EEG datasets, our model consistently accomplished a superior performance over the current approaches in detection and classification assignments. By accounting for both spatial and temporal relationships in an end-to-end paradigm, this work facilitates a versatile, automated EEG understanding across diseases, subjects, and objectives through a singular yet customizable architecture. Extensive empirical validation and further architectural refinement may promote broader clinical adoption prospects.

Siamese Visual Tracking with Spatial-Channel Attention and Ranking Head Network

Trackers based on the Siamese network have received much attention in recent years, owing to its remarkable performance, and the task of object tracking is to predict the location of the target in current frame. However, during the tracking process, distractors with similar appearances affect the judgment of the tracker and lead to tracking failure. In order to solve this problem, we propose a Siamese visual tracker with spatial-channel attention and a ranking head network. Firstly, we propose a Spatial Channel Attention Module, which fuses the features of the template and the search region by capturing both the spatial and the channel information simultaneously, allowing the tracker to recognize the target to be tracked from the background. Secondly, we design a ranking head network. By introducing joint ranking loss terms including classification ranking loss and confidence&IoU ranking loss, classification and regression branches are linked to refine the tracking results. Through the mutual guidance between the classification confidence score and IoU, a better positioning regression box is selected to improve the performance of the tracker. To better demonstrate that our proposed method is effective, we test the proposed tracker on the OTB100, VOT2016, VOT2018, UAV123, and GOT-10k testing datasets. On OTB100, the precision and success rate of our tracker are 0.925 and 0.700, respectively. Considering accuracy and speed, our method, overall, achieves state-of-the-art performance.

MFMANet: Multi-feature Multi-attention Network for efficient subtype classification on non-small cell lung cancer CT images

Lung cancer is one of the most prevalent diseases worldwide, being the most common type of cancer. Non-small cell lung cancer (NSCLC) with a five-year survival rate of less than 20% and widely varying treatment modalities is diagnosed in more than 85% of cases. In this study, a multi-feature multi-attention network (MFMANet) was proposed for NSCLC subtype classification to address the problem of low classification accuracy between subtypes due to small lesion size and similar background. In MFMANet, the multi-scale spatial channel attention module (MSAM) and multi-feature fusion global local attention module (MFGLA), are proposed. MSAM is able to preserve spatial features during channel feature fusion to capture high-order statistical information. MFGLA performs effective fusion of multiple features to avoid interference caused by scale differences. The global and local information is extracted by global and local attention branches to enhance the perception of small lesion regions. The performance of the proposed MFMANet was validated on two public datasets and compared with other classification networks including ResNet18, ShuffleNetv2, MobileNetv3, and MnasNet. MFMANet achieved 99.06% and 91.67% accuracy in the binary classification of CT images of lung adenocarcinoma (ADC) and lung squamous cell carcinoma (SCC), outperforming other methods. This study confirms that the proposed MFMANet provides effective solution for the subtypes classification of NSCLC.

SL-HarDNet: Skin lesion segmentation with HarDNet.

Automatic segmentation of skin lesions from dermoscopy is of great significance for the early diagnosis of skin cancer. However, due to the complexity and fuzzy boundary of skin lesions, automatic segmentation of skin lesions is a challenging task. In this paper, we present a novel skin lesion segmentation network based on HarDNet (SL-HarDNet). We adopt HarDNet as the backbone, which can learn more robust feature representation. Furthermore, we introduce three powerful modules, including: cascaded fusion module (CFM), spatial channel attention module (SCAM) and feature aggregation module (FAM). Among them, CFM combines the features of different levels and effectively aggregates the semantic and location information of skin lesions. SCAM realizes the capture of key spatial information. The cross-level features are effectively fused through FAM, and the obtained high-level semantic position information features are reintegrated with the features from CFM to improve the segmentation performance of the model. We apply the challenge dataset ISIC-2016&PH2 and ISIC-2018, and extensively evaluate and compare the state-of-the-art skin lesion segmentation methods. Experiments show that our SL-HarDNet performance is always superior to other segmentation methods and achieves the latest performance.

A Multiscale Spatiotemporal Fusion Network Based on an Attention Mechanism

Spatiotemporal fusion is an effective and cost-effective method to obtain both high temporal resolution and high spatial resolution images. However, existing methods do not sufficiently extract the deeper features of the image, resulting in fused images which do not recover good topographic detail and poor fusion quality. In order to obtain higher quality spatiotemporal fusion images, a novel spatiotemporal fusion method based on deep learning is proposed in this paper. The method combines an attention mechanism and a multiscale feature fusion network to design a network that more scientifically explores deeper features of the image for different input image characteristics. Specifically, a multiscale feature fusion module is introduced into the spatiotemporal fusion task and combined with an efficient spatial-channel attention module to improve the capture of spatial and channel information while obtaining more effective information. In addition, we design a new edge loss function and incorporate it into the compound loss function, which helps to generate fused images with richer edge information. In terms of both index performance and image details, our proposed model has excellent results on both datasets compared with the current mainstream spatiotemporal fusion methods.

HSCA-Net: A Hybrid Spatial-Channel Attention Network in Multi-Scale Feature Pyramid for Document Layout Analysis

Document images often contain various page components and complex logical structures, which makes document layout analysis task challenging. For most deep learning based document layout analysis methods, convolutional neural networks (CNNs) are adopted as the image feature extraction networks. In this paper, a hybrid spatial-channel attention network (HSCA-Net) is proposed to improve feature extraction capability by exerting attention mechanism to explore more salient properties within document pages. The HSCA-Net contains two modules: spatial attention module (SAM) and channel attention module (CAM). They are embedded in the multi-scale feature network by lateral attention connection. SAM extracts contextual information with learning offset in spatial dimension and CAM performs feature recalibration by focusing more on feature channels with important contents. The lateral attention connection is to incorporate SAM and CAM into multi-scale feature pyramid network and retain more of the original feature information. The effectiveness and adaptability of HSCA-Net are evaluated through multiple experiments on publicly available datasets PubLayNet, ICDAR-POD and Article Regions. The mAP on these datasets is as high as 0.940,0.939 and 0.967 respectively, which demonstrate that our HSCA-Net achieves competitive results on document layout analysis task. Text.

Novel Spatial–Spectral Channel Attention Neural Network for Land Cover Change Detection with Remote Sensed Images

Land cover change detection (LCCD) with remote-sensed images plays an important role in observing Earth’s surface changes. In recent years, the use of a spatial-spectral channel attention mechanism in information processing has gained interest. In this study, aiming to improve the performance of LCCD with remote-sensed images, a novel spatial-spectral channel attention neural network (SSCAN) is proposed. In the proposed SSCAN, the spatial channel attention module and convolution block attention module are employed to process pre- and post-event images, respectively. In contrast to the scheme of traditional methods, the motivation of the proposed operation lies in amplifying the change magnitude among the changed areas and minimizing the change magnitude among the unchanged areas. Moreover, a simple but effective batch-size dynamic adjustment strategy is promoted to train the proposed SSCAN, thus guaranteeing convergence to the global optima of the objective function. Results from comparative experiments of seven cognate and state-of-the-art methods effectively demonstrate the superiority of the proposed network in accelerating the network convergence speed, reinforcing the learning efficiency, and improving the performance of LCCD. For example, the proposed SSCAN can achieve an improvement of approximately 0.17–23.84% in OA on Dataset-A.

A Deeply Supervised Attentive High-Resolution Network for Change Detection in Remote Sensing Images

Change detection (CD) is a crucial task in remote sensing (RS) to distinguish surface changes from bitemporal images. Recently, deep learning (DL) based methods have achieved remarkable success for CD. However, the existing methods lack robustness to various kinds of changes in RS images, which suffered from problems of feature misalignment and inefficient supervision. In this paper, a deeply supervised attentive high-resolution network (DSAHRNet) is proposed for remote sensing image change detection. First, we design a spatial-channel attention module to decode change information from bitemporal features. The attention module is able to model spatial-wise and channel-wise contexts. Second, to reduce feature misalignment, the extracted features are refined by stacked convolutional blocks in parallel. Finally, a novel deeply supervised module is introduced to generate more discriminative features. Extensive experimental results on three challenging benchmark datasets demonstrate that the proposed DSAHRNet outperforms other state-of-the-art methods, and achieves a great trade-off between performance and complexity.

GA-Net: A geometry prior assisted neural network for road extraction

With geospatial intelligence research developing rapidly, automatic road extraction is becoming a fundamental and challenging task. Due to the special geometric structure and spectral information of road networks, existing methods suffer from incomplete and fractured results. In this work, a novel road extraction convolutional neural network, incorporating the road boundary details and road junction information via a dual-branch multi-task structure, is proposed to learn synergistic feature representations and strengthen road connectivity. Firstly, a BiFPN-based feature aggregation module is utilised to bridge the semantic gap between low-level and high-level feature maps, allowing multi-scale spatial details to be fully fused. Secondly, the boundary auxiliary branch, using a U-shaped network with a spatial-channel attention module, captures residential information for the backbone to enhance the subtleties of road edges. Thirdly, the node inferring branch models the road junction position jointly with the road surface, aiming to strengthen the topology structure and reduce the fragmented road segments. We perform experiments on three diverse road datasets, namely the DeepGlobe dataset, Massachusetts dataset, and SpaceNet dataset. The results demonstrate that our model shows an overall performance improvement over some SOTA algorithms and the IoU indicator achieves 3.86%, 0.79%, and 1.71% improvements over Unet on the three datasets, respectively. • A novel Geometric Prior Assisted Neural Network (GA-Net) is proposed for automatic road extraction. The boundary information is introduced to the model, which strengthens the geometric details of road edges. The road junction information is jointly learned with the road surface information, which reduces the missing gap between road segments. • A feature fusion module is designed to bridge the semantic gap between multi-scale features maps. An attention module is designed to strengthen the road geometric information. The two light modules can be easily ensemble with other models. • We achieves better results than some SOTA algorithms for three popular datasets. And results of our model shows better completeness and connectivity.

Tracker Meets Night: A Transformer Enhancer for UAV Tracking

Most previous progress in object tracking is realized in daytime scenes with favorable illumination. State-of-the-arts can hardly carry on their superiority at night so far, thereby considerably blocking the broadening of visual tracking-related unmanned aerial vehicle (UAV) applications. To realize reliable UAV tracking at night, a spatial-channel Transformer-based low-light enhancer (namely SCT), which is trained in a novel task-inspired manner, is proposed and plugged prior to tracking approaches. To achieve semantic-level low-light enhancement targeting the high-level task, the novel spatial-channel attention module is proposed to model global information while preserving local context. In the enhancement process, SCT denoises and illuminates nighttime images simultaneously through a robust non-linear curve projection. Moreover, to provide a comprehensive evaluation, we construct a challenging nighttime tracking benchmark, namely DarkTrack2021, which contains 110 challenging sequences with over 100 K frames in total. Evaluations on both the public UAVDark135 benchmark and the newly constructed DarkTrack2021 benchmark show that the task-inspired design enables SCT with significant performance gains for nighttime UAV tracking compared with other top-ranked low-light enhancers. Real-world tests on a typical UAV platform further verify the practicability of the proposed approach. The DarkTrack2021 benchmark and the code of the proposed approach are publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/vision4robotics/SCT</uri> .

Multi-scale features fused network with multi-level supervised path for crowd counting

Many CNN-based methods which utilize the density map to regress the count number of crowd are introduced to solve the crowd counting problem lately. Due to the head scale variations caused by the perspective change and background noise, these methods cannot address these two problems well in highly crowded scenario. In order to solve these two problems, we introduce a multi-scale features fused network with multi-level supervised path to produce the high-quality density map in this paper. Our model utilizes the first 13 layers of VGG16 model as the backbone, the multi-level supervised path in our model employs the multi-level dilated convolution module (MLD) to supervise the whole network at multi-level, and generate the attention map for the density map, which is used to handle the scale variations. The other path is used to fuse multi-scale features to generate the density map with soft spatial-channel attention module (SSCA) which aims to produce a saliency weight map of same size. In the end, the final density map is captured by the feature map multiply the attention map. In addition, a new objective function is proposed to train our network. A large number of experimental results show that compared with other networks, our method achieves better experimental results on four challenging datasets (UCF_CC_50, ShanghaiTech, UCF-QRNF and WorldExpo’10 dataset).

Deep Attention and Graphical Neural Network for Multiple Sclerosis Lesion Segmentation From MR Imaging Sequences.

The segmentation of multiple sclerosis (MS) lesions from MR imaging sequences remains a challenging task, due to the characteristics of variant shapes, scattered distributions and unknown numbers of lesions. However, the current automated MS segmentation methods with deep learning models face the challenges of (1) capturing the scattered lesions in multiple regions and (2) delineating the global contour of variant lesions. To address these challenges, in this paper, we propose a novel attention and graph-driven network (DAG-Net), which incorporates (1) the spatial correlations for embracing the lesions in distant regions and (2) the global context for better representing lesions of variant features in a unified architecture. Firstly, the novel local attention coherence mechanism is designed to construct dynamic and expansible graphs for the spatial correlations between pixels and their proximities. Secondly, the proposed spatial-channel attention module enhances features to optimize the global contour delineation, by aggregating relevant features. Moreover, with the dynamic graphs, the learning process of the DAG-Net is interpretable, which in turns support the reliability of segmentation results. Extensive experiments were conducted on a public ISBI2015 dataset and an in-house dataset in comparison to state-of-the-art methods, based on geometrical and clinical metrics. The experimental results validate the effectiveness of proposed DAG-Net on segmenting variant and scatted lesions in multiple regions.

Bidirectional difference locating and semantic consistency reasoning for change captioning

Change captioning is an emerging task to describe the changes between a pair of images. The difficulty in this task is to discover the differences between the two images. Recently, some methods have been proposed to address this problem. However, they all employ unidirectional difference localization to identify the changes. This can lead to ambiguity about the nature of the changes. Instead, we propose a framework with bidirectional difference localization and semantic consistency reasoning to describe the image changes. First, we locate the changes in the two images by capturing bidirectional differences. Then we design a decoder with spatial-channel attention to generate the change caption. Finally, we introduce semantic consistency reasoning to constrain our bidirectional difference localization module and spatial-channel attention module. Extensive experiments on three public data sets show that the performance of our proposed model outperforms the state-of-the-art change captioning models by a large margin.

Multiple Sclerosis Lesions Segmentation Using Attention-Based CNNs in FLAIR Images.

Objective: Multiple Sclerosis (MS) is an autoimmune and demyelinating disease that leads to lesions in the central nervous system. This disease can be tracked and diagnosed using Magnetic Resonance Imaging (MRI). A multitude of multimodality automatic biomedical approaches are used to segment lesions that are not beneficial for patients in terms of cost, time, and usability. The authors of the present paper propose a method employing just one modality (FLAIR image) to segment MS lesions accurately. Methods: A patch-based Convolutional Neural Network (CNN) is designed, inspired by 3D-ResNet and spatial-channel attention module, to segment MS lesions. The proposed method consists of three stages: (1) the Contrast-Limited Adaptive Histogram Equalization (CLAHE) is applied to the original images and concatenated to the extracted edges to create 4D images; (2) the patches of size n}{}80times 80times 80times2n are randomly selected from the 4D images; and (3) the extracted patches are passed into an attention-based CNN which is used to segment the lesions. Finally, the proposed method was compared to previous studies of the same dataset. Results: The current study evaluates the model with a test set of ISIB challenge data. Experimental results illustrate that the proposed approach significantly surpasses existing methods of Dice similarity and Absolute Volume Difference while the proposed method uses just one modality (FLAIR) to segment the lesions. Conclusion: The authors have introduced an automated approach to segment the lesions, which is based on, at most, two modalities as an input. The proposed architecture comprises convolution, deconvolution, and an SCA-VoxRes module as an attention module. The results show, that the proposed method outperforms well compared to other methods.

Local and global encoder network for semantic segmentation of Airborne laser scanning point clouds

Interpretation of Airborne Laser Scanning (ALS) point clouds is a critical procedure for producing various geo-information products like 3D city models, digital terrain models and land use maps. In this paper, we present a local and global encoder network (LGENet) for semantic segmentation of ALS point clouds. Adapting the KPConv network, we first extract features by both 2D and 3D point convolutions to allow the network to learn more representative local geometry. Then global encoders are used in the network to exploit contextual information at the object and point level. We design a segment-based Edge Conditioned Convolution to encode the global context between segments. We apply a spatial-channel attention module at the end of the network, which not only captures the global interdependencies between points but also models interactions between channels. We evaluate our method on two ALS datasets namely, the ISPRS benchmark dataset and DCF2019 dataset. For the ISPRS benchmark dataset, our model achieves state-of-the-art results with an overall accuracy of 0.845 and an average F1 score of 0.737. With regards to the DFC2019 dataset, our proposed network achieves an overall accuracy of 0.984 and an average F1 score of 0.834.

Satellite Components Detection from Optical Images Based on Instance Segmentation Networks

For satellite interactions missions such as autonomous docking, the key to successful completion of these missions is to autonomously and accurately detect targets’ information. The missions with high precision requirement need not only to obtain the categories and location of target satellite components but also to obtain the high-level information such as the corner point. In this paper, a satellite components detection (SCD) method based on improved CenterMask is proposed, which is the state-of-the-art instance segmentation network, to autonomously and accurately detect the target components. First, the fully convolutional one-stage object detection detector is optimized to better obtain categories and bounding boxes of targets. Next, a spatial-channel attention module is proposed, and it is introduced in the mask branch to improve the performance of segmentation. Finally, a satellite components dataset is built for model training, and the optimized SCD model is obtained after training. To better satisfy missions with a higher speed requirement, a speed-up method is also proposed, which greatly improves the detection speed at the expense of very little accuracy. Experiments show that, compared with CenterMask, our method improves 2.8 and 1.5% of box average precision () and mask , and the speed-up version improves the speed of 5.2 frames per second.

Quantitative Analysis of Metallographic Image Using Attention-Aware Deep Neural Networks.

As a detection tool to identify metal or alloy, metallographic quantitative analysis has received increasing attention for its ability to evaluate quality control and reveal mechanical properties. The detection procedure is mainly operated manually to locate and characterize the constitution in metallographic images. The automatic detection is still a challenge even with the emergence of several excellent models. Benefiting from the development of deep learning, with regard to two different metallurgical structural steel image datasets, we propose two attention-aware deep neural networks, Modified Attention U-Net (MAUNet) and Self-adaptive Attention-aware Soft Anchor-Point Detector (SASAPD), to identify structures and evaluate their performance. Specifically, in the case of analyzing single-phase metallographic image, MAUNet investigates the difference between low-frequency and high-frequency and prevents duplication of low-resolution information in skip connection used in an U-Net like structure, and incorporates spatial-channel attention module with the decoder to enhance interpretability of features. In the case of analyzing multi-phase metallographic image, SASAPD explores and ranks the importance of anchor points, forming soft-weighted samples in subsequent loss design, and self-adaptively evaluates the contributions of attention-aware pyramid features to assist in detecting elements in different sizes. Extensive experiments on the above two datasets demonstrate the superiority and effectiveness of our two deep neural networks compared to state-of-the-art models on different metrics.