Channel-wise Feature Research Articles

Squeeze & Excitation joint with Combined Channel and Spatial Attention for Pathology Image Super-Resolution

Super-resolution (SR) techniques are pivotal in enhancing low-resolution images and crucial in medical diagnosis, where detail and clarity are paramount. Traditional pixel-loss-based SR methods, while adept at producing high-resolution (HR) images, often result in artifice content. This loss of information compromises both the visual experience and the accuracy of subsequent diagnoses. Addressing this, we have developed an innovative SR approach integrating a joint Squeeze and Excitation (SE) mechanism with a Combined Channel and Spatial Attention (CCSA) mechanism. The SE mechanism effectively recalibrates channel-wise feature responses, enhancing the representational capacity of the network. Meanwhile, the CCSA mechanism focuses on extracting spatial and channel-wise features, ensuring that critical high-frequency details are preserved. The dual approach significantly refines the quality of the images, maintaining essential details necessary for accurate medical diagnosis. To validate our proposed approach, we used a benchmark dataset, bcSR, tailored to challenge SR models to focus on broader and more critical regions. Comparative analysis proves that our model excels in performance over existing state-of-the-art methods. In conclusion, our proposed SR Network, with its innovative SE and CCSA mechanisms, offers a potent tool for pathology image SR. It elevates the quality of super-resolved images, which will significantly aid in the accuracy and efficiency of medical diagnoses, providing a valuable asset to medical professionals.

Hybrid attentive prototypical network for few-shot action recognition

Most previous few-shot action recognition works tend to process video temporal and spatial features separately, resulting in insufficient extraction of comprehensive features. In this paper, a novel hybrid attentive prototypical network (HAPN) framework for few-shot action recognition is proposed. Distinguished by its joint processing of temporal and spatial information, the HAPN framework strategically manipulates these dimensions from feature extraction to the attention module, consequently enhancing its ability to perform action recognition tasks. Our framework utilizes the R(2+1)D backbone network, coupling the extraction of integrated temporal and spatial features to ensure a comprehensive understanding of video content. Additionally, our framework introduces the novel Residual Tri-dimensional Attention (ResTriDA) mechanism, specifically designed to augment feature information across the temporal, spatial, and channel dimensions. ResTriDA dynamically enhances crucial aspects of video features by amplifying significant channel-wise features for action distinction, accentuating spatial details vital for capturing the essence of actions within frames, and emphasizing temporal dynamics to capture movement over time. We further propose a prototypical attentive matching module (PAM) built on the concept of metric learning to resolve the overfitting issue common in few-shot tasks. We evaluate our HAPN framework on three classical few-shot action recognition datasets: Kinetics-100, UCF101, and HMDB51. The results indicate that our framework significantly outperformed state-of-the-art methods. Notably, the 1-shot task, demonstrated an increase of 9.8% in accuracy on UCF101 and improvements of 3.9% on HMDB51 and 12.4% on Kinetics-100. These gains confirm the robustness and effectiveness of our approach in leveraging limited data for precise action recognition.

G-Net: Implementing an enhanced brain tumor segmentation framework using semantic segmentation design.

A fundamental computer vision task called semantic segmentation has significant uses in the understanding of medical pictures, including the segmentation of tumors in the brain. The G-Shaped Net architecture appears in this context as an innovative and promising design that combines components from many models to attain improved accuracy and efficiency. In order to improve efficiency, the G-Shaped Net architecture synergistically incorporates four fundamental components: the Self-Attention, Squeeze Excitation, Fusion, and Spatial Pyramid Pooling block structures. These factors work together to improve the precision and effectiveness of brain tumor segmentation. Self-Attention, a crucial component of G-Shaped architecture, gives the model the ability to concentrate on the image's most informative areas, enabling accurate localization of tumor boundaries. By adjusting channel-wise feature maps, Squeeze Excitation completes this by improving the model's capacity to capture fine-grained information in the medical pictures. Since the G-Shaped model's Spatial Pyramid Pooling component provides multi-scale contextual information, the model is capable of handling tumors of various sizes and complexity levels. Additionally, the Fusion block architectures combine characteristics from many sources, enabling a thorough comprehension of the image and improving the segmentation outcomes. The G-Shaped Net architecture is an asset for medical imaging and diagnostics and represents a substantial development in semantic segmentation, which is needed more and more for accurate brain tumor segmentation.

A Novel Real-Time Detection and Classification Method for ECG Signal Images Based on Deep Learning.

In this paper, a novel deep learning method Mamba-RAYOLO is presented, which can improve detection and classification in the processing and analysis of ECG images in real time by integrating three advanced modules. The feature extraction module in our work with a multi-branch structure during training can capture a wide range of features to ensure efficient inference and rich feature extraction. The attention mechanism module utilized in our proposed network can dynamically focus on the most relevant spatial and channel-wise features to improve detection accuracy and computational efficiency. Then, the extracted features can be refined for efficient spatial feature processing and robust feature fusion. Several sets of experiments have been carried out to test the validity of the proposed Mamba-RAYOLO and these indicate that our method has made significant improvements in the detection and classification of ECG images. The research offers a promising framework for more accurate and efficient medical ECG diagnostics.

Improved diabetic retinopathy severity classification using squeeze-and-excitation and sparse light weight multi-level attention u-net with transfer learning from xception.

Diabetic Retinopathy (DR) is a significant cause of vision loss in diabetic patients, making early detection and accurate severity classification essential for effective management and prevention. This study aims to develop an enhanced DR severity classification approach using advanced model architectures and transfer learning to improve diagnostic accuracy and support better patient care. We propose a novel model, Xception Squeeze-and-Excitation Sparse Lightweight Multi-Level Attention U-Net (XceSE_SparseLwMLA-UNet), designed to classify DR severity using fundus images from the Messidor 1 and Messidor 2 datasets. The XceSE_SparseLwMLA-UNet integrates several advanced mechanisms: the Squeeze-and-Excitation (SE) mechanism for adaptive feature recalibration, the Sparse Lightweight Multi-Level Attention (SparseLwMLA) mechanism for effective contextual information integration, and transfer learning from the Xception architecture to enhance feature extraction capabilities. The SE mechanism refines channel-wise feature responses, while SparseLwMLA enhances the model's ability to identify complex DR patterns. Transfer learning utilizes pre-trained weights from Xception to improve generalization across DR severity levels. The proposed XceSE_SparseLwMLA-UNet model demonstrates superior performance in DR severity classification, achieving higher accuracy and improved multi-class F1 scores compared to existing models. The model's color-coded segmentation outputs offer interpretable visual representations, aiding medical professionals in assessing DR severity levels. The XceSE_SparseLwMLA-UNet model shows promise for advancing early DR diagnosis and management by enhancing classification accuracy and providing valuable visual insights. Its integration of advanced architectural features and transfer learning contributes to better patient care and improved visual health outcomes.

Attentive multi-granularity perception network for person search

Person search is an extremely challenging task that seeks to identify individuals through joint person detection and person re-identification from uncropped real scene images. Previous studies primarily focus on learning rich features to enhance identification. However, arbitrary feature enhancement strategies may introduce unwanted background noise. Moreover, different scenarios usually exhibit varying pedestrian appearances or even intricate occlusions, leading to inconsistent/incomplete pedestrian features in different images. In this paper, we introduce a novel Attentive Multi-granularity Perception (AMP) module seamlessly integrated into our AMPN network. This module specifically addresses appearance variations and occlusions within a person's Region of Interest (RoI). The AMP module harnesses discriminative relationship features from various local regions, significantly enhancing identification accuracy. It comprises two principal components: the Pedestrian Perception Enhancement (PPE) block and the Background Interference Suppressor (BIS). The PPE block introduces a Spatial-wise Feature Mixer and a Channel-wise Feature Mixer, which effectively capture and refine discriminative relation features. Simultaneously, the BIS operates in parallel with the PPE block, enriching the discriminative relation features and enhancing the distinctiveness between the foreground and background. Our AMP module is plug-and-play and can integrate with other person search models. Extensive experiments validate our model's merits, achieving state-of-the-art performance on CUHK-SYSU and a 4.8% mAP gain over SeqNet on PRW at a desirable speed. Our code is accessible at https://github.com/zqx951102/AMPN.

Hierarchical channel-spatial interaction network for partial-to-partial point cloud registration

ABSTRACT Partial-to-partial point cloud registration is a fundamental and challenging task in the three-dimensional computer vision. In this letter, we propose a novel Hierarchical Channel-Spatial Interaction Network (HCSINet), which can hierarchically explore information interaction in terms of the channel and spatial dimensions, enabling well partial point cloud perception for partial-to-partial registration. Specifically, the Channel-wise Feature Interaction (CFI) module is regarded as the basic unit of the encoder sub-network, which achieves feature hierarchical interaction to learn the overlapping scores and intuitively improve the importance of overlapping areas. Furthermore, to improve the discrimination of the overlapping points, the Co-Attention-based Spatial Association (CA-SA) module progressively excavates multi-scale spatial correlation of overlapping points by conjointly computing collaborative attention. Experimental results verify the validity of our HCSINet, which achieves competitive partial-to-partial registration performances compared to existing traditional and deep learning-based methods. The source code is publicly available https://github.com/lxnudt/HCSINet.

Efficient cross-modality feature interaction for multispectral armored vehicle detection

Detecting armed vehicles from a UAV platform is challenging due to the complexity of ground environment. This paper presents a dual-stream multispectral armored vehicle detection method to tackle this problem. First, considering that there is a paucity of datasets containing multispectral armored vehicle images, a multispectral armored vehicle detection dataset is constructed for this study. The dataset consists of 5853 pairs of RGB and infrared images, featuring a total of 15,878 instances of armored vehicles. Then, a cross-modal feature interaction module is designed to enable efficient feature interaction between multispectral images. This module uses the cross-modal channel-wise feature difference method to model the channel differences between the two modal features and obtains the cross-modal channel difference matrix. The cross-modal channel difference matrix is then employed to extract the unique features of the two modal features, allowing for efficient cross-modal feature interaction by complementing each other's unique features. Experiment results demonstrate that the proposed model has excellent detection performance and is capable of coping with various challenges brought by complex ground environments.

Automated crack localization for road safety using contextual u-net with spatial-channel feature integration

Accurate and timely crack localization is crucial for road safety and maintenance, but image processing and hand-crafted feature engineering methods, often fail to distinguish cracks from background noise under diverse lighting and surface conditions. This paper proposes a framework utilizing contextual U-Net deep learning model to automatically localize cracks in road images. The framework design considers crack localization as a task of pixel-level segmenting, and analyzing each pixel in a road image to determine if it belongs to a crack or not. The proposed U-Net model uses a robust EfficientNet encoder to capture crucial details (spatial features) and overall patterns (channel-wise features) within the road image. This balanced approach helps the model learn effectively from both individual elements and the context of the images, leading to improved crack detection. A customized hierarchical attention mechanism is designed to make U-Net model contextually adaptive to focus on relevant features at different scales and resolutions for accurately localizing road cracks that can vary widely in size and shape. The model's effectiveness is demonstrated through extensive evaluations on the benchmarked and custom-made datasets.

Learning content-aware feature fusion for guided depth map super-resolution

RGB-D data including paired RGB color images and depth maps is widely used in downstream computer vision tasks. However, compared with the acquisition of high-resolution color images, the depth maps captured by consumer-level sensors are always in low resolution. Within decades of research, the most state-of-the-art (SOTA) methods of depth map super-resolution cannot adaptively tune the guidance fusion for all feature positions by channel-wise feature concatenation with spatially sharing convolutional kernels. This paper proposes JTFNet to resolve this issue, which simulates the traditional Joint Trilateral Filter (JTF). Specifically, a novel JTF block is introduced to adaptively tune the fusion pattern between the color features and the depth features for all feature positions. Moreover, based on the variant of JTF block whose target features and guidance features are in the cross-scale shape, the fusion for depth features is performed in a bi-directional way. Therefore, the error accumulation along scales can be effectively mitigated by iteratively HR feature guidance. Compared with the SOTA methods, the sufficient experiment is conducted on the mainstream synthetic datasets and real datasets, i.e., Middlebury, NYU and ToF-Mark, which shows remarkable improvement of our JTFNet.

Sh-DeepLabv3+: An Improved Semantic Segmentation Lightweight Network for Corn Straw Cover Form Plot Classification

Straw return is one of the main methods for protecting black soil. Efficient and accurate straw return detection is important for the sustainability of conservation tillage. In this study, a rapid straw return detection method is proposed for large areas. An optimized Sh-DeepLabv3+ model based on the aforementioned detection method and the characteristics of straw return in Jilin Province was then used to classify plots into different straw return cover types. The model used Mobilenetv2 as the backbone network to reduce the number of model parameters, and the channel-wise feature pyramid module based on channel attention (CA-CFP) and a low-level feature fusion module (LLFF) were used to enhance the segmentation of the plot details. In addition, a composite loss function was used to solve the problem of class imbalance in the dataset. The results show that the extraction accuracy is optimal when a 2048 × 2048-pixel scale image is used as the model input. The total parameters of the improved model are 3.79 M, and the mean intersection over union (MIoU) is 96.22%, which is better than other comparative models. After conducting a calculation of the form–grade mapping relationship, the error value of the area prediction was found to be less than 8%. The results show that the proposed rapid straw return detection method based on Sh-DeepLabv3+ can provide greater support for straw return detection.

An Automated Multi-scale Feature Fusion Network for Spine Fracture Segmentation Using Computed Tomography Images.

Spine fractures represent a critical health concern with far-reaching implications for patient care and clinical decision-making. Accurate segmentation of spine fractures from medical images is a crucial task due to its location, shape, type, and severity. Addressing these challenges often requires the use of advanced machine learning and deep learning techniques. In this research, a novel multi-scale feature fusion deep learning model is proposed for the automated spine fracture segmentation using Computed Tomography (CT) to these challenges. The proposed model consists of six modules; Feature Fusion Module (FFM), Squeeze and Excitation (SEM), Atrous Spatial Pyramid Pooling (ASPP), Residual Convolution Block Attention Module (RCBAM), Residual Border Refinement Attention Block (RBRAB), and Local Position Residual Attention Block (LPRAB). These modules are used to apply multi-scale feature fusion, spatial feature extraction, channel-wise feature improvement, segmentation border results border refinement, and positional focus on the region of interest. After that, a decoder network is used to predict the fractured spine. The experimental results show that the proposed approach achieves better accuracy results in solving the above challenges and also performs well compared to the existing segmentation methods.

MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Unit Detection

The Facial Action Coding System (FACS) encodes the action units (AUs) in facial images, which has attracted extensive research attention due to its wide use in facial expression analysis. Many methods that perform well on automatic facial action unit (AU) detection primarily focus on modeling various AU relations between corresponding local muscle areas or mining global attention–aware facial features; however, they neglect the dynamic interactions among local-global features. We argue that encoding AU features just from one perspective may not capture the rich contextual information between regional and global face features, as well as the detailed variability across AUs, because of the diversity in expression and individual characteristics. In this article, we propose a novel Multi-level Graph Relational Reasoning Network (termed MGRR-Net ) for facial AU detection. Each layer of MGRR-Net performs a multi-level (i.e., region-level, pixel-wise, and channel-wise level) feature learning. On the one hand, the region-level feature learning from the local face patch features via graph neural network can encode the correlation across different AUs. On the other hand, pixel-wise and channel-wise feature learning via graph attention networks (GAT) enhance the discrimination ability of AU features by adaptively recalibrating feature responses of pixels and channels from global face features. The hierarchical fusion strategy combines features from the three levels with gated fusion cells to improve AU discriminative ability. Extensive experiments on DISFA and BP4D AU datasets show that the proposed approach achieves superior performance than the state-of-the-art methods.

Convolutional Channel-Wise Competitive Learning for the Forward-Forward Algorithm

The Forward-Forward (FF) Algorithm has been recently proposed to alleviate the issues of backpropagation (BP) commonly used to train deep neural networks. However, its current formulation exhibits limitations such as the generation of negative data, slower convergence, and inadequate performance on complex tasks. In this paper we take the main ideas of FF and improve them by leveraging channel-wise competitive learning in the context of convolutional neural networks for image classification tasks. A layer-wise loss function is introduced that promotes competitive learning and eliminates the need for negative data construction. To enhance both the learning of compositional features and feature space partitioning, a channel-wise feature separator and extractor block is proposed that complements the competitive learning process. Our method outperforms recent FF-based models on image classification tasks, achieving testing errors of 0.58%, 7.69%, 21.89%, and 48.77% on MNIST, Fashion-MNIST, CIFAR-10 and CIFAR-100 respectively. Our approach bridges the performance gap between FF learning and BP methods, indicating the potential of our proposed approach to learn useful representations in a layer-wise modular fashion, enabling more efficient and flexible learning. Our source code and supplementary material are available at https://github.com/andreaspapac/CwComp.

3D-IncNet: Head and Neck (H&N) Primary Tumors Segmentation and Survival Prediction.

Cancer begins when healthy cells change and grow out of control, forming a mass called a tumor. Head and neck (H&N) cancers usually develop in or around the head and neck, including the mouth (oral cavity), nose and sinuses, throat (pharynx), and voice box (larynx). 4% of all cancers are H&N cancers with a very low survival rate (a five-year survival rate of 64.7%). FDG-PET/CT imaging is often used for early diagnosis and staging of H&N tumors, thus improving these patients' survival rates. This work presents a novel 3D-Inception-Residual aided with 3D depth-wise convolution and squeeze and excitation block. We introduce a 3D depth-wise convolution-inception encoder consisting of an additional 3D squeeze and excitation block and a 3D depth-wise convolution-based residual learning decoder (3D-IncNet), which not only helps to recalibrate the channel-wise features but adaptively through explicit inter-dependencies modeling but also integrate the coarse and fine features resulting in accurate tumor segmentation. We further demonstrate the effectiveness of inception-residual encoder-decoder architecture in achieving better dice scores and the impact of depth-wise convolution in lowering the computational cost. We applied random forest for survival prediction on deep, clinical, and radiomics features. Experiments are conducted on the benchmark HECKTOR21 challenge, which showed significantly better performance by surpassing the state-of-the-artwork and achieved 0.836 and 0.811 concordance index and dice scores, respectively. We made the model and code publicly available.

MSFF: An automated multi-scale feature fusion deep learning model for spine fracture segmentation using MRI

Spinal fractures pose a significant healthcare challenge, impacting patient care and clinical decisions. Accurately identifying these fractures from medical images is crucial due to their location, shape, type, and severity. Meeting these challenges often necessitates employing advanced machine learning and deep learning techniques. This study introduces a novel deep learning model called MSFF (Multi-Scale Feature Fusion) specifically designed for automated spine fracture segmentation using MRI images. The MSFF architecture comprises five key modules: the Feature Fusion Module (FFM), Squeeze and Excitation (SEM), Atrous Spatial Pyramid Pooling (ASPP), Residual Convolution Block Attention Module (RCBAM), Residual Border Refinement Attention Block (RBRAB), and Local Position Residual Attention Block (LPRAB). These modules collectively facilitate multi-scale feature fusion, spatial feature extraction, channel-wise feature enhancement, segmentation border refinement, and focused attention on the region of interest. Subsequently, a decoder network is employed to predict fractured spine areas. Experimental findings demonstrate that the proposed MSFF approach attains superior accuracy in addressing the challenges and outperforms existing segmentation methods.

Improving Transferability of Universal Adversarial Perturbation with Feature Disruption.

Deep neural networks (DNNs) are shown to be vulnerable to universal adversarial perturbations (UAP), a single quasi-imperceptible perturbation that deceives the DNNs on most input images. The current UAP methods can be divided into data-dependent and data-independent methods. The former exhibits weak transferability in black-box models due to overly relying on model-specific features. The latter shows inferior attack performance in white-box models as it fails to exploit the model's response information to benign images. To address the above issues, this paper proposes a novel universal adversarial attack to generate UAP with strong transferability by disrupting the model-agnostic features (e.g., edges or simple texture), which are invariant to the models. Specifically, we first devise an objective function to weaken the significant channel-wise features and strengthen the less significant channel-wise features, which are partitioned by the designed strategy. Furthermore, the proposed objective function eliminates the dependency on labeled samples, allowing us to utilize out-of-distribution (OOD) data to train UAP. To enhance the attack performance with limited training samples, we exploit the average gradient of the mini-batch input to update the UAP iteratively, which encourages the UAP to capture the local information inside the mini-batch input. In addition, we introduce the momentum term to accumulate the gradient information at each iterative step for the purpose of perceiving the global information over the training set. Finally, extensive experimental results demonstrate that the proposed methods outperform the existing UAP approaches. Additionally, we exhaustively investigate the transferability of the UAP across models, datasets, and tasks.

Channel-Wise Feature Decorrelation for Enhanced Learned Image Compression

Channel-Wise Feature Decorrelation for Enhanced Learned Image Compression

Enhancing non-intrusive load monitoring with channel attention guided bi-directional temporal convolutional network for sequence-to-point learning

This study addresses the crucial aspect of identifying individual appliance power consumption without extensive sensor deployment, a cornerstone of modern smart grid planning and demand response. Non-Intrusive Load Monitoring (NILM) offers a solution by estimating appliance energy usage from aggregated meter data, increasingly powered by deep learning techniques. However, the depth-dependent effectiveness of load feature extraction can lead to gradient-related issues. To mitigate this, novel approach using a Bi-directional Temporal Convolutional Network (BiTCN) as a residual block foundation, enhanced by a Squeeze-and-Excitation Network (SENet) attention mechanism for channel-wise feature extraction. A departure from conventional methods involves substituting bidirectional non-causal convolution with channel attention for improved feature extraction was introduced. Evaluation on REDD and UK-DALE datasets demonstrates our model’s superiority in load disaggregation compared to existing approaches, emphasizing its potential in advancing NILM through effective deep learning strategies

Visual image design of the internet of things based on AI intelligence

Visual object detection has emerged as a critical technology for Unmanned Arial Vehicle (UAV) use due to advances in computer vision. New developments in fields like communication technology and the UAV needs to be able to act autonomously by gathering data and then making choices. These tendencies have brought us to cutting-edge levels of health care, transportation, energy, monitoring, and security for visual image detection and manufacturing endeavors. These include coordination in communication via IoT, sustainability of IoT network, and optimization challenges in path planning. Because of their limited battery life, these gadgets are limited in their range of communication. UAVs can be seen as terminal devices connected to a large network where a swarm of other UAVs is coordinating their motions, directing one another, and maintaining watch over locations outside its visual range. One of the essential components of UAV-based applications is the ability to recognize objects of interest in aerial photographs taken by UAVs. While aerial photos might be useful, object detection is challenging. As a result, capturing aerial photographs with UAVs is a unique challenge since the size of things in these images might vary greatly. The study proposal included specific information regarding the Detection of Visual Images by UAVs (DVI-UAV) using the IoT and Artificial Intelligence (AI). Included in the study of AI is the concept of DSYolov3. The DSYolov3 model was presented to deal with these problems in the UAV industry. By fusing the channel-wise feature across multiple scales using a spatial pyramid pooling approach, the proposed study creates a novel module, Multi-scale Fusion of Channel Attention (MFCAM), for scale-variant object identification tasks. The method's effectiveness and efficiency have been thoroughly tested and evaluated experimentally. The suggested method would allow us to outperform most current detectors and guarantee that the models will be useable on UAVs. There will be a 95 % success rate in terms of visual image detection, a 94 % success rate in terms of computation cost, a 97 % success rate in terms of accuracy, and a 95 % success rate in terms of effectiveness.