Global Branch Research Articles

Convolution-Enhanced Bi-Branch Adaptive Transformer With Cross-Task Interaction for Food Category and Ingredient Recognition.

Recently, visual food analysis has received more and more attention in the computer vision community due to its wide application scenarios, e.g., diet nutrition management, smart restaurant, and personalized diet recommendation. Considering that food images are unstructured images with complex and unfixed visual patterns, mining food-related semantic-aware regions is crucial. Furthermore, the ingredients contained in food images are semantically related to each other due to the cooking habits and have significant semantic relationships with food categories under the hierarchical food classification ontology. Therefore, modeling the long-range semantic relationships between ingredients and the categories-ingredients semantic interactions is beneficial for ingredient recognition and food analysis. Taking these factors into consideration, we propose a multi-task learning framework for food category and ingredient recognition. This framework mainly consists of a food-orient Transformer named Convolution-Enhanced Bi-Branch Adaptive Transformer (CBiAFormer) and a multi-task category-ingredient recognition network called Structural Learning and Cross-Task Interaction (SLCI). In order to capture the complex and unfixed fine-grained patterns of food images, we propose a query-aware data-adaptive attention mechanism called Bi-Branch Adaptive Attention (BiA-Attention) in CBiAFormer, which consists of a local fine-grained branch and a global coarse-grained branch to mine local and global semantic-aware regions for different input images through an adaptive candidate key/value sets assignment for each query. Additionally, a convolutional patch embedding module is proposed to extract the fine-grained features which are neglected by Transformers. To fully utilize the ingredient information, we propose SLCI, which consists of cross-layer attention to model the semantic relationships between ingredients and two cross-task interaction modules to mine the semantic interactions between categories and ingredients. Extensive experiments show that our method achieves competitive performance on three mainstream food datasets (ETH Food-101, Vireo Food-172, and ISIA Food-200). Visualization analyses of CBiAFormer and SLCI on two tasks prove the effectiveness of our method. Codes will be released upon publication. Code and models are available at https://github.com/Liuyuxinict/CBiAFormer.

Adaptive Blind Super-Resolution Network for Spatial-Specific and Spatial-Agnostic Degradations.

Prior methodologies have disregarded the diversities among distinct degradation types during image reconstruction, employing a uniform network model to handle multiple deteriorations. Nevertheless, we discover that prevalent degradation modalities, including sampling, blurring, and noise, can be roughly categorized into two classes. We classify the first class as spatial-agnostic dominant degradations, less affected by regional changes in image space, such as downsampling and noise degradation. The second class degradation type is intimately associated with the spatial position of the image, such as blurring, and we identify them as spatial-specific dominant degradations. We introduce a dynamic filter network integrating global and local branches to address these two degradation types. This network can greatly alleviate the practical degradation problem. Specifically, the global dynamic filtering layer can perceive the spatial-agnostic dominant degradation in different images by applying weights generated by the attention mechanism to multiple parallel standard convolution kernels, enhancing the network's representation ability. Meanwhile, the local dynamic filtering layer converts feature maps of the image into a spatially specific dynamic filtering operator, which performs spatially specific convolution operations on the image features to handle spatial-specific dominant degradations. By effectively integrating both global and local dynamic filtering operators, our proposed method outperforms state-of-the-art blind super-resolution algorithms in both synthetic and real image datasets.

Adaptive spatial-temporal neural network for ADHD identification using functional fMRI.

Computer aided diagnosis methods play an important role in Attention Deficit Hyperactivity Disorder (ADHD) identification. Dynamic functional connectivity (dFC) analysis has been widely used for ADHD diagnosis based on resting-state functional magnetic resonance imaging (rs-fMRI), which can help capture abnormalities of brain activity. However, most existing dFC-based methods only focus on dependencies between two adjacent timestamps, ignoring global dynamic evolution patterns. Furthermore, the majority of these methods fail to adaptively learn dFCs. In this paper, we propose an adaptive spatial-temporal neural network (ASTNet) comprising three modules for ADHD identification based on rs-fMRI time series. Specifically, we first partition rs-fMRI time series into multiple segments using non-overlapping sliding windows. Then, adaptive functional connectivity generation (AFCG) is used to model spatial relationships among regions-of-interest (ROIs) with adaptive dFCs as input. Finally, we employ a temporal dependency mining (TDM) module which combines local and global branches to capture global temporal dependencies from the spatially-dependent pattern sequences. Experimental results on the ADHD-200 dataset demonstrate the superiority of the proposed ASTNet over competing approaches in automated ADHD classification.

Few-Shot Learning Meets Transformer: Unified Query-Support Transformers for Few-Shot Classification

The goal of Few-shot classification (FSL) is to identify unseen classes with very limited samples has attracted more and more attention. Usually, it is formulated as a metric learning problem. The core issue of few-shot classification is how to learn (1) consistent representations for images in both support and query sets and (2) effective metric learning for images between support and query sets. In this paper, we show that the two challenges can be well modeled simultaneously via a unified Query-Support TransFormer (QSFormer) model. To be specific, the proposed QSFormer involves <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">global</i> query-support sample Transformer (sampleFormer) branch and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">local</i> patch Transformer (patchFormer) learning branch. sampleFormer aims to capture the dependence of samples in support and query sets for image representation. It adopts the Encoder, QS-Decoder and Cross-Attention to respectively model the Support, Query (image) representation and Metric learning for few-shot classification task. Also, as a complementary to global learning branch, we adopt a local patch Transformer to extract structural representation for each image sample by capturing the long-range dependence of local image patches. In addition, we introduce a novel Cross-scale Interactive Feature Extractor (CIFE) to extract and fuse different scale CNN features as an effective backbone module for the proposed few-shot learning method. We integrate these into a unified framework and train it in an end-to-end way. A large number of experiments are conducted on four popular datasets to validate the superiority and effectiveness of the proposed QSFormer.

Multi-scale Perception and Feature Refinement Network for multi-class segmentation of intracerebral hemorrhage in CT images

Intracerebral hemorrhage (ICH) poses a severe threat to human health and well-being. Automatic segmentation of hematomas in CT images can provide essential diagnostic assistance to physicians, and ensure improved treatment and recovery outcomes for patients. Existing methods for intracerebral hemorrhage segmentation mainly focus on identifying hemorrhage areas, without the ability to accurately distinguish and outline different types of hematomas. However, different types of hemorrhage exhibit a high degree of similarity in terms of gray matter level and shape, and the scale of hematomas can vary significantly. To address this issue, we propose a Multi-scale Perception and Feature Refinement Network (MPFR-Net) for automatic segmentation of both intraparenchymal and intraventricular hemorrhages. Specifically, we propose a Multi-scale Perception Module (MPM), which consists of the integration of features at different levels and the local and global multi-scale branches. MPM allows for the efficient extraction of multi-scale features and the establishment of long-range relationships between the target and background. Additionally, we propose a Feature Refinement Module (FRM) to refine fuzzy detail information that has been lost after down-sampling to the deep layer, while simultaneously supplementing the small target information from shallow features. To improve the clinical adaptability of our method, we further collect 608 patient cases from multiple hospitals to construct a multi-center dataset, termed as ICH-Seg in which each case contains both intraparenchymal and intraventricular hemorrhages. From the quantitative and visual results, MPFR-Net outperforms previous methods on both private and public datasets, showing promising segmentation for intracerebral hemorrhage and potential clinical applications.

QDRJL: Quaternion dynamic representation with joint learning neural network for heart sound signal abnormal detection

At present, deep learning based heart sound diagnosis algorithms are mostly complex and large models for high accuracy, which are difficult to deploy on mobile devices due to the high number of parameters and large computational cost. The current mainstream approach for processing heart sound signals involves utilizing their Mel-frequency cepstral coefficients (MFCC) features. However, most existing methods have overlooked the multi-channel characteristics of MFCC. To address this issue, we propose a Quaternion Dynamic Representation with Joint Learning (QDRJL) neural network for learning MFCC multi-channel features. Our proposed approach combines quaternion dynamic convolution with dynamic weighting and the Quaternion Interior Learning Block (QILB). Finally, we present a global and energy joint learning branch for jointly learning MFCC features. The success of the proposed quaternion network depends on its ability to utilize the internal relations between quaternion-valued input features and the definition of the dynamic weight variables in the augmented quaternion domain. We assessed various state-of-the-art classification algorithms for detecting heart sounds and found that our proposed classifier achieved an accuracy of up to 97.2%, outperforming existing models. Our experimental evaluation, using the 2016 PhysioNet/CinC Challenge dataset, revealed that our model could reduce the number of network parameters to 25% due to quaternion properties.

Crucial feature capture and discrimination for limited training data SAR ATR

Deep learning-based methods have demonstrated exceptional performance in the field of synthetic aperture radar automatic target recognition (SAR ATR). However, obtaining a sufficient number of labeled SAR images remains a significant challenge that can negatively affect the performance of these methods. This is because most deep learning models use the entire target image as input. However, but research has shown that with limited training data, the model may not be able to capture discriminative regions of the image. Instead, they might focus on more useless even harmful image regions for recognition, leading to poor recognition results. In this study, we propose a novel SAR ATR framework that addresses the limitation of limited training data. The proposed framework primarily comprises a global assisted branch, locally enhanced branch, feature capture module, and feature discrimination module. The global-assisted branch conducts an initial recognition and provides a loss based on the entire SAR image during each training epoch and the feature capture module automatically segments and captures crucial image regions for current recognition, which we refer to as the “golden key” of the image. The local enhanced branch then performs another recognition and provides another loss function based on the image parts. Instead of updating the model with two basic recognition losses to roughly search for and capture crucial image parts, a feature discrimination module is proposed to combine the global and local branches in a subtle manner to improve local feature separability or compactness for similar inter-class or dissimilar inner-class sample pairs in global features. This adaptively forced the model to capture more crucial image parts and extract more effective features. Experimental results and comparisons of the MSTAR and OpenSARship datasets indicated that the proposed method achieves superior recognition performance compared to existing methods. The effectiveness of our method was further demonstrated through the visualization of the golden key of the testing images and the recognition performance in ablation experiments. We will release our codes and more experimental results at https://github.com/cwwangSARATR/SARATR_FeaCapture_Discrimination.

MAXFormer: Enhanced transformer for medical image segmentation with multi-attention and multi-scale features fusion

Convolutional neural networks(CNN), especially U-shaped networks, have become the mainstream approach for medical image segmentation. However, due to the intrinsic locality of convolutional operations, CNN has inherent limitations in capturing long-range dependencies. Although Transformer-based methods have demonstrated remarkable performance in computer vision by modeling long-range dependencies, their high computational complexity and reliance on large-scale pre-training present challenges, particularly for higher-resolution medical images. In this paper, we introduce MAXFormer, a U-shaped hierarchical network that effectively leverages global context within individual samples and relationships between different samples. Our Transformer module reformulates the self-attention mechanism into two parts: local–global attention and external attention. The local–global attention provides an efficient alternative to self-attention with linear complexity, employing a parallel architecture that allows local–global spatial interactions. The local attention branch captures high-frequency local information, while the global attention branch captures low-frequency global information. Furthermore, we have designed the Refined Fused Connection module to effectively merge feature outputs from each encoder block with the decoder output, mitigating spatial detail loss due to downsampling. Extensive experiments on two different medical image segmentation datasets show that our proposed method outperforms other state-of-the-art methods without requiring pre-training weights. Code will be available at https://github.com/zhiwei-liang/MAXFormer.

Supplement and Suppression: Both Boundary and Nonboundary are Helpful for Salient Object Detection.

Current methods aggregate multilevel features from the backbone and introduce edge information to get more refined saliency maps. However, little attention is paid to how to suppress the regions with similar saliency appearances in the background. These regions usually exist in the vicinity of salient objects and have high contrast with the background, which is easy to be misclassified as foreground. To solve this problem, we propose a gated feature interaction network (GFINet) to integrate multiple saliency features, which can utilize nonboundary features with background information to suppress pseudosalient objects and simultaneously apply boundary features to supplement edge details. Different from previous methods that only consider the complementarity between saliency and boundary, the proposed network introduces nonboundary features into the decoder to filter the pseudosalient objects. Specifically, GFINet consists of global features aggregation branch (GFAB), boundary and nonboundary features' perception branch (B&NFPB), and gated feature interaction module (GFIM). According to the global features generated by GFAB, boundary and nonboundary features produced by B&NFPB and GFIM employ a gate structure to adaptively optimize the saliency information interchange between abovementioned features and, thus, predict the final saliency maps. Besides, due to the imbalanced distribution between the boundary pixels and nonboundary ones, the binary cross-entropy (BCE) loss is difficult to predict the pixels near the boundary. Therefore, we design a border region aware (BRA) loss to further boost the quality of boundary and nonboundary, which can guide the network to focus more on the hard pixels near the boundary by assigning different weights to different positions. Compared with 12 counterparts, experimental results on five benchmark datasets show that our method has better generalization and improves the state-of-the-art approach by 4.85% averagely in terms of the regional and boundary evaluation measures. In addition, our model is more efficient with an inference speed of 50.3 FPS when processing a 320 x 320 image. Code has been made available at https://github.com/lesonly/GFINet.

Spectral Segmentation Multi-Scale Feature Extraction Residual Networks for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is a vital task in hyperspectral image processing and applications. Convolutional neural networks (CNN) are becoming an effective approach for categorizing hyperspectral remote sensing images as deep learning technology advances. However, traditional CNN usually uses a fixed kernel size, which limits the model’s capacity to acquire new features and affects the classification accuracy. Based on this, we developed a spectral segmentation-based multi-scale spatial feature extraction residual network (MFERN) for hyperspectral image classification. MFERN divides the input data into many non-overlapping sub-bands by spectral bands, extracts features in parallel using the multi-scale spatial feature extraction module MSFE, and adds global branches on top of this to obtain global information of the full spectral band of the image. Finally, the extracted features are fused and sent into the classifier. Our MSFE module has multiple branches with increasing ranges of the receptive field (RF), enabling multi-scale spatial information extraction at both fine- and coarse-grained levels. On the Indian Pines (IP), Salinas (SA), and Pavia University (PU) HSI datasets, we conducted extensive experiments. The experimental results show that our model has the best performance and robustness, and our proposed MFERN significantly outperforms other models in terms of classification accuracy, even with a small amount of training data.

Global-and-local aware network for low-light image enhancement

Photos taken under nighttime or backlit conditions often suffer from complex and unpredictable degradation, such as low visibility, messy noise, and distorted color. Previous methods mainly focused on global brightness and contrast while ignoring structural and textural details, or they handled the fusion of features without adequately considering their intrinsic association, resulting in incomplete feature representations. To address this issue, we propose a global-and-local aware network (GLAN) by projecting the features into the frequency domain and incorporating them in a knowledge-sharing manner. This method effectively integrates the global modeling capability of the transformer and the local sensitivity of the convolutional neural network to represent structure and texture. First, the global branch, which is comprised of transformer blocks, performs feature extraction under the global receptive field, while the local branch constructs multi-scale features to provide local fine-grained details. Then, we design a novel adaptive multi-scale feature block (AMSFB) that deploys channel split operation to decrease the calculation amount. To better learn the channel and spatial correlations of intermediate features, we introduce a multi-scale channel attention module (MSCAM) and a pixel attention module (PAM) into the AMSFB. Finally, a frequency-aware interaction module (FAIM) is developed for bidirectional information supplementation, which builds feature descriptors simultaneously covering low-frequency and high-frequency information based on the discrete cosine transform (DCT). Through extensive quantitative and qualitative experiments, our method can achieve competitive results compared with over ten state-of-the-art image enhancement methods on eight benchmark datasets.

PMSG-Net: A priori-guided multilevel graph transformer fusion network for immunotherapy efficacy prediction

In the case of specific immunotherapy regimens and access to pre-treatment CT scans, developing reliable, interpretable intelligent image biomarkers to predict efficacy is essential for physician decision-making and patient treatment selection. However, varying levels of prognosis show a similar appearance on CT scans. It becomes challenging to stratify patients by a single pre-treatment CT scan when presenting subtle differences in images for experienced experts and existing prognostic classification methods. In addition, the pattern of peri-tumoural radiological structures also determines the patient's response to ICIs. Therefore, it is essential to develop a method that focuses on the clinical priori features of the tumour edges but also makes full use of the rich information within the 3D tumour. This paper proposes a priori-guided multilevel graph transformer fusion network (PMSG-Net). Specifically, a graph convolutional network is first used to obtain a feature representation of the tumour edge, and complementary information from that detailed representation is used to enhance the global representation. In the tumour global representation branch (MSGNet), we designed the cascaded scale-enhanced swin transformer to obtain attributes of graph nodes, and efficiently learn and model spatial dependencies and semantic connections at different scales through multi-hop context-aware attention (MCA), yielding a richer global semantic representation. To our knowledge, this is the first attempt to use graph neural networks to predict the efficacy of immunotherapy, and the experimental results show that this method outperforms the current mainstream methods.

Pengaruh Pelayanan, Kualitas Jaringan, Harga Yang Kompetitif Terhadap Kepuasan Pelanggan Dan Dampaknya Terhadap Kinerja Keuangan PT Global Media Data Prima Cabang Kediri

This study aims to determine the effect of service, network quality, competitive prices on customer satisfaction and the impact on financial performance of pt global media data prima kediri branch. This case study was conducted at PT Global Media Data Prima, Kediri branch. Barokah in 2020-2021 which consists of variables (X) including: service, network quality, competitive prices, variable (Y) customer satisfaction and external variables of financial performance. The sampling technique uses a cluster random sampling technique with random sample types based on area. The results of this study indicate that H1 Service has no effect on customer satisfaction, whereas H2 network quality has no effect on customer satisfaction, and H3 competitive prices have a significant effect on customer satisfaction. Based on the three equations, it proves that customer satisfaction can affect financial performance, the higher the customer satisfaction, the higher the financial performance obtained by the company. The smaller the customer satisfaction, the smaller the financial performance obtained by the company.

Exploiting Complementary Dynamic Incoherence for DeepFake Video Detection

Recently, manipulated videos based on DeepFake technology have spread widely on social media, causing concerns about the authenticity of video content and personal privacy protection. Although existing DeepFake detection methods achieve remarkable progress in some specific scenarios, their detection performance usually drops drastically when detecting unseen manipulation methods. Compared with static information such as human face, dynamic information depicting the movements of facial features is more difficult to forge without leaving visual or statistical traces. Hence, in order to achieve better generalization ability, we focus on dynamic information analysis to disclose such traces and propose a novel Complementary Dynamic Interaction Network (CDIN). Inspired by the DeepFake detection methods based on mouth region analysis, both the global (entire face) and local (mouth region) dynamics are analyzed with properly designed network branches, respectively, and their feature maps at various levels are communicated with each other using a newly proposed Complementary Cross Dynamics Fusion Module (CCDFM). With CCDFM, the global branch will pay more attention to anomalous mouth movements and the local branch will gain more information about the global context. Finally, a multi-task learning scheme is designed to optimize the network with both the global and local information. Extensive experiments have demonstrated that our approach achieves better detection results compared with several SOTA methods, especially in detecting video forgeries manipulated by unseen methods.

Three-dimensional ultrasound image reconstruction based on 3D-ResNet in the musculoskeletal system using a 1D probe: ex vivo and in vivo feasibility studies

Objective. Three-dimensional (3D) ultrasound (US) is needed to provide sonographers with a more intuitive panoramic view of the complex anatomical structure, especially the musculoskeletal system. In actual scanning, sonographers may perform fast scanning using a one-dimensional (1D) array probe .at random angles to gain rapid feedback, which leads to a large US image interval and missing regions in the reconstructed volume. Approach. In this study, a 3D residual network (3D-ResNet) modified by a 3D global residual branch (3D-GRB) and two 3D local residual branches (3D-LRBs) was proposed to retain detail and reconstruct high-quality 3D US volumes with high efficiency using only sparse two-dimensional (2D) US images. The feasibility and performance of the proposed algorithm were evaluated on ex vivo and in vivo sets. Main r esults. High-quality 3D US volumes in the fingers, radial and ulnar bones, and metacarpophalangeal joints were obtained by the 3D-ResNet, respectively. Their axial, coronal, and sagittal slices exhibited rich texture and speckle details. Compared with kernel regression, voxel nearest-neighborhood, squared distance weighted methods, and a 3D convolution neural network in the ablation study, the mean peak-signal-to-noise ratio and mean structure similarity of the 3D-ResNet were up to 28.53 ± 1.29 dB and 0.98 ± 0.01, respectively, and the corresponding mean absolute error dropped to 0.023 ± 0.003 with a better resolution gain of 1.22 ± 0.19 and shorter reconstruction time. Significance. These results illustrate that the proposed algorithm can rapidly reconstruct high-quality 3D US volumes in the musculoskeletal system in cases of a large amount of data loss. This suggests that the proposed algorithm has the potential to provide rapid feedback and precise analysis of stereoscopic details in complex and meticulous musculoskeletal system scanning with a less limited scanning speed and pose variations for the 1D array probe.

Modified Architecture of Tomasulo’s Algorithm using Adaptive Branch Predictor

This paper proposes modified architecture to implement the Tomasulo’s Algorithm using Adaptive branch prediction scheme, which evaluate the branch outcome by taking both local and global history dynamically. The choice of local vs. global branch prediction is made dynamically on a path based predictor that decides which branch must be taken on past correctness of choice.

Partial Label Multi-organ Segmentation based on Local Feature Enhancement.

The automatic segmentation of abdominal organs from CT images is essential for surgical planning of abdominal diseases. However, each medical institution only annotates some organs according to its own clinical practice. This brings the partial annotation problem to multi-center abdominal multi-organ segmentation. To address this issue, we introduce a 3D local feature enhanced multi-head segmentation network for multi-organ segmentation of abdominal regions in multiple partially labeled datasets. More specifically, our proposed architecture consists of two branches, the global branch with 3D Transformer and U-Net fusion named 3D TransUNet as the backbone, and the local 3D U-Net branch that provides additional abdominal organ structure information to the global branch to generate more accurate segmentation results. We evaluate our method on four publicly available CT datasets with four different partial label. Our experiments show that the proposed approach provides better accuracy and robustness, with 93.01% average Dice-score-coefficient (DSC) and 3.489 mm Hausdorff Distance (HD) outperforming three existing state-of-the-art methods.

Context-aware lightweight remote-sensing image super-resolution network.

In recent years, remote-sensing image super-resolution (RSISR) methods based on convolutional neural networks (CNNs) have achieved significant progress. However, the limited receptive field of the convolutional kernel in CNNs hinders the network's ability to effectively capture long-range features in images, thus limiting further improvements in model performance. Additionally, the deployment of existing RSISR models to terminal devices is challenging due to their high computational complexity and large number of parameters. To address these issues, we propose a Context-Aware Lightweight Super-Resolution Network (CALSRN) for remote-sensing images. The proposed network primarily consists of Context-Aware Transformer Blocks (CATBs), which incorporate a Local Context Extraction Branch (LCEB) and a Global Context Extraction Branch (GCEB) to explore both local and global image features. Furthermore, a Dynamic Weight Generation Branch (DWGB) is designed to generate aggregation weights for global and local features, enabling dynamic adjustment of the aggregation process. Specifically, the GCEB employs a Swin Transformer-based structure to obtain global information, while the LCEB utilizes a CNN-based cross-attention mechanism to extract local information. Ultimately, global and local features are aggregated using the weights acquired from the DWGB, capturing the global and local dependencies of the image and enhancing the quality of super-resolution reconstruction. The experimental results demonstrate that the proposed method is capable of reconstructing high-quality images with fewer parameters and less computational complexity compared with existing methods.

Global branches and local states of the human gut microbiome define associations with environmental and intrinsic factors

The gut microbiome is important for human health, yet modulation requires more insight into inter-individual variation. Here, we explored latent structures of the human gut microbiome across the human lifespan, applying partitioning, pseudotime, and ordination approaches to >35,000 samples. Specifically, three major gut microbiome branches were identified, within which multiple partitions were observed in adulthood, with differential abundances of species along branches. Different compositions and metabolic functions characterized the branches’ tips, reflecting ecological differences. An unsupervised network analysis from longitudinal data from 745 individuals showed that partitions exhibited connected gut microbiome states rather than over-partitioning. Stability in the Bacteroides-enriched branch was associated with specific ratios of Faecalibacterium:Bacteroides. We also showed that associations with factors (intrinsic and extrinsic) could be generic, branch- or partition-specific. Our ecological framework for cross-sectional and longitudinal data allows a better understanding of overall variation in the human gut microbiome and disentangles factors associated with specific configurations.

Global and pyramid convolutional neural network with hybrid attention mechanism for hyperspectral image classification

Convolutional neural networks (CNNs) have shown impressive results in the hyperspectral image (HSI) classification. However, they still face certain limitations that can impact their effectiveness. The kernels in standard convolution have fixed spatial sizes and spectral depths, which cannot capture global semantic features from HSI and the existing methods for extracting multiscale information are inadequate. Based on these, we propose a global and pyramid convolutional network with a hybrid attention mechanism (GPHANet) for HSI classification. GPHANet adopts a two-branch architecture to extract both local and global hyperspectral features, leveraging the strengths of each to enhance classification performance. The local branch employs dynamic pyramid convolution, which customizes parameters for convolutional kernels of multiple scales based on the input image. This design enables the model to accurately and comprehensively extract multiscale feature information, capturing subtle variations in the HSI. The global branch utilizes circular convolution with a global receptive field, enabling each position in the convolutional layers to gather information from the entire input space. This allows the model to learn global contextual features, enhancing its understanding of the overall structure and semantics of the HSI. After extracting deep global and local features, we apply a compact hybrid attention mechanism (HAM) to capture long-range dependencies along both spatial and spectral dimensions, resulting in a more discriminative feature representation. On the Pavia University, Salinas, and Hong Hu datasets, the proposed model achieved overall accuracies of 99.22%, 98.74%, and 96.80%, respectively, using limited training samples. These results are much better compared to the state-of-the-art methods.