Global Features Research Articles

Bridging branches and attributes: spectral-spatial global-local interaction network for hyperspectral image classification

ABSTRACT The CNN-Transformer joint model stands as the leading architecture for contemporary hyperspectral image (HSI) classification, integrating global and local features through either successive or dual-branch CNN and Transformer networks. However, these methods often fall short in effectively incorporating spatial-spectral information with local-global attributes, resulting in incomplete feature representation. To address these challenges, we propose a spectral-spatial global-local interaction network that transmits global and local features into the spatial and spectral branches, facilitated by cross-interaction operators to ensure adequate feature flow. Initially, CNNs are employed to separately extract shallow features for the spectral and spatial branches. We then introduce a Spectral-Spatial Global-Local Interaction block designed for deep feature extraction, enhancing the flow of spectral and spatial features with local and global attributes using parallel transformers and dynamic convolutions. Transformers model the long-range dependencies of global spectral and spatial features, while dynamic convolutions enhance the context sensitivity of local spectral and spatial representations. Quadruple cross-interaction blocks are proposed to traverse both the spectral-spatial branches and local-global attribute dimensions, facilitating information exchange for complementary HSI representation. Extensive experiments and ablation studies on four public HSI datasets demonstrate the superiority of our proposed method. convolutional neural networks (CNNs); spectral and spatial; global and local; hyperspectral image classification; cross-interaction.

Patch-based dual-domain photon-counting CT data correction with residual-based WGAN-ViT

Objective.x-ray photon-counting detectors have recently gained popularity due to their capabilities in energy discrimination power, noise suppression, and resolution refinement. The latest extremity photon-counting computed tomography (PCCT) scanner leverages these advantages for tissue characterization, material decomposition, beam hardening correction, and metal artifact reduction. However, technical challenges such as charge splitting and pulse pileup can distort the energy spectrum and compromise image quality. Also, there is a clinical need to balance radiation dose and imaging speed for contrast-enhancement and other studies. This paper aims to address these challenges by developing a dual-domain correction approach to enhance PCCT reconstruction quality quantitatively and qualitatively.Approach.We propose a novel correction method that operates in both projection and image domains. In the projection domain, we employ a residual-based Wasserstein generative adversarial network to capture local and global features, suppressing pulse pileup, charge splitting, and data noise. This is facilitated with traditional filtering methods in the image domain to enhance signal-to-noise ratio while preserving texture across each energy channel. To address GPU memory constraints, our approach utilizes a patch-based volumetric refinement network.Main results.Our dual-domain correction approach demonstrates significant fidelity improvements across both projection and image domains. Experiments on simulated and real datasets reveal that the proposed model effectively suppresses noise and preserves intricate details, outperforming the state-of-the-art methods.Significance.This approach highlights the potential of dual-domain PCCT data correction to enhance image quality for clinical applications, showing promise for advancing PCCT image fidelity and applicability in preclinical/clinical environments.

Instance-level semantic segmentation of nuclei based on multimodal structure encoding

BackgroundAccurate segmentation and classification of cell nuclei are crucial for histopathological image analysis. However, existing deep neural network-based methods often struggle to capture complex morphological features and global spatial distributions of cell nuclei due to their reliance on local receptive fields.MethodsThis study proposes a graph neural structure encoding framework based on a vision-language model. The framework incorporates: (1) A multi-scale feature fusion and knowledge distillation module utilizing the Contrastive Language-Image Pre-training (CLIP) model’s image encoder; (2) A method to transform morphological features of cells into textual descriptions for semantic representation; and (3) A graph neural network approach to learn spatial relationships and contextual information between cell nuclei.ResultsExperimental results demonstrate that the proposed method significantly improves the accuracy of cell nucleus segmentation and classification compared to existing approaches. The framework effectively captures complex nuclear structures and global distribution features, leading to enhanced performance in histopathological image analysis.ConclusionsBy deeply mining the morphological features of cell nuclei and their spatial topological relationships, our graph neural structure encoding framework achieves high-precision nuclear segmentation and classification. This approach shows significant potential for enhancing histopathological image analysis, potentially leading to more accurate diagnoses and improved understanding of cellular structures in pathological tissues.

Hybrid U-Net Model with Visual Transformers for Enhanced Multi-Organ Medical Image Segmentation

Medical image segmentation is an essential process that facilitates the precise extraction and localization of diseased areas from medical pictures. It can provide clear and quantifiable information to support clinicians in making final decisions. However, due to the lack of explicit modeling of global relationships in CNNs, they are unable to fully use the long-range dependencies among several image locations. In this paper, we propose a novel model that can extract local and global semantic features from the images by utilizing CNN and the visual transformer in the encoder. It is important to note that the self-attention mechanism treats a 2D image as a 1D sequence of patches, which can potentially disrupt the image’s inherent 2D spatial structure. Therefore, we utilized the structure of the transformer using visual attention and large kernel attention, and we added a residual convolutional attention module (RCAM) and multi-scale fusion convolution (MFC) into the decoder. They can help the model better capture crucial features and fine details to improve detail and accuracy of segmentation effects. On the synapse multi-organ segmentation (Synapse) and the automated cardiac diagnostic challenge (ACDC) datasets, our model performed better than the previous models, demonstrating that it is more precise and robust in multi-organ medical image segmentation.

Spatial Cognitive Electroencephalogram Network Topological Features Extraction Based on Cross Fuzzy Entropy Network Graph

Spatial cognition, a critical component of human cognitive function, can be enhanced through targeted training, such as virtual reality (VR)-based interventions. Recent advances in electroencephalography (EEG)-based functional connectivity analysis have highlighted the importance of network topology features for understanding cognitive processes. In this paper, a framework based on a cross fuzzy entropy network graph (CFENG) is proposed to extract spatial cognitive EEG network topological features. This framework involves calculating the similarity and symmetry between EEG channels using cross fuzzy entropy, constructing weighted directed network graphs, transforming one-dimensional EEG signals into two-dimensional brain functional connectivity networks, and extracting both local and global topological features. The model’s performance is evaluated and interpreted using an XGBoost classifier. Experiments on an EEG dataset from group spatial cognitive training validated the CFENG model. In the Gamma band, the CFENG achieved 97.82% classification accuracy, outperforming existing methods. Notably, the asymmetrically distributed EEG channels Fp1, P8, and Cz contributed most to spatial cognitive signal classification. An analysis after 28 days of training revealed that specific VR games enhanced functional centrality in spatial cognition-related brain regions, reduced information flow path length, and altered information flow symmetry. These findings support the feasibility of VR-based spatial cognitive training from a brain functional connectivity perspective.

Cross-ViT based benign and malignant classification of pulmonary nodules.

The benign and malignant discrimination of pulmonary nodules plays a very important role in diagnosing the extent of lung cancer lesions. There are many methods using Convolutional neural network (CNN) for benign and malignant classification of pulmonary nodules, but traditional CNN models focus more on the local features of pulmonary nodules and lack the extraction of global features of pulmonary nodules. To solve this problem, a Cross fusion attention ViT (Cross-ViT) network that fuses local features extracted by CNN and global features extracted by Transformer is proposed. The network first extracts different features independently through two branches and then performs feature fusion through the Cross fusion attention module. Cross-ViT can effectively capture and process both local and global information of lung nodules, which improves the accuracy of classifying the benign and malignant nature of pulmonary nodules. Experimental validation was performed on the LUNA16 dataset, and the accuracy, precision, recall and F1 score reached 91.04%, 91.42%, 92.45% and 91.92%, respectively, and the accuracy, precision, recall and F1 score with SENet as CNN branch reached 92.43%, 94.27%, 91.68% and 92.96%, respectively. The results show that the accuracy, precision, recall and F1 score of the proposed method are 0.3%, 0.11%, 4.52% and 3.03% higher than those of the average optimal method, respectively, and the performance of Cross-ViT network for benign and malignant classification is better than most classification methods.

Deep Learning-Based Point Cloud Classification of Obstacles for Intelligent Vehicles

Intelligent driving research has focused much attention on point cloud obstacles since they are a class of high-dimensional data that can adequately depict the shape and placement of obstacles, unlike picture data. Currently, deep learning technology is primarily employed for vehicle autonomy point cloud obstacle classification tasks. These techniques typically struggle with low classification accuracy, processing efficiency, and model stability. To tackle the abovementioned issues, this paper suggests a novel random forest algorithm that integrates the out-of-bag error theory and can consistently and accurately evaluate the influence of point cloud properties. Then, building on the novel algorithm, this paper suggests a modified PointNet network that incorporates the effects of both global and local features on the classification task, therefore increasing the conventional network’s classification accuracy. To assess the effectiveness of this novel approach in the experimental portion, we set up an evaluation system based on the metrics for average accuracy, overall accuracy, and a confusion matrix. According to the simulation results, the overall accuracy of the proposed network in terms of classification accuracy is 94.4% and the average accuracy is 84.9%, which are then compared to the prototype PointNet and its variants. The classification accuracies for the four types of obstacles are 97.6%, 63.6%, 92.5%, and 86.1%. In addition, the proposed method is effective at improving both the computational complexity and stability of the network.

Online and Cross-User Finger Movement Pattern Recognition by Decoding Neural Drive Information from Surface Electromyogram.

Cross-user variability is a well-known challenge that leads to severe performance degradation and impacts the robustness of practical myoelectric control systems. To address this issue, a novel method for myoelectric recognition of finger movement patterns is proposed by incorporating a neural decoding approach with unsupervised domain adaption (UDA) learning. In our method, the neural decoding approach is implemented by extracting microscopic features characterizing individual motor unit (MU) activities obtained from a two-stage online surface electromyogram (SEMG) decomposition. A specific deep learning model is designed and initially trained using labeled data from a set of existing users. The model can update adaptively when recognizing the movement patterns of a new user. The final movement pattern was determined by a fuzzy weighted decision strategy. SEMG signals were collected from the finger extensor muscles of 15 subjects to detect seven dexterous finger-movement patterns. The proposed method achieved a movement pattern recognition accuracy of ([Formula: see text])% over seven movements under cross-user testing scenarios, much higher than that of the conventional methods using global SEMG features. Our study presents a novel robust myoelectric pattern recognition approach at a fine-grained MU level, with wide applications in neural interface and prosthesis control.

Aberrant Functional Connectivity and Brain Network Organization in High-Schizotypy Individuals: An Electroencephalography Study.

Oscillatory synchrony plays a crucial role in establishing functional connectivity across distinct brain regions. Within the realm of schizophrenia, suggested to be a neuropsychiatric disconnection syndrome, discernible aberrations arise in the organization of brain networks. We aim to investigate whether the resting-state functional network is already altered in healthy individuals with high schizotypy traits, highlighting the pivotal influence of brain rhythms in driving brain network alterations. Two-minute resting-state electroencephalography recordings were conducted on healthy participants with low and high schizotypy scores. Subsequently, spectral Granger causality was used to compute functional connectivity in theta, alpha, beta, and gamma frequency bands, and graph theory metrics were employed to assess global and local brain network features. Results highlighted that high-schizotypy individuals exhibit a lower local efficiency in theta and alpha frequencies and a decreased global efficiency across theta, alpha, and beta frequencies. Moreover, high schizotypy is characterized by a lower nodes' centrality and a frequency-specific decrease of functional connectivity, with a reduced top-down connectivity mostly in slower frequencies and a diminished bottom-up connectivity in faster rhythms. These results show that healthy individuals with a higher risk of developing psychosis exhibit a less efficient functional brain organization, coupled with a systematic decrease in functional connectivity impacting both bottom-up and top-down processing. These frequency-specific network alterations provide robust support for the dimensional model of schizophrenia, highlighting distinctive neurophysiological signatures in high-schizotypy individuals.

DeepGlioSeg: advanced glioma MRI data segmentation with integrated local-global representation architecture

IntroductionGlioma segmentation is vital for diagnostic decision-making, monitoring disease progression, and surgical planning. However, this task is hindered by substantial heterogeneity within gliomas and imbalanced region distributions, posing challenges to existing segmentation methods.MethodsTo address these challenges, we propose the DeepGlioSeg network, a U-shaped architecture with skip connections for continuous contextual feature integration. The model includes two primary components. First, a CTPC (CNN-Transformer Parallel Combination) module leverages parallel branches of CNN and Transformer networks to fuse local and global features of glioma images, enhancing feature representation. Second, the model computes a region-based probability by comparing the number of pixels in tumor and background regions and assigns greater weight to regions with lower probabilities, thereby focusing on the tumor segment. Test-time augmentation (TTA) and volume-constrained (VC) post-processing are subsequently applied to refine the final segmentation outputs.ResultsExtensive experiments were conducted on three publicly available glioma MRI datasets and one privately owned clinical dataset. The quantitative and qualitative findings consistently show that DeepGlioSeg achieves superior segmentation performance over other state-of-the-art methods.DiscussionBy integrating CNN- and Transformer-based features in parallel and adaptively emphasizing underrepresented tumor regions, DeepGlioSeg effectively addresses the challenges associated with glioma heterogeneity and imbalanced region distributions. The final pipeline, augmented with TTA and VC post-processing, demonstrates robust segmentation capabilities. The source code for this work is publicly available at https://github.com/smallboy-code/Brain-tumor-segmentation.

Improving Identification of Drug-Target Binding Sites Based on Structures of Targets Using Residual Graph Transformer Network

Improving identification of drug-target binding sites can significantly aid in drug screening and design, thereby accelerating the drug development process. However, due to challenges such as insufficient fusion of multimodal information from targets and imbalanced datasets, enhancing the performance of drug-target binding sites prediction models remains exceptionally difficult. Leveraging structures of targets, we proposed a novel deep learning framework, RGTsite, which employed a Residual Graph Transformer Network to improve the identification of drug-target binding sites. First, a residual 1D convolutional neural network (1D-CNN) and the pre-trained model ProtT5 were employed to extract the local and global sequence features from the target, respectively. These features were then combined with the physicochemical properties of amino acid residues to serve as the vertex features in graph. Next, the edge features were incorporated, and the residual graph transformer network (GTN) was applied to extract the more comprehensive vertex features. Finally, a fully connected network was used to classify whether the vertex was a binding site. Experimental results showed that RGTsite outperformed the existing state-of-the-art methods in key evaluation metrics, such as F1-score (F1) and Matthews Correlation Coefficient (MCC), across multiple benchmark datasets. Additionally, we conducted interpretability analysis for RGTsite through the real-world cases, and the results confirmed that RGTsite can effectively identify drug-target binding sites in practical applications.

Mode recognition in a kerosene-fueled scramjet combustor by a Swin Transformer neural network

Recognizing the combustion mode in scramjet engines is critical for suppressing oscillations and stabilizing the combustion process in hypersonic aircrafts. Current accesses mainly depend on mechanical measurement and dominant frequencies based on image analysis methods, such as proper orthogonal decomposition and dynamic mode decomposition. However, these traditional methods either lack of precision or fall short of the need for prior knowledge, poor generalization, and low efficiency, posing challenges in practical implementations, especially when online controlling is highlighted in the scramjet combustions. Recently, machine learning (ML) has been introduced to the combustion community due to its superiority in high flexibility and efficiency in addressing complex problems. The classical convolutional neural network (CNN) architectures have been reported to achieve efficient combustion mode recognition in furnace combustion, swirling combustor, and rotating detonation engines. However, those CNN-based models are incapable of utilizing the global flame features and the coherences of local areas, resulting in insufficient accuracy and robustness in scramjet combustions with high inflow speed and distinct mode variations. To address this problem, this paper reports a Swin (shifted window) Transformer model, an advanced ML structure outstanding in capturing both global and local features by its self-attention mechanism with high computational efficiency, to identify combustion modes in scramjet engines. The Swin-T was trained and validated in a kerosene-fueled cavity-based scramjet combustor, and results show that it can achieve a considerable accuracy of 95.28%. Comparisons with CNN-based models further indicate that Swin-T outperforms in accuracy, efficiency, and robustness by around 0.7%, 80%, and 3%, respectively.

Disentangled global and local features of multi-source data variational autoencoder: An interpretable model for diagnosing IgAN via multi-source Raman spectral fusion techniques.

A single Raman spectrum reflects limited molecular information. Effective fusion of the Raman spectra of serum and urine source domains helps to obtain richer feature information. However, most of the current studies on immunoglobulin A nephropathy (IgAN) based on Raman spectroscopy are based on small sample data and low signal-to-noise ratio. If a multi-source data fusion strategy is directly adopted, it may even reduce the accuracy of disease diagnosis. To this end, this paper proposes a data enhancement and spectral optimization method based on variational autoencoders to obtain reconstructed Raman spectra with doubled sample size and improved signal-to-noise ratio. In the diagnosis of IgAN in multi-source domain Raman spectra, this paper builds a global and local feature decoupled variational autoencoder (DMSGL-VAE) model based on multi-source data. First, the statistical features after spectral segmentation are extracted, and the latent variables obtained by the variational encoder are decoupled through the decoupling module. The global representation and local representation obtained represent the global shared information and local unique information of the serum and urine source domains, respectively. Then, the cross-source reconstruction loss and decoupling loss are used to constrain the decoupling, and the effectiveness of the decoupling is proved quantitatively and qualitatively. Finally, the features of different source domains were integrated to diagnose IgAN, and the results were analyzed for important features using the SHapley Additive exPlanations algorithm. The experimental results showed that the AUC value of the DMSGL-VAE model for diagnosing IgAN on the test set was as high as 0.9958. The SHAP algorithm was used to further prove that proteins, hydroxybutyrate, and guanine are likely to be common biological fingerprint substances for the diagnosis of IgAN by serum and urine Raman spectroscopy. In summary, the DMSGL-VAE model designed based on Raman spectroscopy in this paper can achieve rapid, non-invasive, and accurate screening of IgAN in terms of classification performance. And interpretable analysis may help doctors further understand IgAN and make more efficient diagnostic measures in the future.

Image classification based on enhanced learning of global and local features with structural priors

Image classification based on enhanced learning of global and local features with structural priors

Developing a Machine Learning-Based Algorithm for Facial Expression Recognition

With the rise of artificial intelligence, machine learning (ML) is increasingly integrated into daily life. Facial expressions, lasting about 1/20th of a second and difficult to conceal, convey nuanced emotions beyond words. They are categorized as macro expressions, displayed under normal circumstances, and micro expressions, fleeting and subconscious. However, recognizing expressions in photos is challenging due to varied backgrounds, appearances, age, and race, impacting accuracy. Addressing this, our research focuses on a machine learning-based facial expression recognition algorithm. Our experimental results, using CASIA and NTERFACE databases, demonstrate that ML outperforms KNN and ANN with recognition rates 41.54% and 26.94% higher, respectively. By adaptively learning global and local features from complete expression images and face parts, our algorithm offers practical and significant advancements in expression recognition.

Tailoring the pressure effects to optimize the global structural features in Ni80P20 Metallic Glasses

Tailoring the pressure effects to optimize the global structural features in Ni80P20 Metallic Glasses

Multiscale Res2RAFT particle image velocimetry algorithm optimized with global attention mechanism

Particle Image Velocimetry (PIV) technology is widely used in experimental fluid mechanics. However, due to the complexity of underwater environments, traditional PIV methods face significant challenges in terms of measurement accuracy and adaptability. With the advancement of computer vision and deep learning technologies, an increasing number of optical flow networks have been applied to PIV tasks, yet their performance in fine-grained displacement field prediction remains limited. To address this issue, this paper proposes a new optical flow network improvement model, Res2RAFT. By introducing the Bottle2neck module from the Res2Net network, we constructed the Res2BasicEncoder to enhance multiscale feature extraction capabilities. Additionally, a Global Attention Module was incorporated into the model to improve the prediction accuracy of complex flow fields by enhancing global dimension interaction features. Experimental results show that Res2RAFT achieves average endpoint errors of 2.0758, 8.0548, and 11.7254 on the Cylinder, Direct Numerical Simulation turbulence, and Surface Quasi-Geostrophic categories, respectively, representing improvements of 17.4%, 30.3%, and 27.3% compared to the RAFT-PIV model. Overall, the model achieved an accuracy improvement exceeding 20% in complex flow fields, providing new possibilities for the application of PIV technology in more complex fluid scenarios.

Signed graph embedding via multi-order neighborhood feature fusion and contrastive learning.

Signed graphs have been widely applied to model real-world complex networks with positive and negative links, and signed graph embedding has become a popular topic in the field of signed graph analysis. Although various signed graph embedding methods have been proposed, most of them still suffer from the generality problem. Namely, they cannot simultaneously achieve the satisfactory performance in multiple downstream tasks. In view of this, in this paper we propose a signed embedding method named MOSGCN which exhibits two significant characteristics. Firstly, MOSGCN designs a multi-order neighborhood feature fusion strategy based on the structural balance theory, enabling it to adaptively capture local and global structure features for more informative node representations. Secondly, MOSGCN is trained by using the signed graph contrastive learning framework, which further helps it learn more discriminative and robust node representations, leading to the better generality. We select link sign prediction and community detection as the downstream tasks, and conduct extensive experiments to test the effectiveness of MOSGCN on four benchmark datasets. The results illustrate the good generality of MOSGCN and the superiority by comparing to state-of-the-art methods.

Global Context Volume construction and semantics-guided disparity refinement for stereo matching

The accuracy of stereo matching has been greatly improved with the advent of convolutional neural networks. However, existing methods still perform poorly in regions of low-texture and reflections due to insufficient matching information. In this paper, we propose a novel semantic stereo matching network named GSSNet, which combines global context information with high-level semantic clues to further enrich the matching features. Specifically, we employ Swin Transformer as a joint feature extractor to capture multi-scale global features for both semantic segmentation and disparity estimation. By fusing low-resolution multi-scale feature maps progressively, we construct a Global Context Volume (GCV) representation covering long-range receptive field for initial disparity estimation. Initial disparities are further refined with a high-level embedding learned from the semantic segmentation branch of the network through residual learning. During training, due to the lack of joint annotations, we also propose a simple and effective pseudo-labeling strategy. Comprehensive experimental results demonstrated on various datasets manifest the effectiveness of the proposed GSSNet. Among all published methods as of 9, February 2024, our approach ranks 1st on KITTI 2012 leaderboard and 3rd on KITTI 2015 leaderboard, and produces competitive results on Scene Flow and Cityscapes datasets. Code will be available at https://github.com/Twil-7/GSSNet.

SAGPNet: A shape-aware and adaptive strip self-attention guided progressive network for SAR marine oil spill detection.

The oil spill is a significant source of marine pollution, causing severe harm to marine ecosystems. Detecting oil spills accurately using synthetic aperture radar (SAR) images is crucial for protecting the environment. However, oil spill targets in SAR images are small and resemble other objects "look-alike". Traditional semantic segmentation networks for MOSD may lose critical information during downsampling Hence, we propose a shape-aware and adaptive strip self-attention guided progressive network (SAGPNet) for MOSD. Firstly, we adopted the progressive strategy to reduce detailed information loss. Second, we improved the traditional U-Net by redesigning its encoder unit. Specifically, we proposed a shape-aware and multi-scale feature extraction module and an adaptive strip self-attention module (ASSAM). These modifications allow the model to extract shape, multi-scale, and global information during the encoding process, addressing the challenges posed by small targets and "look-alike". Third, we utilize the ASSAM to extract global features from the final encoding layer of the earlier stage of the progressive network to guide the encoding features of the subsequent stage, aiming to recognize the overall shape of the oil spill and ensure that the model preserves crucial contextual information, further mitigate the information loss caused by downsampling. Finally, we designed a joint loss to address pixel imbalance between oil spills and other targets. We use three public oil spill detection datasets to evaluate the performance of SAGPNet. The experimental results show superior performance compared to other state-of-the-art methods, highlighting the effectiveness of SAGPNet in addressing the challenges associated with MOSD.