Convolutional Feature Extraction Research Articles

Hybrid convolutional network with enhanced graph attention mechanism (HCN-EGAM) for hyperspectral image classification

ABSTRACT Due to their reliance on superficial spectral or spatial information, current hyperspectral image classification techniques frequently fail to generate similarity measures that are accurate enough to create graph structures. We have developed a new end-to-end model, the hybrid convolutional network with enhanced graph attention mechanism (HCN-EGAM), to address this issue. It integrates an enhanced graph attention mechanism with 3D–2D hybrid convolution. In order to improve classification accuracy, this model synergistically uses the graph attention network module with the hybrid convolutional feature extraction module. The model starts by creating a 3D–2D hybrid convolutional network. This lets it quickly pull out deep spatial-spectral features that make different ground objects stand out in hyperspectral images. The model then employs these deep spatial-spectral properties to construct a graph structure, thereby enhancing its feature representation capabilities. Lastly, we use the Enhanced Graph Attention Mechanism (EGAM) to learn long-range spatial associations, successfully distinguishing between intra-class differences and inter-class similarities between distinct samples. Comparing our suggested method to other state-of-the-art methods, experimental assessments on three datasets, including the University of Pavia, Kennedy Space Center (KSC) and Indian Pines (IP) datasets, show a considerable improvement in classification accuracy.

An Adaptive Method for Predicting Bearing Remaining Useful Life Across Various Degradation Stages

Abstract Bearing degradation is a multi-stage, multi-trend and highly complex process, significant information discrepancies and extreme imbalances exist in degradation data across different stages. These complexities hinder the accuracy of predictive model in predicting the remaining useful life throughout all stages of the bearing's degradation. In this paper, a novel prediction model based on Adaptive Convolutional Neural Network (ACNN)-Multiple Kernel Convolutional Long Short-Term Memory (MKConvLSTM) is proposed, which utilizes adaptive feature extraction and multi-scale dynamic selection to solve the problem of multi-stage, multi trend and highly complex information in bearing degradation. First, the ACNN is used to perform convolutional feature extraction and adaptive mapping on input samples, effectively distinguishing the degradation stages Then, the MKConvLSTM generates features at different time scales and dynamically selects these features to capture temporal information during the degradation process, enriching the model's capability to represent complex information and improving its predictive performance. To validate the effectiveness of the proposed model, experiments were conducted on the PHM2012 datasets and XJTU datasets. The MAE and RMSE of ACNN-MKConvLSTM reaches 0.078 and 0.099 on the first dataset, 0.086 and 0.107 on the second dataset, respectively. approximately 23% and 17% improvement Compared to the baseline model, respectively. Experimental results demonstrate that the model exhibits high accuracy and robustness in bearing remaining useful life prediction, effectively addressing the impact of feature variations across different degradation stages on prediction performance.

SODD-YOLOv8: an insulator defect detection algorithm based on feature enhancement and variable row convolution

Abstract To address the detection challenges in defective images of insulators in transmission lines, including tiny object size, significant scale variations, a wide variety of defects, and complex background interference. In this study, an improved insulator defect detection algorithm is proposed, based on the YOLOv8s framework and combining feature enhancement and deformable convolution techniques. Firstly, to address the image feature distortion problem caused by aerial photography, a deformable convolutional feature extraction module (DCFEM) is introduced, which is designed to enhance the model’s ability to adapt to the local geometric deformation, so as to effectively recover the distorted feature information in the image. Moreover, to enhance the detection ability of the model for small objects, a small object feature enhancement module is designed, which adopts an efficient multi-scale attention mechanism, and aims to enhance the feature extraction ability of small objects, improve the sensitivity to small-size defects, and improve the detection accuracy. Eventually, to optimize the computational efficiency of the model, the average pooling-sparse convolution-batch normalization (BN) module is proposed. This module combines average pooling, sparse convolution and BN techniques to achieve a lightweight model while maintaining a high level of feature extraction capability. Experimental results on the China power line insulator dataset show that the improved model achieves a 4.3 percentage point improvement in the mAP metric compared to YOLOv8s, and the number of parameters in the model is reduced by 10%. The proposed scheme not only improves the accuracy and efficiency of defect detection, but also reduces the demand for computational resources, thus providing a more reliable and efficient solution for insulator defect detection in practical applications.

Improved YOLOv7 Electric Work Safety Belt Hook Suspension State Recognition Algorithm Based on Decoupled Head

Safety is the eternal theme of power systems. In view of problems such as time-consuming and poor real-time performance in the correct use of seat belt hooks by manual supervision operators in the process of power operation, this paper proposes an improved YOLOv7 seat belt hook suspension state recognition algorithm. Firstly, the feature extraction part of the YOLOv7 backbone network is improved, and the M-Spatial Pyramid Pooling Concurrent Spatial Pyramid Convolution (M-SPPCSPC) feature extraction module is constructed to replace the Spatial Pyramid Pooling Concurrent Spatial Pyramid Convolution (SPPCSPC) module of the backbone network, which reduces the amount of computation and improves the detection speed of the backbone network while keeping the sensory field of the backbone network unchanged. Second, a decoupled head, which realizes the confidence and regression frames separately, is introduced to alleviate the negative impact of the conflict between the classification and regression tasks, consequently improving the network detection accuracy and accelerating the network convergence. Ultimately, a dynamic non-monotonic focusing mechanism is introduced in the output layer, and the Wise Intersection over Union (WioU) loss function is used to reduce the competitiveness of high-quality anchor frames while reducing the harmful gradient generated by low-quality anchor frames, which ultimately improves the overall performance of the detection network. The experimental results show that the mean Average Precision (mAP@0.5) value of the improved network reaches 81.2%, which is 7.4% higher than that of the original YOLOv7, therefore achieving better detection results for multiple-state recognition of hooks.

Domain Transfer-Based Hypergraph Convolutional Network for Posture Anomaly Detection in Physical Education Teaching

Human skeleton-based posture anomaly detection has been widely applied in the field of physical education teaching. The existing spatio-temporal graph convolutional networks (ST-GCN) can fully utilize the local and global information of the human skeleton for action recognition, but the entire model requires a large amount of computation and the modeling of high-order relationships between joint points of the human skeleton is insufficient. To this end, this paper proposes a novel domain adaptive hypergraph convolutional network for basketball posture anomaly analysis by exploiting 2D skeleton information. First, we designed an effective hypergraph convolution feature extraction network to improve the high-order dependency modeling. To further improve the performance of the hypergraph convolutional network, we introduce domain adaptive learning technology to supervise the feature extraction learning of the target domain (2D skeleton) through the source domain (3D skeleton). At last, we construct a basketball action teaching analysis dataset for model evaluation. We conducted a large number of comparative experiments on the public dataset NTU RGB+D and our self-built dataset. All the results showed that our proposed hypergraph convolutional model effectively extracts features of 2D human skeletons, and by introducing domain adaptive learning, the performance of basketball anomaly detection is further improved.

Semi-Supervised Subcategory Centroid Alignment-Based Scene Classification for High-Resolution Remote Sensing Images

It is usually hard to obtain adequate annotated data for delivering satisfactory scene classification results. Semi-supervised scene classification approaches can transfer the knowledge learned from previously annotated data to remote sensing images with scarce samples for satisfactory classification results. However, due to the differences between sensors, environments, seasons, and geographical locations, cross-domain remote sensing images exhibit feature distribution deviations. Therefore, semi-supervised scene classification methods may not achieve satisfactory classification accuracy. To address this problem, a novel semi-supervised subcategory centroid alignment (SSCA)-based scene classification approach is proposed. The SSCA framework is made up of two components, namely the rotation-robust convolutional feature extractor (RCFE) and the neighbor-based subcategory centroid alignment (NSCA). The RCFE aims to suppress the impact of rotation changes on remote sensing image representation, while the NSCA aims to decrease the impact of intra-category variety across domains on cross-domain scene classification. The SSCA algorithm and several competitive approaches are validated on two datasets to demonstrate its effectiveness. The results prove that the proposed SSCA approach performs better than most competitive approaches by no less than 2% overall accuracy.

SSG-Net: A Multi-Branch Fault Diagnosis Method for Scroll Compressors Using Swin Transformer Sliding Window, Shallow ResNet, and Global Attention Mechanism (GAM).

The reliable operation of scroll compressors is crucial for the efficiency of rotating machinery and refrigeration systems. To address the need for efficient and accurate fault diagnosis in scroll compressor technology under varying operating states, diverse failure modes, and different operating conditions, a multi-branch convolutional neural network fault diagnosis method (SSG-Net) has been developed. This method is based on the Swin Transformer, the Global Attention Mechanism (GAM), and the ResNet architecture. Initially, the one-dimensional time-series signal is converted into a two-dimensional image using the Short-Time Fourier Transform, thereby enriching the feature set for deep learning analysis. Subsequently, the method integrates the window attention mechanism of the Swin Transformer, the 2D convolution of GAM attention, and the shallow ResNet's two-dimensional convolution feature extraction branch network. This integration further optimizes the feature extraction process, enhancing the accuracy of fault feature recognition and sensitivity to data variability. Consequently, by combining the global and local features extracted from these three branch networks, the model significantly improves feature representation capability and robustness. Finally, experimental results on scroll compressor datasets and the CWRU dataset demonstrate diagnostic accuracies of 97.44% and 99.78%, respectively. These results surpass existing comparative models and confirm the model's superior recognition precision and rapid convergence capabilities in complex fault environments.

Learning Unsupervised Cross-Domain Model for TIR Target Tracking

The limited availability of thermal infrared (TIR) training samples leads to suboptimal target representation by convolutional feature extraction networks, which adversely impacts the accuracy of TIR target tracking methods. To address this issue, we propose an unsupervised cross-domain model (UCDT) for TIR tracking. Our approach leverages labeled training samples from the RGB domain (source domain) to train a general feature extraction network. We then employ a cross-domain model to adapt this network for effective target feature extraction in the TIR domain (target domain). This cross-domain strategy addresses the challenge of limited TIR training samples effectively. Additionally, we utilize an unsupervised learning technique to generate pseudo-labels for unlabeled training samples in the source domain, which helps overcome the limitations imposed by the scarcity of annotated training data. Extensive experiments demonstrate that our UCDT tracking method outperforms existing tracking approaches on the PTB-TIR and LSOTB-TIR benchmarks.

Enhancing local representation learning through global–local integration with functional connectivity for EEG-based emotion recognition

Emotion recognition based on electroencephalogram (EEG) signals is crucial in understanding human affective states. Current research has limitations in extracting local features. The representation capabilities of local features are limited, making it difficult to comprehensively capture emotional information. In this study, a novel approach is proposed to enhance local representation learning through global–local integration with functional connectivity for EEG-based emotion recognition. By leveraging the functional connectivity of brain regions, EEG signals are divided into global embeddings that represent comprehensive brain connectivity patterns throughout the entire process and local embeddings that reflect dynamic interactions within specific brain functional networks at particular moments. Firstly, a convolutional feature extraction branch based on the residual network is designed to extract local features from the global embedding. To further improve the representation ability and accuracy of local features, a multidimensional collaborative attention (MCA) module is introduced. Secondly, the local features and patch embedded local embeddings are integrated into the feature coupling module (FCM), which utilizes hierarchical connections and enhanced cross-attention to couple region-level features, thereby enhancing local representation learning. Experimental results on three public datasets show that compared with other methods, this method improves accuracy by 4.92% on the DEAP, by 1.11% on the SEED, and by 7.76% on the SEED-IV, demonstrating its superior performance in emotion recognition tasks.

ASD-YOLO: An aircraft surface defects detection method using deformable convolution and attention mechanism

Aircraft surface defect (ASD) detection is crucial for ensuring flight safety. Addressing challenges such as large-scale variations, irregular shapes, and sample imbalance in ASD detection, this paper proposes a ASD-YOLO network based on YOLOv5. ASD-YOLO incorporates several enhancements to improve recognition capabilities. Firstly, a new deformable convolutional feature extraction module (DCNC3) is designed to better learn defects of different shapes, which is combined with a global attention mechanism (GAM) to pay more attention to defects region information. Secondly, the feature representation of small defects is bolstered by the contextual enhancement module (CEM). Lastly, to alleviate sample imbalance problem, we introduce an exponential sliding average weight function (EMA-Slide). Experimental results on two datasets show improvements in mean Average Precision (mAP) by 5.7% and 3.4%, respectively, surpassing mainstream algorithms and offering a novel approach to ASD detection.

Profit Estimation Model and Financial Risk Prediction Combining Multi-scale Convolutional Feature Extractor and BGRU Model

Profit Estimation Model and Financial Risk Prediction Combining Multi-scale Convolutional Feature Extractor and BGRU Model

Implementation of CNN of Mobile-based COVID-19 Chest X-Ray Images

The COVID-19 pandemic outbreak is the most significant event from 2019 until 2021. A medical examination of radiological images is carried out to check the condition of the patient's lungs. The limitations of this examination need alternative computer-assisted applications for patient CXR. This research aims to implement a back-end and front-end-based Convolutional Neural Network (CNN) model. Its advantage is that it can detect CXR images in real-time and non-real-time using multi-classification, namely normal, pneumonia, and COVID-19. The CNN model carries out the process of convolutional feature extraction and multi-layer perceptron classification at the back-end stage. In contrast, it uses an Android mobile-based application at the front-end stage. The research results show that the non-real-time condition has an accuracy of 98%, while the real-time is 95% lower. This research produces model and application performance that is flexible for user needs. The results can be recommended for developing applications for more comprehensive users.

LBRT: Local-Information-Refined Transformer for Image Copy-Move Forgery Detection.

The current deep learning methods for copy-move forgery detection (CMFD) are mostly based on deep convolutional neural networks, which frequently discard a large amount of detail information throughout convolutional feature extraction and have poor long-range information extraction capabilities. The Transformer structure is adept at modeling global context information, but the patch-wise self-attention calculation still neglects the extraction of details in local regions that have been tampered with. A local-information-refined dual-branch network, LBRT (Local Branch Refinement Transformer), is designed in this study. It performs Transformer encoding on the global patches segmented from the image and local patches re-segmented from the global patches using a global modeling branch and a local refinement branch, respectively. The self-attention features from both branches are precisely fused, and the fused feature map is then up-sampled and decoded. Therefore, LBRT considers both global semantic information modeling and local detail information refinement. The experimental results show that LBRT outperforms several state-of-the-art CMFD methods on the USCISI dataset, CASIA CMFD dataset, and DEFACTO CMFD dataset.

Monocular depth estimation using multi-dimensional dynamic convolution

To address the issue of inaccurate prediction results and low accuracy at the object edges in existing self-supervised monocular depth estimation algorithms, this study proposes a self-supervised monocular depth estimation algorithm based on dynamic convolution (OE-Depth). The algorithm employs a multi-dimensional dynamic convolution feature extraction network to acquire more comprehensive feature representations, thereby enhancing the predictive capability. Additionally, the algorithm is optimized by integrating a triplet loss term and employing metric learning techniques to refine the networks performance at the object edges. Experimental evaluations conducted on the KITTI dataset validate the effectiveness of the proposed algorithm, demonstrating an optimization of the error class index by nearly 10%. Notably, the most stringent criterion < 1.25 achieves an accuracy of 0.908 for depth prediction.

Multi kernel cross sparse graph attention convolutional neural network for brain magnetic resonance imaging super-resolution

High-resolution Magnetic Resonance Imaging (MRI) is pivotal in both diagnosing and treating brain tumors, assisting physicians in diagnosis and treatment by displaying anatomical structures. Utilizing convolutional neural network-based super-resolution methods enables the efficient acquisition of high-resolution MRI images.However, Convolutional neural networks are limited by their kernel size, which restricts their ability to capture a wider field of view, potentially leading to feature omission and difficulties in establishing global and local feature relationships. To overcome these shortcomings, We have designed a novel network architecture that highlights three main modules: (i) Multiple Convolutional Feature (MCF)extraction module, which diversifies convolution operations for extracting image features, achieving comprehensive feature representation. (ii) Multiple Groupss of Cross-Iterative Feature(MGCIF) modules, promoting inter-channel feature interactions and emphasizing crucial features needed for subsequent learning. (iii) A Graph Neural Network Module based on a sparse attention mechanism, capable of connecting distant pixel features and identifying influential neighboring pixels for target pixel inpainting. To evaluate the accuracy of our proposed network, we conducted tests on four datasets, comprising two sets of brain tumor data and two sets of healthy head MRI data, all of which underwent varying degrees of degradation. We conducted experiments using nineteen super-resolution (SR) models. Our experiments were carried out on four datasets, and the results demonstrate that our method outperforms the current leading-edge methods. In the four datasets, our model showed improvements in Peak Signal-to-Noise Ratio (PSNR) scores compared to the second-place model,with increase of 1.16 %,1.08 %,0.19 % and 0.53 % for ×2,and 2.26 %,1.67 %,0.13 % and 0.45 % for × 4,respectively.

Knowledge Inference Combining Convolutional Feature Extraction and Path Semantics Integration

Many knowledge representation models extract local patterns or semantic features using fact embeddings but often overlook path semantics. There is room for improvement in pathbased approaches that rely solely on single paths. A customized convolutional neural network (CNN) architecture is proposed to encode multiple paths generated by random walks into vector sequences. For each path, the feature sequence is then merged into a single vector using bidirectional long short-term memory (LSTM) by concatenating both forward and backward hidden states. Semantic relevance between different paths and candidate relations is computed using the attention mechanism. The state vectors of the relations are calculated using weighted paths. These paths help determine the probabilities of the candidate relations, which are then used to assess the validity of the triples. Link prediction experiments on two benchmark datasets, NELL995 and FB15k-237, demonstrate the advantages of our solution. Our model shows a 7.19% improvement at Hits@3 on FB15k-237 compared to Att-Model + Type, another advanced model. The model is further applied to a large complex dataset, FC17, as well as a sparse dataset, NELL-One, for few-shot reasoning.

An acoustic imaging recognition based cutting tools wear state prediction method

Online monitoring of cutting tools wear is an important component of advanced manufacturing technology, which can greatly improve the processing efficiency and reduce the production cost. In this paper, a cutting tools wear state prediction method based on acoustic imaging recognition is developed. By applying the advantages of the functional generalized inverse beamforming method in the sound field reconstruction, the acoustic signal is used as the carrier to reconstruct the three-dimensional space radiated sound field. And then, slice the reconstructed sound field image and input it into the convolutional neural network model as a sample, to process and classify the image and mines the feature information related to state from the sound field image. By incorporating amplitude and phase information of the sound field, the presented method utilizes spatial domain mapping to accurately identify the noise source and address challenges such as low recognition rate and difficult diagnosis under weak fault conditions. Furthermore, the paper also demonstrates the recognition of sound field states through a fault experiment in sound box simulation, based on these theories. And the recognition of sound field states is achieved through a simulation fault experiment conducted on the sound box, thereby validating the feasibility of the state monitoring method based on pattern recognition of sound and image. Finally, the experimental object is selected as the four-edge carbide milling cutter, and the cutting tools wear state is monitored by integrating sound field reconstruction techniques with convolution feature extraction methods to validate the robustness of the proposed approach.

RLPeri: Accelerating Visual Perimetry Test with Reinforcement Learning and Convolutional Feature Extraction

Visual perimetry is an important eye examination that helps detect vision problems caused by ocular or neurological conditions. During the test, a patient's gaze is fixed at a specific location while light stimuli of varying intensities are presented in central and peripheral vision. Based on the patient's responses to the stimuli, the visual field mapping and sensitivity are determined. However, maintaining high levels of concentration throughout the test can be challenging for patients, leading to increased examination times and decreased accuracy. In this work, we present RLPeri, a reinforcement learning-based approach to optimize visual perimetry testing. By determining the optimal sequence of locations and initial stimulus values, we aim to reduce the examination time without compromising accuracy. Additionally, we incorporate reward shaping techniques to further improve the testing performance. To monitor the patient's responses over time during testing, we represent the test's state as a pair of 3D matrices. We apply two different convolutional kernels to extract spatial features across locations as well as features across different stimulus values for each location. Through experiments, we demonstrate that our approach results in a 10-20% reduction in examination time while maintaining the accuracy as compared to state-of-the-art methods. With the presented approach, we aim to make visual perimetry testing more efficient and patient-friendly, while still providing accurate results.

Abstract 7379: Deep learning based risk stratification of pre-operative CT scans is prognostic of overall survival in kidney cancers

Abstract Introduction: Preoperative risk stratification of kidney cancer patients is critical for treatment selection and improving mortality rates. Current clinical approaches such as cancer grading are limited by inter-reviewer variability, require invasive biopsy procedures, and might not capture the full extent of patient risk. We sought to evaluate a deep learning (DL) model to quantify patient risk and prognosticate overall survival in kidney cancer patients via CT scans. Methods: This retrospective study used preoperative contrast CT scans from 4 public multi-institutional collections with date of death/censoring available. A deep learning (DL) network architecture was developed to analyze input CT scans via a convolutional feature extractor and a Cox proportional hazards-based cost function. The DL model assigned a risk score to each patient, with the median risk used as a cutoff for classifying patients as high-risk or low-risk. Model performance for prognosticating overall survival (OS) was evaluated separately in training, internal validation, and external validation cohorts via Kaplan-Meier analysis in terms of concordance index and hazard ratios, with risk groups compared via Wilcoxon log-rank tests (α = 0.05). Results: In the training/internal validation cohort (N=418, 8 institutions) and external validation cohort (N=102, 1 institution), the median age was 60 and 59 years, the male/female ratios were 62%/38% and 75%/25%, with a median follow up of 2.8 and 3.1 years, respectively. The DL model yielded a training c-index of 0.66 (95% CI: 0.63-0.68), internal validation c-index of 0.73 (95% CI: 0.70-0.76) and an external validation c-index of 0.67 (95% CI: 0.64-0.70). Kaplan-Meier analysis of associated survival curves based on DL-determined high- and low-risk patients demonstrated statistically significant separation in all cohorts: training (p = 0.003, HR = 8.63, 95% CI: 8.28-8.98), internal validation (p = 0.027, HR = 4.89, 95% CI: 4.52-5.26), and external validation (p = 0.01, HR = 6.54, 95% CI: 6.18-6.90). Baseline clinicopathological characteristics were not significantly different between DL high and low risk patients in external validation (chi-square testing; ISUP grade p=0.375, age p=0.824, race p=0.567, and sex p=0.440). ISUP grade was also not a significant predictor of OS in internal validation (HR=1.57, p=0.211, 95% CI: 1.28-1.85) or external validation (p = 0.465, HR=0.16, 95% CI: 0.33-0.74). Conclusion: A deep learning model based on pre-operative CT scans was found to accurately prognosticate overall survival in kidney cancers in a multi-institutional setting, independent of clinicopathological factors. This model is being validated on larger cohorts, and in the context of targeted therapies. Citation Format: Brennan Flannery, Mohsen Hariri, Thomas DeSilvio, Amir Sadri, Jane Nguyen, Erick M. Remer, Smitha Krishnamurthi, Satish E. Viswanath. Deep learning based risk stratification of pre-operative CT scans is prognostic of overall survival in kidney cancers [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 7379.

Hyperspectral Image Classification Based on 3D–2D Hybrid Convolution and Graph Attention Mechanism

Convolutional neural networks and graph convolutional neural networks are two classical deep learning models that have been widely used in hyperspectral image classification tasks with remarkable achievements. However, hyperspectral image classification models based on graph convolutional neural networks using only shallow spectral or spatial features are insufficient to provide reliable similarity measures for constructing graph structures, limiting their classification performance. To address this problem, we propose a new end-to-end hyperspectral image classification model combining 3D–2D hybrid convolution and a graph attention mechanism (3D–2D-GAT). The model utilizes the collaborative work of hybrid convolutional feature extraction module and GAT module to improve classification accuracy. First, a 3D–2D hybrid convolutional network is constructed and used to quickly extract the discriminant deep spatial-spectral features of various ground objects in hyperspectral image. Then, the graph is built based on deep spatial-spectral features to enhance the feature representation ability. Finally, a network of graph attention mechanism is adopted to learn long-range spatial relationship and distinguish the intra-class variation and inter-class similarity among different samples. The experimental results on three datasets, Indian Pine, the University of Pavia and Salinas Valley show that the proposed method can achieve higher classification accuracy compared with other advanced methods.