Local Feature Extraction Research Articles

DSG-GAN:A dual-stage-generator-based GAN for cross-modality synthesis from PET to CT

PET/CT devices typically use CT images for PET attenuation correction, leading to additional radiation exposure. Alternatively, in a standalone PET imaging system, attenuation and scatter correction cannot be performed due to the absence of CT images. Therefore, it is necessary to explore methods for generating pseudo-CT images from PET images. However, traditional PET-to-CT synthesis models encounter conflicts in multi-objective optimization, leading to disparities between synthetic and real images in overall structure and texture. To address this issue, we propose a staged image generation model. Firstly, we construct a dual-stage generator, which synthesizes the overall structure and texture details of images by decomposing optimization objectives and employing multiple loss functions constraints. Additionally, in each generator, we employ improved deep perceptual skip connections, which utilize cross-layer information interaction and deep perceptual selection to effectively and selectively leverage multi-level deep information and avoid interference from redundant information. Finally, we construct a context-aware local discriminator, which integrates context information and extracts local features to generate fine local details of images and reasonably maintain the overall coherence of the images. Experimental results demonstrate that our approach outperforms other methods, with SSIM, PSNR, and FID metrics reaching 0.8993, 29.6108, and 29.7489, respectively, achieving the state-of-the-art. Furthermore, we conduct visual experiments on the synthesized pseudo-CT images in terms of image structure and texture. The results indicate that the pseudo-CT images synthesized in this study are more similar to real CT images, providing accurate structure information for clinical disease analysis and lesion localization.

Industrial defect detection and location based on greedy membrane clustering algorithm

This paper introduces a related model of membrane calculation in the defect detection and positioning of industrial components. It has the characteristics of distributed and parallel computing, and can efficiently search for better solutions in a given feature space. Inspired by the membrane clustering algorithm, this paper proposes a greedy membrane clustering algorithm and names it GMCA. GMCA is applied after the extraction of local features of normal samples. It uses a greedy strategy to construct a sub-feature set that describes the local characteristics of normal samples. During training, GMCA can learn the membrane cluster center of normal image blocks and each sub-feature within the cluster. At test time, the anomaly map is obtained by calculating the distance from the test sample block to the corresponding cluster center and the maximum distance from the cluster center to the nearest neighbor in the training sample. This solves the limitation of traditional algorithms requiring dataset alignment. In the unsupervised dataset MvTec AD, samples can be divided into object categories and texture categories according to the background of images. The pixel-level anomaly location index (AUROC) of this method on object category data reaches 98.3%. The image-level anomaly detection index (AUROC) on texture category data reaches 99.1%.

Global–local feature learning for fine-grained food classification based on Swin Transformer

Separable object parts, such as the head and tail in a bird, are vital for fine-grained visual classifications. For those objects without separable parts, the classification task relies only on local and global textural image features. Although the Swin Transformer architecture was proposed to efficiently capture both local and global visual features, it still exhibits a bias towards global features. Therefore, our goal is to enhance the local feature learning capability of the Swin Transformer by adding four new modules of the Local Feature Extraction Network (L-FEN), Convolution Patch-Merging (CP), Multi-Path (MP), and Multi-View (MV). The L-FEN enhances the Swin transformer with the improved local feature capture. The CP is a localized and hierarchical adaptation of the Swin’s Patch Merging technique. The MP method integrates features across various Swin stages to accentuate local details. Meanwhile, the MV Swin transformer block supersedes traditional Swin blocks with those incorporating varied receptive fields, ensuring a broader scope of local feature capture. Our enhanced architecture, named Global–Local Swin Transformer (GL-Swin), is applied to solve a fine-grained food classification task. On three major food datasets: ISIA Food-500 UEC Food-256, and Food-101, our GL-Swin achieved accuracies of 66.75%, 85.78%, and 92.93% respectively, consistently outperforming other leading methods.

Semantics-enhanced discriminative descriptor learning for LiDAR-based place recognition

LiDAR-based place recognition (LPR) aims to localize autonomous vehicles and mobile robots relative to pre-built maps or retrieve previously visited places. However, the complexity of real-world scenes and changes in viewpoint are significant challenges for place recognition. As high-level information, semantics makes it easier to distinguish geometrically similar scene situations. Unlike most existing methods that rely solely on a single type of information (geometric or semantic) to construct scene descriptors, we consider the complementary nature of the semantic and geometric information and propose a semantics-enhanced discriminative feature learning method for LPR. Specifically, we first develop a Multi-layer Fusion Feature Extraction Network (MFFEN) based on the transformer encoder to hierarchically fuse local geometric and semantic information and utilize contextual information for extracting discriminative local features. To obtain semantic information, we introduce the dynamic graph convolution network to extract local semantic features with local relations. In addition, to weaken the interference of redundancy and dynamic objects in the scene, we design a semantics-guided local attention network (SLAN) to focus on salient local features that are helpful for recognizing scenes, thereby enhancing the descriptive ability of the global descriptor. Extensive experiments on public datasets KITTI and KITTI-360 demonstrate that the proposed method performs better than recent LiDAR-based methods on the 3D place recognition task. For instance, it achieves the mean F1max score of 96.9% on the KITTI dataset, surpassing the strongest prior model by 2.7%.

FE-FAIR: Feature-Enhanced Fused Attention for Image Super-Resolution

Transformers have performed better than traditional convolutional neural networks (CNNs) for image super-resolution (SR) reconstruction in recent years. Currently, shifted window multi-head self-attention based on the swin transformer is a typical method. Specifically, the multi-head self-attention is used to extract local features in each window, and then a shifted window strategy is used to discover information interaction between different windows. However, this information interaction method needs to be more efficient and include some global feature information, which limits the model’s performance to a certain extent. Furthermore, optimizing the utilization of shallow features, which exhibit significant energy reserves and invaluable low-frequency information, is critical for advancing the efficacy of super-resolution techniques. In order to solve the above issues, we propose the feature-enhanced fused attention (FE-FAIR) method for image super-resolution. Specifically, we design the multi-scale feature extraction module (MSFE) as a shallow feature extraction layer to extract rich low-frequency information from different scales. In addition, we propose the fused attention block (FAB), which introduces channel attention in the form of residual connection based on shifted window self-attention, effectively achieving the fusion of global and local features. Simultaneously, we also discuss other methods to enhance the performance of the FE-FAIR method, such as optimizing the loss function, increasing the window size, and using pre-training strategies. Compared with state-of-the-art SR methods, our proposed method demonstrates better performance. For instance, FE-FAIR outperforms SwinIR by over 0.9 dB when evaluated on the Urban100 (×4) dataset.

Multi-task prediction of hot rolled steel mechanical properties based on CA-MMCE

The accurate prediction of mechanical properties of multi-task hot rolling steel is of great significance to the development of manufacturing, construction, and other industries, and can effectively improve the efficiency of continuous quality control of the hot rolling process. Aiming at the problem that the existing deep models lack interpretability in the field of steel rolling, the hot rolling process was deeply analyzed, and the temperature fluctuation was integrated into the input features as a process variable through feature augmentation to obtain the correlation between features. In order to solve the problem that the conventional Attention mechanism only focuses on the overall global information, a Convolutional Attention mechanism (CA) is proposed to focus on local feature information and improve the ability of the network to extract local key features. At the same time, so as to solve the negative transfer problem of the multi-task model, a Multi-gate Mixture of Convolutional Experts (MMCE) is proposed, which uses multi-gate to adjust the influence of each convolutional expert network on different tasks to capture the correlation between tasks. Thus, it can ensure that multiple tasks can be accurately predicted. Experimental results show that, compared with the existing models, the proposed model achieves excellent prediction performance in multi-task prediction and is advanced.

A lightweight multi-feature fusion vision transformer bearing fault diagnosis method with strong local sensing ability in complex environments

Fault diagnosis of rolling bearings in complex environments is a difficult problem. First, the median filter can remove the noise in the vibration signals, however, it cannot adaptively adjust the filter weights according to the input signals. Second, the popular vision transformer (ViT) cannot extract local feature information under complex conditions and has a large number of parameters, which result in increased computational complexity. To solve these problems, a lightweight multi-feature fusion ViT bearing fault diagnosis method with strong local awareness in complex environments is proposed. Firstly, to learn the features and statistical distributions of the input signals, the gradient descent method is used to continuously and iteratively update the weights and filter the signals. Then, to better extract critical local fault information, a local sensing module is constructed using multi-scale wide convolutional neural network. Finally, an improved lightweight multi-feature fusion ViT is constructed to perform global feature extraction and fault identification. The results show that the proposed method has better noise reduction effect and feature extraction ability, and can accurately identify the fault types under the complex environments.

EEG Emotion Recognition Based on Self-Distillation Convolutional Graph Attention Network

A convolution graph attention model based on self-distillation convolutional graph attention network (SDC-GAT) is proposed for multi-channel EEG emotion recognition. Firstly, two-dimensional feature matrix based on EEG time-domain features are constructed, and the matrix is fed into the graph attention neural network to learn the internal connections between electrical brain channels located in different brain regions. Meanwhile, the three-dimensional feature matrix is constructed according to the relative positions of the electrode channels, and the self-distillation network is employed to extract local high-level abstract features containing electrode spatial position information from the three-dimensional feature matrix. Finally, outputs of the two networks are integrated to determine the emotional states. Experiments were performed on the DEAP dataset. The experimental results show that the spatial domain information of the electrode channel and the internal connection relationship between different channels are beneficial for emotion recognition. In addition, the proposed model can effectively fuse these information to improve the performance of multi-channel EEG emotion recognition.

Human Selective Matting

Existing human matting methods are incapable of accurately estimating the alpha mattes of arbitrarily selected humans from a group photo. An alternative solution is to apply them to the corresponding cropped image patches. However, this option obtains an inaccurate alpha estimation due to the interference of the body parts of the neighboring humans. In addition, these methods are only trained on finely annotated synthetic data, which causes poor performance in real-world scenarios due to the domain shift. To address these problems, we propose human selective matting (HSMatt), which performs matting for arbitrarily selected humans from a group photo given only a simple bounding box as guidance. Specifically, we design a global–local context network to extract both local and global semantic context features. A human-aware trimap network is then proposed to generate human-aware trimaps for the selected humans, which adopts stacked bidirectional inference modules with intermediate supervision to progressively refine the estimated trimap. Finally, a partially supervised matting network is introduced to estimate the alpha matte, which uses a sample-varying loss to train the network on both the finely annotated synthetic data and coarsely annotated real-world data, resulting in high accuracy and good generalization. To evaluate the proposed HSMatt, we construct the first human selective matting dataset, named HSM-200K, which contains over 200,000 human images with instance-level alpha matte annotations. Experimental results demonstrate that the proposed HSMatt outperforms state-of-the-art methods.

3D-integrated multilayered physical reservoir array for learning and forecasting time-series information

A wide reservoir computing system is an advanced architecture composed of multiple reservoir layers in parallel, which enables more complex and diverse internal dynamics for multiple time-series information processing. However, its hardware implementation has not yet been realized due to the lack of a high-performance physical reservoir and the complexity of fabricating multiple stacks. Here, we achieve a proof-of-principle demonstration of such hardware made of a multilayered three-dimensional stacked 3 × 10 × 10 tungsten oxide memristive crossbar array, with which we further realize a wide physical reservoir computing for efficient learning and forecasting of multiple time-series data. Because a three-layer structure allows the seamless and effective extraction of intricate three-dimensional local features produced by various temporal inputs, it can readily outperform two-dimensional based approaches extensively studied previously. Our demonstration paves the way for wide physical reservoir computing systems capable of efficiently processing multiple dynamic time-series information.

Spatial deformable transformer for 3D point cloud registration

Deformable attention only focuses on a small group of key sample-points around the reference point and make itself be able to capture dynamically the local features of input feature map without considering the size of the feature map. Its introduction into point cloud registration will be quicker and easier to extract local geometric features from point cloud than attention. Therefore, we propose a point cloud registration method based on Spatial Deformable Transformer (SDT). SDT consists of a deformable self-attention module and a cross-attention module where the deformable self-attention module is used to enhance local geometric feature representation and the cross-attention module is employed to enhance feature discriminative capability of spatial correspondences. The experimental results show that compared to state-of-the-art registration methods, SDT has a better matching recall, inlier ratio, and registration recall on 3DMatch and 3DLoMatch scene, and has a better generalization ability and time efficiency on ModelNet40 and ModelLoNet40 scene.

LGDNet: local feature coupling global representations network for pulmonary nodules detection.

Detection of suspicious pulmonary nodules from lung CT scans is a crucial task in computer-aided diagnosis (CAD) systems. In recent years, various deep learning-based approaches have been proposed and demonstrated significant potential for addressing this task. However, existing deep convolutional neural networks exhibit limited long-range dependency capabilities and neglect crucial contextual information, resulting in reduced performance on detecting small-size nodules in CT scans. In this work, we propose a novel end-to-end framework called LGDNet for the detection of suspicious pulmonary nodules in lung CT scans by fusing local features and global representations. To overcome the limited long-range dependency capabilities inherent in convolutional operations, a dual-branch module is designed to integrate the convolutional neural network (CNN) branch that extracts local features with the transformer branch that captures global representations. To further address the issue of misalignment between local features and global representations, an attention gate module is proposed in the up-sampling stage to selectively combine misaligned semantic data from both branches, resulting in more accurate detection of small-size nodules. Our experiments on the large-scale LIDC dataset demonstrate that the proposed LGDNet with the dual-branch module and attention gate module could significantly improve the nodule detection sensitivity by achieving a final competition performance metric (CPM) score of 89.49%, outperforming the state-of-the-art nodule detection methods, indicating its potential for clinical applications in the early diagnosis of lung diseases.

Integrating category-related key regions with a dual-stream network for remote sensing scene classification

Remote sensing image scene classification has made great progress with deep learning. Due to complex backgrounds and the large number of objects with inhomogeneous sizes, remote sensing image scene classification still remains challenging. In fact, only a few parts of regions are expected to be representative of the scene. In this paper, we propose a dual-stream framework for remote sensing classification with the help of category-related regions. Swin Transformer-based global feature extractor is used to produce coarse classification results. A category-related key region localization module is developed to extract the representative region from the whole image. The ResNet50 is employed as a local feature extractor to mine critical features from prominent regions. Finally, the classification results is obtained with the shared weighted classify head on global-local features. Extensive experiments show that our method provides a promising approach for key region locations and performs best on three public remote sensing datasets.

Transformaer-based model for lung adenocarcinoma subtypes.

Lung cancer has the highest morbidity and mortality rate among all types of cancer. Histological subtypes serve as crucial markers for the development of lung cancer and possess significant clinical values for cancer diagnosis, prognosis, and prediction of treatment responses. However, existing studies only dichotomize normal and cancerous tissues, failing to capture the unique characteristics of tissue sectionsand cancertypes. Therefore, we have pioneered the classification of lung adenocarcinoma (LAD) cancer tissues into five subtypes (acinar, lepidic, micropapillary, papillary, and solid) based on sectiondata in whole-slide image sections. In addition, a novel model called HybridNet was designed to improve the classificationperformance. HybridNet primarily consists of two interactive streams: a Transformer and a convolutional neural network (CNN). The Transformer stream captures rich global representations using a self-attention mechanism, while the CNN stream extracts local semantic features to optimize image details. Specifically, during the dual-stream parallelism, the feature maps of the Transformer stream as weights are weighted and summed with those of the CNN stream backbone; at the end of the parallelism, the respective final features are concatenated to obtain more discriminative semanticinformation. Experimental results on a private dataset of LAD showed that HybridNet achieved 95.12% classification accuracy, and the accuracy of five histological subtypes (acinar, lepidic, micropapillary, papillary, and solid) reached 94.5%, 97.1%, 94%, 91%, and 99% respectively; the experimental results on the public BreakHis dataset show that HybridNet achieves the best results in three evaluation metrics: accuracy, recall and F1-score, with 92.40%, 90.63%, and 91.43%,respectively. The process of classifying LAD into five subtypes assists pathologists in selecting appropriate treatments and enables them to predict tumor mutation burden (TMB) and analyze the spatial distribution of immune checkpoint proteins based on this and other clinical data. In addition, the proposed HybridNet fuses CNN and Transformer information several times and is able to improve the accuracy of subtype classification, and also shows satisfactory performance on public datasets with some generalizationability.

ERNet: Edge Regularization Network for Cerebral Vessel Segmentation in Digital Subtraction Angiography Images.

Stroke is a leading cause of disability and fatality in the world, with ischemic stroke being the most common type. Digital Subtraction Angiography images, the gold standard in the operation process, can accurately show the contours and blood flow of cerebral vessels. The segmentation of cerebral vessels in DSA images can effectively help physicians assess the lesions. However, due to the disturbances in imaging parameters and changes in imaging scale, accurate cerebral vessel segmentation in DSA images is still a challenging task. In this paper, we propose a novel Edge Regularization Network (ERNet) to segment cerebral vessels in DSA images. Specifically, ERNet employs the erosion and dilation processes on the original binary vessel annotation to generate pseudo-ground truths of False Negative and False Positive, which serve as constraints to refine the coarse predictions based on their mapping relationship with the original vessels. In addition, we exploit a Hybrid Fusion Module based on convolution and transformers to extract local features and build long-range dependencies. Moreover, to support and advance the open research in the field of ischemic stroke, we introduce FPDSA, the first pixel-level semantic segmentation dataset for cerebral vessels. Extensive experiments on FPDSA illustrate the leading performance of our ERNet.

A fusion TFDAN-Based framework for rotating machinery fault diagnosis under noisy labels

Traditional fault diagnosis (FD) for rotating machinery solely based on vibration signals has problems such as inconvenient collection, low accuracy, and poor robustness. This article proposes a fusion framework based on tensor fusion and dual attention network (TFDAN), utilizing acoustic and vibration signals from two datasets of centrifugal pumps and cylindrical roller bearings. Firstly, continuous wavelet transform (CWT) is used to transform two original signals into two-dimensional time–frequency maps to highlight time–frequency features. Then, the images are fed into the fusion framework, where tensor fusion can construct the corresponding time–frequency maps of the working conditions into multi-channel datasets, enhancing the feature connections between acoustic and vibration signals. The dual attention network takes on the fused samples, extracts local features of the image using its positional attention module, and further aggregates feature correlations between channels using its channel attention mechanism. Finally, in order to simulate the actual production situation, different proportions of noisy labels are added to the dataset. In response to the impact of noisy labels, we incorporate an improved contrastive regularization function (ICRF) into the model, fully utilizing its advantage of preventing overfitting of noisy labels. The effectiveness of our proposed method has been demonstrated through two experimental cases. Compared with other methods, our method has better performance in terms of diagnostic accuracy and robustness to noisy labels.

Determination of zearalenone content in wheat by modified one-dimensional convolutional neural network-based Raman spectra

In this study, a Raman spectral model trained based on Transformer-improved one-dimensional convolutional neural network (1D-CNN) was proposed for the quantitative analysis of zearalenone in wheat. Raman spectra of wheat flour samples with varying levels of mold contamination were acquired using a portable Raman spectrometer. The 1D-CNN was introduced to extract local features of Raman spectra using three convolutional layers, and then the Transformer module was employed to encode and decode these local features for obtaining global information. This approach ensured that the network could better learn complex input spectra data, thereby improving the detection accuracy and precision of the model. The results showed that the improved 1D-CNN model had better generalization performance compared to the 1D-CNN model, with a root mean square error of prediction (RMSEP) of 2.5759 μg/kg, a coefficient of determination (RP2) of 0.9837, and a relative prediction deviation (RPD) of 7.9516. The study demonstrated that the 1D-CNN model improved by Transformer could achieve better feature self-learning and multivariate model correction of Raman spectroscopy data. This research introduces a novel reference method for chemometric analysis of Raman spectra.

A combination of multi‐scale and attention based on the U‐shaped network for retinal vessel segmentation

AbstractAutomated partitioning of retinal vessels depicted in fundus images is beneficial in the detection of specific ailments like hypertension and diabetes. However, retinal vessel images have the problems of a large semantic range, more spatial detail, and limited differentiation among the blood vessels and surroundings, which make vessel segmentation challenging. To overcome these obstacles, we designed a new U‐shaped network named SMP‐Net. First, we propose the sequencer‐convolution (SC) module to obtain the ability to extract both local and global features, thereby improving segmentation accuracy. The SC module is used to filter out shallow noise and enable the fusion of deep and shallow features in the maximum skip connection of the U‐shaped network. Then, the residual multi‐kernel pooling (MP) module is designed to obtain additional contextual information while also mitigating the loss of spatial information caused by constant pooling and convolution to improve vessel coherence. Finally, the pixel attention (PA) module redistributes the weight of each pixel using an element‐wise product multiplication operation. This increases the weight of the vascular feature pixels and improves the ability to identify blood vessels in blurred backgrounds. The proposed method has been demonstrated to be effective through sufficient experimentation on publicly available retinal datasets such as DRIVE, STARE, and CHASE_DB1.

Mental fatigue state recognition method based on convolution neural network and long short-term memory

The pace of modern life is accelerating, the pressure of life is gradually increasing, and the long-term accumulation of mental fatigue poses a threat to health. By analyzing physiological signals and parameters, this paper proposes a method that can identify the state of mental fatigue, which helps to maintain a healthy life. The method proposed in this paper is a new recognition method of psychological fatigue state of electrocardiogram signals based on convolutional neural network and long short-term memory. Firstly, the convolution layer of one-dimensional convolutional neural network model is used to extract local features, the key information is extracted through pooling layer, and some redundant data is removed. Then, the extracted features are used as input to the long short-term memory model to further fuse the ECG features. Finally, by integrating the key information through the full connection layer, the accurate recognition of mental fatigue state is successfully realized. The results show that compared with traditional machine learning algorithms, the proposed method significantly improves the accuracy of mental fatigue recognition to 96.3%, which provides a reliable basis for the early warning and evaluation of mental fatigue.

Multi-node knowledge graph assisted distributed fault detection for large-scale industrial processes based on graph attention network and bidirectional LSTMs

Modern industrial processes are characterized by extensive, multiple operation units, and strong coupled correlation of subsystems. Fault detection of large-scale processes is still a challenging problem, especially for tandem plant-wide processes in multiple fields such as water treatment process. In this paper, a novel distributed graph attention network-bidirectional long short-term memory (D-GATBLSTM) fault detection model is proposed for large-scale industrial processes. Firstly, a multi-node knowledge graph (MNKG) is constructed using a joint data and knowledge driven strategy. Secondly, for large-scale industrial process, a global feature extractor of graph attention networks (GATs) is constructed, on the basis of which, sub-blocks are decomposed based on MNKG. Then, local feature extractors of bidirectional long short-term memory (Bi-LSTM) for each sub-block are constructed, in which correlations among multiple sub-blocks are considered. Finally, a multi-subblock fusion collaborative prediction model is constructed and the comprehensive fault detection results are given by the grid search method. The effectiveness of our D-GATBLSTM is exemplified in a secure water treatment process case, where it outperforms baseline models compared, with 27% improvement in precision, 15% increase in recall, and overall F-score enhancement of 0.22.