Local Feature Extraction Research Articles

Non-Intrusive Load Monitoring Based on an Efficient Deep Learning Model With Local Feature Extraction

Non-Intrusive Load Monitoring Based on an Efficient Deep Learning Model With Local Feature Extraction

CapsEnhancer: An Effective Computational Framework for Identifying Enhancers Based on Chaos Game Representation and Capsule Network.

Enhancers are a class of noncoding DNA, serving as crucial regulatory elements in governing gene expression by binding to transcription factors. The identification of enhancers holds paramount importance in the field of biology. However, traditional experimental methods for enhancer identification demand substantial human and material resources. Consequently, there is a growing interest in employing computational methods for enhancer prediction. In this study, we propose a two-stage framework based on deep learning, termed CapsEnhancer, for the identification of enhancers and their strengths. CapsEnhancer utilizes chaos game representation to encode DNA sequences into unique images and employs a capsule network to extract local and global features from sequence "images". Experimental results demonstrate that CapsEnhancer achieves state-of-the-art performance in both stages. In the first and second stages, the accuracy surpasses the previous best methods by 8 and 3.5%, reaching accuracies of 94.5 and 95%, respectively. Notably, this study represents the pioneering application of computer vision methods to enhancer identification tasks. Our work not only contributes novel insights to enhancer identification but also provides a fresh perspective for other biological sequence analysis tasks.

An artificial intelligence approach for the estimation of conduction heat transfer using deep neural networks

PurposeThis study aims to use deep neural networks (DNNs) to learn the conduction heat transfer physics and estimate temperature distribution images in a physical domain without using any physical model or mathematical governing equation.Design/methodology/approachTwo novel DNNs capable of learning the conduction heat transfer physics were defined. The first DNN (U-Net autoencoder residual network [UARN]) was designed to extract local and global features simultaneously. In the second DNN, a conditional generative adversarial network (CGAN) was used to enhance the accuracy of UARN, which is referred to as CGUARN. Then, novel loss functions, introduced based on outlier errors, were used to train the DNNs.FindingsA UARN neural network could learn the physics of heat transfer. Within a few epochs, it reached mean and outlier errors that other DNNs could never reach after many epochs. The composite outlier-mean error as a loss function showed excellent performance in training DNNs for physical images. A UARN could excellently capture local and global features of conduction heat transfer, whereas the composite error could accurately guide DNN to extract high-level information by estimating temperature distribution images.Originality/valueThis study offers a unique approach to estimating physical information, moving from traditional mathematical and physical models to machine learning approaches. Developing novel DNNs and loss functions has shown promising results, opening up new avenues in heat transfer physics and potentially other fields.

MSDE-Net: A Multi-Scale Dual-Encoding Network for Surgical Instrument Segmentation.

Minimally invasive surgery, which relies on surgical robots and microscopes, demands precise image segmentation to ensure safe and efficient procedures. Nevertheless, achieving accurate segmentation of surgical instruments remains challenging due to the complexity of the surgical environment. To tackle this issue, this paper introduces a novel multiscale dual-encoding segmentation network, termed MSDE-Net, designed to automatically and precisely segment surgical instruments. The proposed MSDE-Net leverages a dual-branch encoder comprising a convolutional neural network (CNN) branch and a transformer branch to effectively extract both local and global features. Moreover, an attention fusion block (AFB) is introduced to ensure effective information complementarity between the dual-branch encoding paths. Additionally, a multilayer context fusion block (MCF) is proposed to enhance the network's capacity to simultaneously extract global and local features. Finally, to extend the scope of global feature information under larger receptive fields, a multi-receptive field fusion (MRF) block is incorporated. Through comprehensive experimental evaluations on two publicly available datasets for surgical instrument segmentation, the proposed MSDE-Net demonstrates superior performance compared to existing methods.

MobileDepth: Monocular Depth Estimation Based on Lightweight Vision Transformer

ABSTRACT As deep learning takes off, monocular depth estimation based on convolutional neural networks (CNNs) has made impressive progress. CNNs are superior at extracting local characteristics from a single image; however, they are unable to manage long-range dependence and thus have a substantial impact on the performance of monocular depth estimation. In addition to this, as architectures based on CNNs frequently utilize down sampling operations, numbers of pixel-level features, which are extremely crucial for dense prediction tasks, are lost in the encoder phase. Unlike CNNs, ViT is capable of capturing global feature information, but it requires numbers of parameters and data augmentation owing to its lack of inductive bias. To address the aforementioned difficulties, in this study, we propose a Dilated Self Attention Block (DSAB) as well as a Local and Global Feature Extraction (LGFE) module. The former resolves the inference speed issue of standard ViT models, and we accomplish this by limiting the number of self-attention computations among tokens. The latter combines the advantages of CNNs and ViT, first extracting local representation information in low-dimensional space through standard convolution and then mapping the input tensor to high-dimensional space to capture global information, achieving the simultaneous extraction of global and local characteristics.

Deep hybrid transformer network for robust modulation classification in wireless communications

Modulation classification is a research hot-spot in the field of machine learning and the knowledge-based systems of wireless communication, involving the identification of different types of wireless signals. However, classification targets like radio signals usually involve long-sequence and large-scale data, and the classification of modulation types is affected by environmental noises, resulting in unsatisfactory performance. To overcome these challenges, we propose a novel Deep Hybrid Transformer Network (DH-TR) that makes full use of the intrinsic properties of multi-head self-attention and different neural modules, facilitating the identification of modulation types from global to local perspectives. In particular, DH-TR uses a convolution stem to extract local features inherent in In-phase and Quadrature (IQ) data, based on which the Gated Recurrent Unit (GRU) is used to capture the step-wise sequential patterns of these signals. Afterward, the self-attention based Transformer branch is employed to learn the global and long-term dependencies among the signal patches. With this innovative hybrid design, DH-TR performs well in processing sequential signal data and can better capture the complex relationships between signals. We validate DH-TR’s effectiveness through extensive experiments on four benchmark datasets, demonstrating its superior performance in modulation classification with higher accuracy and robustness to noise compared to existing methods.

PMTT: Parallel multi-scale temporal convolution network and transformer for predicting the time to aging failure of software systems

Software aging is one of the significant factors affecting the reliability and availability of long-running software systems, such as Android, Cloud systems, etc. The time to aging failure (TTAF) prediction for software systems plays a crucial role in proactive rejuvenation scheduling through machine learning or statistical analysis techniques, due to its ability to determine when to perform rejuvenation to mitigate the aging effects. However, software aging characterization is relatively complicated, and only fitting the variations for a single aging indicator cannot grasp the comprehensive degradation process across different case systems; moreover, since software systems often exhibit long and short-term inherent degradation characteristics, existing prediction models possess a poor ability for modeling both global and local information simultaneously. To tackle the above problems, a novel TTAF prediction framework based on the parallel multi-scale temporal convolution network and transformer (named PMTT) is proposed, by mapping various system running indicators reflecting the software aging to TTAF. PMTT possesses the following distinctive characteristics. First, a local feature extraction module that contains multiple channel TCNs with different scales is developed to extract inherent local information from the raw input. Second, in a parallel manner, a global feature extraction module integrating transformer blocks is built to extract global information representation synchronously using the self-attention mechanism. Afterward, high-level global–local features extracted from different channels are fused, and TTAF is estimated through two fully connected regression layers using the fused features. The proposed PMTT has been compared to seven competitors using run-to-failure data collected from Android and OpenStack systems. The experiments have demonstrated the superiority of PMTT, showing an average improvement of 11.2%, 9.0%, and 9.3% in performance across three evaluation metrics compared with the optimal baseline model.

Bridging Convolutional Neural Networks and Transformers for Efficient Crack Detection in Concrete Building Structures.

Detecting cracks in building structures is an essential practice that ensures safety, promotes longevity, and maintains the economic value of the built environment. In the past, machine learning (ML) and deep learning (DL) techniques have been used to enhance classification accuracy. However, the conventional CNN (convolutional neural network) methods incur high computational costs owing to their extensive number of trainable parameters and tend to extract only high-dimensional shallow features that may not comprehensively represent crack characteristics. We proposed a novel convolution and composite attention transformer network (CCTNet) model to address these issues. CCTNet enhances crack identification by processing more input pixels and combining convolution channel attention with window-based self-attention mechanisms. This dual approach aims to leverage the localized feature extraction capabilities of CNNs with the global contextual understanding afforded by self-attention mechanisms. Additionally, we applied an improved cross-attention module within CCTNet to increase the interaction and integration of features across adjacent windows. The performance of CCTNet on the Historical Building Crack2019, SDTNET2018, and proposed DS3 has a precision of 98.60%, 98.93%, and 99.33%, respectively. Furthermore, the training validation loss of the proposed model is close to zero. In addition, the AUC (area under the curve) is 0.99 and 0.98 for the Historical Building Crack2019 and SDTNET2018, respectively. CCTNet not only outperforms existing methodologies but also sets a new standard for the accurate, efficient, and reliable detection of cracks in building structures.

ResACEUnet: An Improved Transformer Unet Model for 3D Seismic Fault Detection

AbstractDetecting fault constitutes a pivotal aspect of seismic interpretation, significantly influencing the outcomes of petroleum and gas exploration. As artificial intelligence advances, convolutional neural network (CNN) has proven effective in detecting faults in seismic interpretation. Nevertheless, the receptive field of a convolutional layer within CNN is inherently limited, focusing on extracting local features, which lead to the detection of fewer and discontinuous fault features. In this study, integrating the local feature extraction capabilities of CNN with the global feature extraction prowess of transformer, we proposed a U‐shaped hybrid architecture model named ResACEUnet (Attention‐Convolution Unet with Efficient block) to detect fault of three‐dimensional (3D) seismic data. In ResACEUnet, we introduced a module called ACE block, which integrates convolution and attention mechanisms. This module enabled the model to simultaneously extract local features and model global contextual information, capturing more accurate fault features. In addition, we utilized a joint loss function named BCEDice loss, which composed of BCE (binary cross‐entropy) loss and dice loss to tackle the challenge of imbalanced positive and negative samples. The model was trained on a synthetic data set, with a range of data augmentation techniques were employed to bolster its generalization capabilities and robustness. We implemented our proposed method on the offshore F3 seismic data from the Netherlands and seismic data from Kerry3D and Parihaka in New Zealand. Compared to conventional popular models such as Unet, ResUnet, and SwinUnetR, ResACEUnet demonstrated superior capabilities in capturing more features and identifying fault with higher accuracy and continuity.

GLFER-Net: a polyphonic sound source localization and detection network based on global-local feature extraction and recalibration

Polyphonic sound source localization and detection (SSLD) task aims to recognize the categories of sound events, identify their onset and offset times, and detect their corresponding direction-of-arrival (DOA), where polyphonic refers to the occurrence of multiple overlapping sound sources in a segment. However, vanilla SSLD methods based on convolutional recurrent neural network (CRNN) suffer from insufficient feature extraction. The convolutions with kernel of single scale in CRNN fail to adequately extract multi-scale features of sound events, which have diverse time-frequency characteristics. It results in that the extracted features lack fine-grained information helpful for the localization of sound sources. In response to these challenges, we propose a polyphonic SSLD network based on global-local feature extraction and recalibration (GLFER-Net), where the global-local feature (GLF) extractor is designed to extract the multi-scale global features through an omni-directional dynamic convolution (ODConv) layer and multi-scale feature extraction (MSFE) module. The local feature extraction (LFE) unit is designed for capturing detailed information. Besides, we design a feature recalibration (FR) module to emphasize the crucial features along multiple dimensions. On the open datasets of Task3 in DCASE 2021 and 2022 Challenges, we compared our proposed GLFER-Net with six and four SSLD methods, respectively. The results show that the GLFER-Net achieves competitive performance. The modules we designed are verified to be effective through a series of ablation experiments and visualization analyses.

IProL: identifying DNA promoters from sequence information based on Longformer pre-trained model

Promoters are essential elements of DNA sequence, usually located in the immediate region of the gene transcription start sites, and play a critical role in the regulation of gene transcription. Its importance in molecular biology and genetics has attracted the research interest of researchers, and it has become a consensus to seek a computational method to efficiently identify promoters. Still, existing methods suffer from imbalanced recognition capabilities for positive and negative samples, and their recognition effect can still be further improved. We conducted research on E. coli promoters and proposed a more advanced prediction model, iProL, based on the Longformer pre-trained model in the field of natural language processing. iProL does not rely on prior biological knowledge but simply uses promoter DNA sequences as plain text to identify promoters. It also combines one-dimensional convolutional neural networks and bidirectional long short-term memory to extract both local and global features. Experimental results show that iProL has a more balanced and superior performance than currently published methods. Additionally, we constructed a novel independent test set following the previous specification and compared iProL with three existing methods on this independent test set.

Time series modeling and forecasting of epidemic spreading processes using deep transfer learning

Traditional data-driven methods for modeling and predicting epidemic spreading typically operate in an independent and identically distributed setting. However, epidemic spreading on complex networks exhibits significant heterogeneity across different phases, regions, and viruses, indicating that epidemic time series may not be independent and identically distributed due to temporal and spatial variations. In this article, a novel deep transfer learning method integrating convolutional neural networks (CNNs) and bi-directional long short-term memory (BiLSTM) networks is proposed to model and forecast epidemics with heterogeneous data. The proposed method combines a CNN-based layer for local feature extraction, a BiLSTM-based layer for temporal analysis, and a fully connected layer for prediction, and employs transfer learning to enhance the generalization ability of the CNN-BiLSTM model. To improve prediction performance, hyperparameter tuning is conducted using particle swarm optimization during model training. Finally, we adopt the proposed approach to characterize the spatio-temporal spreading dynamics of COVID-19 and infer the pathological heterogeneity among epidemics of severe acute respiratory syndrome (SARS), influenza A (H1N1), and COVID-19. The comprehensive results demonstrate the effectiveness of the proposed approach in exploring the spatiotemporal variations in the spread of epidemics and characterizing the epidemiological features of different viruses. Moreover, the proposed method can significantly reduce modeling and predicting errors in epidemic spread to some extent.

A dual-track feature fusion model utilizing Group Shuffle Residual DeformNet and swin transformer for the classification of grape leaf diseases

Grape cultivation is important globally, contributing to the agricultural economy and providing diverse grape-based products. However, the susceptibility of grapes to disease poses a significant threat to yield and quality. Traditional disease identification methods demand expert knowledge, which limits scalability and efficiency. To address these limitations our research aims to design an automated deep learning approach for grape leaf disease detection. This research introduces a novel dual-track network for classifying grape leaf diseases, employing a combination of the Swin Transformer and Group Shuffle Residual DeformNet (GSRDN) tracks. The Swin Transformer track exploits shifted window techniques to construct hierarchical feature maps, enhancing global feature extraction. Simultaneously, the GSRDN track combines Group Shuffle Depthwise Residual block and Deformable Convolution block to extract local features with reduced computational complexity. The features from both tracks are concatenated and processed through Triplet Attention for cross-dimensional interaction. The proposed model achieved an accuracy of 98.6%, the precision, recall, and F1-score are recorded as 98.7%, 98.59%, and 98.64%, respectively as validated on a dataset containing grape leaf disease information from the PlantVillage dataset, demonstrating its potential for efficient grape disease classification.

Advanced Hyperspectral Image Analysis: Superpixelwise Multiscale Adaptive T-HOSVD for 3D Feature Extraction.

Hyperspectral images (HSIs) possess an inherent three-order structure, prompting increased interest in extracting 3D features. Tensor analysis and low-rank representations, notably truncated higher-order SVD (T-HOSVD), have gained prominence for this purpose. However, determining the optimal order and addressing sensitivity to changes in data distribution remain challenging. To tackle these issues, this paper introduces an unsupervised Superpixelwise Multiscale Adaptive T-HOSVD (SmaT-HOSVD) method. Leveraging superpixel segmentation, the algorithm identifies homogeneous regions, facilitating the extraction of local features to enhance spatial contextual information within the image. Subsequently, T-HOSVD is adaptively applied to the obtained superpixel blocks for feature extraction and fusion across different scales. SmaT-HOSVD harnesses superpixel blocks and low-rank representations to extract 3D features, effectively capturing both spectral and spatial information of HSIs. By integrating optimal-rank estimation and multiscale fusion strategies, it acquires more comprehensive low-rank information and mitigates sensitivity to data variations. Notably, when trained on subsets comprising 2%, 1%, and 1% of the Indian Pines, University of Pavia, and Salinas datasets, respectively, SmaT-HOSVD achieves impressive overall accuracies of 93.31%, 97.21%, and 99.25%, while maintaining excellent efficiency. Future research will explore SmaT-HOSVD's applicability in deep-sea HSI classification and pursue additional avenues for advancing the field.

Research on Product Advertising Design Combining Feature Extraction Technology and Web3D Technology

This work, built on the Unity3D development platform, presents a way for merging feature extraction technology and Web3D technology into advertising design to effectively address the issues of poor efficiency and distortion in the field. Using the candidate text layout generating technique of visual salience, we first build the vector function set based on the three main colors, then we produce the visual communication partition model of advertising design. Next, the number of feature parameters of the shape advertising design is obtained via the establishment of coding coefficient constraint features and the use of an upgraded neural network technique to extract local feature parameter information about the product. Finally, the product design model is brought to life using Web3D technology to boost advertising design's productivity and accuracy. The experiments show that this method not only results in a high rate of correct product identification but also offers a fresh viewpoint on the visual communication of product advertising design by merging the two disciplines.

A method for extracting aquatic animal disease prevention and control events integrated with capsule network

Addressing the issue of long-tail event entity recognition in aquatic animal disease prevention and control, this paper proposes an event extraction method that integrates capsule networks. The method designs two parallel networks: the first utilizes BERT + TextCNN to extract initial and local features from the text, while Multi-BiLSTM further captures multi-dimensional dependency information features. The second network employs capsule networks to extract local features and learns spatial semantic relationships among different event entities. The features extracted from both networks are then fused. Experimental results demonstrate that this method achieves significant recognition performance on the aquatic animal disease prevention and control event dataset, with an F1 score of 75.83%, effectively addressing the challenge of long-tail event entity recognition.

A Visible and Synthetic Aperture Radar Image Fusion Algorithm Based on a Transformer and a Convolutional Neural Network

For visible and Synthetic Aperture Radar (SAR) image fusion, this paper proposes a visible and SAR image fusion algorithm (TCFuse) based on a Transformer and a Convolutional Neural Network (CNN). Firstly, in this paper, the Restormer Block is used to extract cross-modal shallow features. Then, we introduce an improved Transformer–CNN Feature Extractor (TCFE) with a two-branch residual structure. This includes a Transformer branch that introduces the Lite Transformer (LT) and DropKey for extracting global features and a CNN branch that introduces the Convolutional Block Attention Module (CBAM) for extracting local features. Finally, the fused image is output based on global features extracted by the Transformer branch and local features extracted by the CNN branch. The experiments show that the algorithm proposed in this paper can effectively achieve the extraction and fusion of global and local features of visible and SAR images, so that high-quality visible and SAR fusion images can be obtained.

EPSViTs: A hybrid architecture for image classification based on parameter-shared multi-head self-attention

Vision transformers have been successfully applied to image recognition tasks due to their ability to capture long-range dependencies within an image. However, they still suffer from weak local feature extraction, easy loss of channel interaction information in one-dimensional multi-head self-attention modeling, and large number of parameters. This paper proposes a lightweight image classification hybrid architecture named EPSViTs (Efficient Parameter Shared Transformer, EPSViTs). Firstly, a new local feature extraction module is designed to effectively enhance the expression of local features. Secondly, using the parameter sharing approach, a lightweight multi-head self-attention module based on information interaction is designed, which can globally model the image from both spatial and channel dimensions, and mine the potential correlation of the image in space and channel. Extensive experiments are conducted on three public datasets, a subset of ImageNet, Cifar100 and APTOS2019, a private dataset Mushroom66, and the results show that the hybrid architecture EPSViTs proposed in this paper based on parameter sharing for multi-head self-attentive image classification has obvious advantages, especially on a subset of ImageNet to reach 89.18%, which is a 3.8% improvement compared to Edgevits_xxs, verifying the effectiveness of the model.

AIMHNet: An Attribute-Insensitive Multiscale Hourglass Network for Rain Streak and Raindrop Removal

CNN-based methods have made great progress in single-image rain removal. Most recent methods improve performance by increasing the depth of the network. To fully extract local and global features while reducing inference time, we propose a top-to-down attribute-insensitive multiscale hourglass network for rain streak and raindrop removal. For the rain removal task, we expect that the constructed network can accurately identify the various attributes of the rain information characteristics of the small target. Considering the difference in the size, shape, direction and density of rain streak and raindrop, inspired by the performance of hourglass architecture to capture multiscale features in human pose estimation, we introduce an attribute-insensitive hourglass module to recognize the attributes of rain streak and raindrop in a unified framework. This feature extraction module could capture the characteristics of rain streak and raindrop with different attributes. This stacked hourglass blocks down-sample features and then up-samples them back to the original resolution based on discrete wavelet transform and inverse discrete wavelet transform. We perform extensive experiments on five synthetic and real-world de-raining datasets to validate the effectiveness of our proposed network on rain streak and raindrop removal. The qualitative and quantitative results show that our method is suitable for removing rain streak and raindrop in a unified framework. We present the results of generalization and ablation study for key components, we also report the accuracy of semantic segmentation after preprocessing with all rain removal methods. Our source code will be available on the GitHub: https://github.com/Ruini94/AIMHNet .

SR-RDFAN-LOG: Arbitrary-scale logging image super-resolution reconstruction based on residual dense feature aggregation

In the process of oil and gas exploration and development, ultrasonic logging imaging technology visually displays the lithology, structure, fractures, pores, and other characteristics of the wellbore. This technology has been widely applied in the exploration and development of oil and gas resources. However, the low circumferential sampling rate negatively affects the imaging quality and resolution during drilling logging. To address the issues of low resolution and blurriness in ultrasonic logging images caused by these factors, as well as the limitation of existing super-resolution reconstruction algorithms that can only reconstruct fixed scales, this paper presents an Image Super-Resolution Reconstruction Network called Residual Dense Feature Aggregation (SR-RDFAN-LOG), which is specifically designed to improve ultrasonic logging image resolution. The proposed network consists of two stages, including encoding and decoding. In the encoding stage, a local feature extraction module is initially designed to integrate contextual features. Subsequently, a residual dense module is introduced for extracting deep features. Finally, a spatial attention module is constructed to focus on high-frequency details. In the decoding stage, a multilayer perceptron is used to decode the feature maps and achieve super-resolution reconstruction at arbitrary scales. The method is tested on an ultrasonic logging images test set with in-distribution scale factors of × 2 and × 4, and out-of-distribution scale factors of × 8 and × 16. When the scaling factors are × 2 and × 4, compared to the best-performing method, the peak signal-to-noise ratio (PSNR) increased by 0.4892 dB and 0.3849 dB, respectively, the structural similarity (SSIM) values increased by 0.0002 and 0.0012, respectively, and the learned perceptual image patch similarity (LPIPS) decreased by 0.0001 and 0.0008, respectively. Compared to the bicubic interpolation method, the LPIPS decreased by 0.0001 and 0.0008 at scale factors of × 8 and × 16, PSNR increased by 1.9614 dB and 2.1002 dB, respectively, and SSIM values increased by 0.0476 and 0.033, respectively. The LPIPS values decreased by 0.2233 and 0.1070, respectively. The experimental results demonstrate that our method outperforms other mainstream methods, achieves super-resolution reconstruction at arbitrary scale, and provides support for subsequent fracture-cave evaluation and oil and gas exploration.