Encoder Network Research Articles

An Enhanced Feature Learning CNN Using Auto-encoder for Partially Occluded Target Recognition in SAR Imagery

Deep learning algorithms have found extensive utility in Automatic Target Recognition (ATR) using Synthetic Aperture Radar (SAR). Despite this, conventional deep learning-based ATR models often lack in their performance when subjected to Extended Operating Conditions (EOCs). Significant variations in depression, noise interference, limited training data, and partial occlusion are primary factors in EOCs. In this paper, we proposed a deep neural network-based technique to recognize SAR targets for Standard Operating Conditions (SOC) and Extended Operating Conditions (EOC). The proposed approach comprises an encoder and decoder network. The encoder network integrates multiple dilated convolutions to address noise-induced recognition challenges for extracting multi-scale features. Furthermore, a feature refinement module is integrated within the multi-scale channels to enhance the extraction of discriminative and resilient features. Feature refinement selectively emphasizes informative features while suppressing irrelevant ones. Additionally, a salient attention block and a spatial feature learning module are introduced for feature selection. Selected features are then utilized in the top-level network layer of the encoder network to preserve spatial relationships among diverse features. This spatial feature learning module aids in recognition performance, particularly when training data is scarce. The decoder network consists of stacked transposed convolution layers, facilitating the encoder network's acquisition of discriminative image features. Experimental evaluations were conducted on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset to validate the efficacy of the proposed methodology.

A Convolution Auto-Encoders Network for Aero-Engine Hot Jet FT-IR Spectrum Feature Extraction and Classification

Aiming at classification and recognition of aero-engines, two telemetry Fourier transform infrared (FT-IR) spectrometers are utilized to measure the infrared spectrum of the areo-engine hot jet, meanwhile a spectrum dataset of six types of areo-engines is established. In this paper, a convolutional autoencoder (CAE) is designed for spectral feature extraction and classification, which is composed of coding network, decoding network, and classification network. The encoder network consists of convolutional layers and maximum pooling layers, the decoder network consists of up-sampling layers and deconvolution layers, and the classification network consists of a flattened layer and a dense layer. In the experiment, data for the spectral dataset were randomly sampled at a ratio of 8:1:1 to produce the training set, validation set, and prediction set, and the performance measures were accuracy, precision, recall, confusion matrix, F1 score, ROC curve, and AUC value. The experimental result of CAE reached 96% accuracy and the prediction running time was 1.57 s. Compared with the classical PCA feature extraction and SVM, XGBoost, AdaBoost, and Random Forest classifier algorithms, as well as AE, CSAE, and CVAE deep learning classification methods, the CAE network can achieve higher accuracy and efficiency and can complete the spectral classification task.

REDalign: accurate RNA structural alignment using residual encoder-decoder network

BackgroundRNA secondary structural alignment serves as a foundational procedure in identifying conserved structural motifs among RNA sequences, crucially advancing our understanding of novel RNAs via comparative genomic analysis. While various computational strategies for RNA structural alignment exist, they often come with high computational complexity. Specifically, when addressing a set of RNAs with unknown structures, the task of simultaneously predicting their consensus secondary structure and determining the optimal sequence alignment requires an overwhelming computational effort of O(L6)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$O(L^6)$$\\end{document} for each RNA pair. Such an extremely high computational complexity makes these methods impractical for large-scale analysis despite their accurate alignment capabilities.ResultsIn this paper, we introduce REDalign, an innovative approach based on deep learning for RNA secondary structural alignment. By utilizing a residual encoder-decoder network, REDalign can efficiently capture consensus structures and optimize structural alignments. In this learning model, the encoder network leverages a hierarchical pyramid to assimilate high-level structural features. Concurrently, the decoder network, enhanced with residual skip connections, integrates multi-level encoded features to learn detailed feature hierarchies with fewer parameter sets. REDalign significantly reduces computational complexity compared to Sankoff-style algorithms and effectively handles non-nested structures, including pseudoknots, which are challenging for traditional alignment methods. Extensive evaluations demonstrate that REDalign provides superior accuracy and substantial computational efficiency.ConclusionREDalign presents a significant advancement in RNA secondary structural alignment, balancing high alignment accuracy with lower computational demands. Its ability to handle complex RNA structures, including pseudoknots, makes it an effective tool for large-scale RNA analysis, with potential implications for accelerating discoveries in RNA research and comparative genomics.

Lower Limb Motion Recognition Based on sEMG and CNN-TL Fusion Model.

To enhance the classification accuracy of lower limb movements, a fusion recognition model integrating a surface electromyography (sEMG)-based convolutional neural network, transformer encoder, and long short-term memory network (CNN-Transformer-LSTM, CNN-TL) was proposed in this study. By combining these advanced techniques, significant improvements in movement classification were achieved. Firstly, sEMG data were collected from 20 subjects as they performed four distinct gait movements: walking upstairs, walking downstairs, walking on a level surface, and squatting. Subsequently, the gathered sEMG data underwent preprocessing, with features extracted from both the time domain and frequency domain. These features were then used as inputs for the machine learning recognition model. Finally, based on the preprocessed sEMG data, the CNN-TL lower limb action recognition model was constructed. The performance of CNN-TL was then compared with that of the CNN, LSTM, and SVM models. The results demonstrated that the accuracy of the CNN-TL model in lower limb action recognition was 3.76%, 5.92%, and 14.92% higher than that of the CNN-LSTM, CNN, and SVM models, respectively, thereby proving its superior classification performance. An effective scheme for improving lower limb motor function in rehabilitation and assistance devices was thus provided.

Style-Controlled Image Synthesis of Concrete Damages Based on Fusion of Convolutional Encoder and Attention-Enhanced Conditional Generative Adversarial Network

Style-Controlled Image Synthesis of Concrete Damages Based on Fusion of Convolutional Encoder and Attention-Enhanced Conditional Generative Adversarial Network

Multi-stream encoder and multi-layer comparative learning network for fluid classification based on logging data via wavelet threshold denoising

In recent years, the importance of fluid classification in oil and gas exploration has become increasingly evident. However, the inherent complexity of logging data and noise pose significant challenges to this task. To this end, this paper proposes a wavelet threshold denoising-based multi-stream encoder combined with multi-level comparison learning (LogMEC-MCL) framework for fluid classification. The framework begins with comprehensive noise reduction, utilizing wavelet threshold denoising to preprocess the data. It then extracts global temporal features by incorporating attention gated recurrent units within the multi-stream encoder. In parallel, multi-scale convolutional neural networks capture local spatial information, ensuring a more complete understanding of the data. To further improve the discriminative power of the extracted features, the framework includes two contrastive learning modules: instance-level contrastive learning and temporal contrastive learning. These components work together to refine feature differentiation, particularly in challenging cases. Additionally, the framework introduces a custom-designed loss function that combines cross-entropy loss with contrastive loss, thereby optimizing the classification performance. The proposed model was rigorously evaluated using a real-world logging dataset from the Tarim Basin in China. The experimental results demonstrate that LogMEC-MCL consistently outperforms current state-of-the-art models on two test datasets, achieving maximum classification accuracies of 95.70% and 95.50%, respectively.

Channel reconstruction through improvised deep learning architecture for high-speed networks

Efficient acquisition of channel state information (CSI) is quite complicated process but immensely essential to exploit probable benefits of massive multiple input multiple output (MIMO) systems. Therefore, a deep learningbased model is utilized to estimate channel feedback in a massive MIMO system. The proposed improvised deep learning-based channel estimation (IDLCE) model enhances channel reconstruction efficiency by using multiple convolutional layers and residual blocks. The proposed IDLCE model utilizes encoder network to compress CSI matrices where decoder network is used to downlink reconstruct CSI matrices. Here, an additional quantization block is incorporated to improve feedback reconstruction accuracy by reducing channel errors. A COST 2,100 model is adopted to analyse performance efficiency for both indoor and outdoor scenarios. Further, deep learning-based model is used to train thousands of parameter and correlation coefficients much faster and to minimize computational complexity. The proposed IDLCE model evaluate performance in terms of normalized mean square error (NMSE), correlation efficiency and reconstruction accuracy and compared against varied state-of-art-channel estimation techniques. Excellent performance results are obtained with large improvement in channel reconstruction accuracy

Unsupervised Histopathological Sub-Image Analysis for Breast Cancer Diagnosis Using Variational Autoencoders, Clustering, and Supervised Learning

This paper presents an integrated approach to breast cancer diagnosis that combines unsupervised and supervised learning techniques. The method involves using a pre-trained VGG19 model to process sub-images from the BreaKHis dataset, divided into nine parts for comprehensive analysis. This will be followed by a complete description of the architecture and workings of the variational Autoencoder (VAE) used for unsupervised Learning. The encoder network maps the input features to lower dimensions, capturing the most essential information. VAE learns a compressed representation of sub-images, facilitating a more profound understanding of underlying patterns and structures. For this reason, we then employ k-means clustering on the encoded representation to find naturally occurring clusters in our data set comprising a histopathological image. Every single sub-image is later fed into the VGG19-SVM model for classification purposes. During magnification at 100x, this model has attained a fantastic accuracy rate of 98.56%. Combining unsupervised analysis with VAE/k-means clustering and supervised classification with VGG19/SVM can integrate information from both methods, thereby improving the accuracy and robustness of such a task as sub-image classification in breast cancer histopathology.

DDNet: Depth Dominant Network for Semantic Segmentation of RGB-D Images.

Convolutional neural networks (CNNs) have been widely applied to parse indoor scenes and segment objects represented by color images. Nonetheless, the lack of geometric and context information is a problem for most RGB-based methods, with which depth features are only used as an auxiliary module in RGB-D semantic segmentation. In this study, a novel depth dominant network (DDNet) is proposed to fully utilize the rich context information in the depth map. The critical insight is that obvious geometric information from the depth image is more conducive to segmentation than RGB data. Compared with other methods, DDNet is a depth-based network with two branches of CNNs to extract color and depth features. As the core of the encoder network, the depth branch is given a larger fusion weight to extract geometric information, while semantic information and complementary geometric information are provided by the color branch for the depth feature maps. The effectiveness of our proposed depth-based architecture has been demonstrated by comprehensive experimental evaluations and ablation studies on challenging RGB-D semantic segmentation benchmarks, including NYUv2 and a subset of ScanNetv2.

A novel daily runoff forecasting model based on global features and enhanced local feature interpretation

The development of artificial intelligence has introduced new perspectives to the field of hydrological forecasting. However, there is still a lack of research on efficiently identifying the physical characteristics of runoff sequences and developing prediction models that consider global and local sequence features. This study proposes a parallel computing prediction model called IMCAEN (Integrated Multi-Feature Causal Dilated Convolutional Attention Encoder Network) to address these issues. Unlike existing models, this model can monitor fluctuations and anomalies in time series. Incorporating the CDC-AA (Causal Dilated Convolutional Network with Aggregation Attention) and encoder structure captures both local sequence variations and global abrupt anomalies, allowing for comprehensive attention to sequence features. When predicting runoff data from three different hydrological conditions, the IMCAEN model achieved NSEC (Nash-Sutcliffe Efficiency Coefficient) values of 0.98, 0.97, and 0.88, respectively, and outperformed benchmark models in other evaluation indicators as well. Given the opacity of the feature distribution process in AI models, SHAP (SHapleyAdditive exPlanations) analysis and spatial expression of feature distribution are used to assess the contribution of each feature variable to the long-term trend of runoff and to verify the distribution of features trained in each module. The proposed IMCAEN model efficiently captures local and global information in the runoff evolution process through parallel computing and shared features, enabling accurate runoff forecasting and providing critical references for timely warnings and predictions.

Event Camera Denoising Using Asynchronous Spatio-Temporal Event Denoising Neural Network

Abstract. Compared to conventional cameras, event cameras represent a noteworthy advancement in neuromorphic imaging technology, garnering considerable attention from researchers due to their distinct advantages. However, event cameras are susceptible to significant levels of measurement noise, which can detrimentally affect the performance of algorithms reliant on event stream for tasks such as perception and navigation. In this study, we introduce a novel method for denoising event stream, aiming to filter out events that do not accurately reflect genuine logarithmic intensity changes within the observed scene. Our approach focuses on the asynchronous nature and spatiotemporal properties of events, culminating in the development of a novel Asynchronous Spatio- Temporal Event Denoising neural Network(ASTEDNet). This network operates directly on event streams, circumventing the need to convert event stream into denser formats like image frames, thereby preserving their inherent asynchronous nature. Drawing upon principles from graph encoding and temporal convolutional networks, we incorporate spatiotemporal feature attention mechanisms to capture the temporal and spatial correlations between events. This enables the classification of each active event pixel in the original stream as either representing a genuine intensity change or noise. Comparative evaluations conducted on multiple datasets against state-of-the-art methods demonstrate the remarkable efficacy and robustness of our proposed algorithm in noise removal while retaining meaningful event information within the scene.

Deconstructing Deep Active Inference: A Contrarian Information Gatherer.

Active inference is a theory of perception, learning, and decision making that can be applied to neuroscience, robotics, psychology, and machine learning. Recently, intensive research has been taking place to scale up this framework using Monte Carlo tree search and deep learning. The goal of this activity is to solve more complicated tasks using deep active inference. First, we review the existing literature and then progressively build a deep active inference agent as follows: we (1) implement a variational autoencoder (VAE), (2) implement a deep hidden Markov model (HMM), and (3) implement a deep critical hidden Markov model (CHMM). For the CHMM, we implemented two versions, one minimizing expected free energy, CHMM[EFE] and one maximizing rewards, CHMM[reward]. Then we experimented with three different action selection strategies: the ε-greedy algorithm as well as softmax and best action selection. According to our experiments, the models able to solve the dSprites environment are the ones that maximize rewards. On further inspection, we found that the CHMM minimizing expected free energy almost always picks the same action, which makes it unable to solve the dSprites environment. In contrast, the CHMM maximizing reward keeps on selecting all the actions, enabling it to successfully solve the task. The only difference between those two CHMMs is the epistemic value, which aims to make the outputs of the transition and encoder networks as close as possible. Thus, the CHMM minimizing expected free energy repeatedly picks a single action and becomes an expert at predicting the future when selecting this action. This effectively makes the KL divergence between the output of the transition and encoder networks small. Additionally, when selecting the action down the average reward is zero, while for all the other actions, the expected reward will be negative. Therefore, if the CHMM has to stick to a single action to keep the KL divergence small, then the action down is the most rewarding. We also show in simulation that the epistemic value used in deep active inference can behave degenerately and in certain circumstances effectively lose, rather than gain, information. As the agent minimizing EFE is not able to explore its environment, the appropriate formulation of the epistemic value in deep active inference remains an open question.

Multi-scale input layers and dense decoder aggregation network for COVID-19 lesion segmentation from CT scans

Accurate segmentation of COVID-19 lesions from medical images is essential for achieving precise diagnosis and developing effective treatment strategies. Unfortunately, this task presents significant challenges, owing to the complex and diverse characteristics of opaque areas, subtle differences between infected and healthy tissue, and the presence of noise in CT images. To address these difficulties, this paper designs a new deep-learning architecture (named MD-Net) based on multi-scale input layers and dense decoder aggregation network for COVID-19 lesion segmentation. In our framework, the U-shaped structure serves as the cornerstone to facilitate complex hierarchical representations essential for accurate segmentation. Then, by introducing the multi-scale input layers (MIL), the network can effectively analyze both fine-grained details and contextual information in the original image. Furthermore, we introduce an SE-Conv module in the encoder network, which can enhance the ability to identify relevant information while simultaneously suppressing the transmission of extraneous or non-lesion information. Additionally, we design a dense decoder aggregation (DDA) module to integrate feature distributions and important COVID-19 lesion information from adjacent encoder layers. Finally, we conducted a comprehensive quantitative analysis and comparison between two publicly available datasets, namely Vid-QU-EX and QaTa-COV19-v2, to assess the robustness and versatility of MD-Net in segmenting COVID-19 lesions. The experimental results show that the proposed MD-Net has superior performance compared to its competitors, and it exhibits higher scores on the Dice value, Matthews correlation coefficient (Mcc), and Jaccard index. In addition, we also conducted ablation studies on the Vid-QU-EX dataset to evaluate the contributions of each key component within the proposed architecture.

Adaptive Learning of the Latent Space of Wasserstein Generative Adversarial Networks

Generative models based on latent variables, such as generative adversarial networks (GANs) and variational auto-encoders (VAEs), have gained lots of interests due to their impressive performance in many fields. However, many data such as natural images usually do not populate the ambient Euclidean space but instead reside in a lower-dimensional manifold. Thus an inappropriate choice of the latent dimension fails to uncover the structure of the data, possibly resulting in mismatch of latent representations and poor generative qualities. Towards addressing these problems, we propose a novel framework called the latent Wasserstein GAN (LWGAN) that fuses the Wasserstein auto-encoder and the Wasserstein GAN so that the intrinsic dimension of the data manifold can be adaptively learned by a modified informative latent distribution. We prove that there exist an encoder network and a generator network in such a way that the intrinsic dimension of the learned encoding distribution is equal to the dimension of the data manifold. We theoretically establish that our estimated intrinsic dimension is a consistent estimate of the true dimension of the data manifold. Meanwhile, we provide an upper bound on the generalization error of LWGAN, implying that we force the synthetic data distribution to be similar to the real data distribution from a population perspective. Comprehensive empirical experiments verify our framework and show that LWGAN is able to identify the correct intrinsic dimension under several scenarios, and simultaneously generate high-quality synthetic data by sampling from the learned latent distribution.

Advancing infrared and visible image fusion with an enhanced multiscale encoder and attention-based networks

Infrared and visible image fusion aims to produce images that highlight key targets and offer distinct textures, by merging the thermal radiation infrared images with the detailed texture visible images. Traditional auto encoder-decoder-based fusion methods often rely on manually designed fusion strategies, which lack flexibility across different scenarios. Addressing this limitation, we introduce EMAFusion, a fusion approach featuring an enhanced multiscale encoder and a learnable, lightweight fusion network. Our method incorporates skip connections, the convolutional block attention module (CBAM), and nest architecture within the auto encoder-decoder framework to adeptly extract and preserve multiscale features for fusion tasks. Furthermore, a fusion network driven by spatial and channel attention mechanisms is proposed, designed to precisely capture and integrate essential features from both image types. Comprehensive evaluations of the TNO image fusion dataset affirm the proposed method's superiority over existing state-of-the-art techniques, demonstrating its potential for advancing infrared and visible image fusion.

A Vision-Language Model-Based Traffic Sign Detection Method for High-Resolution Drone Images: A Case Study in Guyuan, China.

As a fundamental element of the transportation system, traffic signs are widely used to guide traffic behaviors. In recent years, drones have emerged as an important tool for monitoring the conditions of traffic signs. However, the existing image processing technique is heavily reliant on image annotations. It is time consuming to build a high-quality dataset with diverse training images and human annotations. In this paper, we introduce the utilization of Vision-language Models (VLMs) in the traffic sign detection task. Without the need for discrete image labels, the rapid deployment is fulfilled by the multi-modal learning and large-scale pretrained networks. First, we compile a keyword dictionary to explain traffic signs. The Chinese national standard is used to suggest the shape and color information. Our program conducts Bootstrapping Language-image Pretraining v2 (BLIPv2) to translate representative images into text descriptions. Second, a Contrastive Language-image Pretraining (CLIP) framework is applied to characterize not only drone images but also text descriptions. Our method utilizes the pretrained encoder network to create visual features and word embeddings. Third, the category of each traffic sign is predicted according to the similarity between drone images and keywords. Cosine distance and softmax function are performed to calculate the class probability distribution. To evaluate the performance, we apply the proposed method in a practical application. The drone images captured from Guyuan, China, are employed to record the conditions of traffic signs. Further experiments include two widely used public datasets. The calculation results indicate that our vision-language model-based method has an acceptable prediction accuracy and low training cost.

Consistency-constrained unsupervised video anomaly detection framework based on Co-teaching

Intelligent video surveillance continues to be a vibrant research domain within the field of computer vision. However, existing representation learning frameworks primarily focus on static information extraction frame by frame such as appearance features, they often overlook the valuable dynamic information like optical flow feature inherent in the video data, which is most essential characteristics of sequence data. To mining dynamic features and bridge this gap, our paper introduces a novel anomaly detection framework that balance dynamic information with static information and construct a relationship between appearance features and corresponding optical flow features, where we sets strong consistency constraints, which reduce the loss between dynamic information and corresponding static information, and we leverages collaborative teaching network to ensure a consistent representation of both static and dynamic information for predict. The proposed framework consists of two sets of encoder–decoder pairs complemented by a memory storage module. Operating in parallel with the dual encoder network is a Co-teaching network, with the shared memory module serving as the cornerstone for collaborative training. The Consistency constrained condition guarantees the strong consistency of dynamic and static information in the learned representations. In our experimental phase, we present compelling results that showcase the superior performance of our algorithm across three publicly available datasets.

Detection of Harmful H2S Concentration Range, Health Classification, and Lifespan Prediction of CH4 Sensor Arrays in Marine Environments

Underwater methane (CH4) detection technology is of great significance to the leakage monitoring and location of marine natural gas transportation pipelines, the exploration of submarine hydrothermal activity, and the monitoring of submarine volcanic activity. In order to improve the safety of underwater CH4 detection mission, it is necessary to study the effect of hydrogen sulfide (H2S) in leaking CH4 gas on sensor performance and harmful influence, so as to evaluate the health status and life prediction of underwater CH4 sensor arrays. In the process of detecting CH4, the accuracy decreases when H2S is found in the ocean water. In this study, we proposed an explainable sorted-sparse (ESS) transformer model for concentration interval detection under industrial conditions. The time complexity was decreased to O (n logn) using an explainable sorted-sparse block. Additionally, we proposed the Ocean X generative pre-trained transformer (GPT) model to achieve the online monitoring of the health of the sensors. The ESS transformer model was embedded in the Ocean X GPT model. When the program satisfied the special instructions, it would jump between models, and the online-monitoring question-answering session would be completed. The accuracy of the online monitoring of system health is equal to that of the ESS transformer model. This Ocean-X-generated model can provide a lot of expert information about sensor array failures and electronic noses by text and speech alone. This model had an accuracy of 0.99, which was superior to related models, including transformer encoder (0.98) and convolutional neural networks (CNN) + support vector machine (SVM) (0.97). The Ocean X GPT model for offline question-and-answer tasks had a high mean accuracy (0.99), which was superior to the related models, including long short-term memory–auto encoder (LSTM–AE) (0.96) and GPT decoder (0.98).

A Semi-Supervised Lie Detection Algorithm Based on Integrating Multiple Speech Emotional Features

When people tell lies, they often exhibit tension and emotional fluctuations, reflecting a complex psychological state. However, the scarcity of labeled data in datasets and the complexity of deception information pose significant challenges in extracting effective lie features, which severely restrict the accuracy of lie detection systems. To address this, this paper proposes a semi-supervised lie detection algorithm based on integrating multiple speech emotional features. Firstly, Long Short-Term Memory (LSTM) and Auto Encoder (AE) network process log Mel spectrogram features and acoustic statistical features, respectively, to capture the contextual links between similar features. Secondly, the joint attention model is used to learn the complementary relationship among different features to obtain feature representations with richer details. Lastly, the model combines the unsupervised loss Local Maximum Mean Discrepancy (LMMD) and supervised loss Jefferys multi-loss optimization to enhance the classification performance. Experimental results show that the algorithm proposed in this paper achieves better performance.

Individual differences in spatial working memory strategies differentially reflected in the engagement of control and default brain networks.

Spatial locations can be encoded and maintained in working memory using different representations and strategies. Fine-grained representations provide detailed stimulus information, but are cognitively demanding and prone to inexactness. The uncertainty in fine-grained representations can be compensated by the use of coarse, but robust categorical representations. In this study, we employed an individual differences approach to identify brain activity correlates of the use of fine-grained and categorical representations in spatial working memory. We combined data from six functional magnetic resonance imaging studies, resulting in a sample of $155$ ($77$ women, $25 \pm 5$ years) healthy participants performing a spatial working memory task. Our results showed that individual differences in the use of spatial representations in working memory were associated with distinct patterns of brain activity. Higher precision of fine-grained representations was related to greater engagement of attentional and control brain systems throughout the task trial, and the stronger deactivation of the default network at the time of stimulus encoding. In contrast, the use of categorical representations was associated with lower default network activity during encoding and higher frontoparietal network activation during maintenance. These results may indicate a greater need for attentional resources and protection against interference for fine-grained compared with categorical representations.