Deep Autoencoder Research Articles

Dictionary trained attention constrained low rank and sparse autoencoder for hyperspectral anomaly detection

Dictionary representations and deep learning Autoencoder (AE) models have proven effective in hyperspectral anomaly detection. Dictionary representations offer self-explanation but struggle with complex scenarios. Conversely, autoencoders can capture details in complex scenes but lack self-explanation. Complex scenarios often involve extensive spatial information, making its utilization crucial in hyperspectral anomaly detection. To effectively combine the advantages of both methods and address the insufficient use of spatial information, we propose an attention constrained low-rank and sparse autoencoder for hyperspectral anomaly detection. This model includes two encoders: an attention constrained low-rank autoencoder (AClrAE) trained with a background dictionary and incorporating a Global Self-Attention Module (GAM) to focus on global spatial information, resulting in improved background reconstruction; and an attention constrained sparse autoencoder (ACsAE) trained with an anomaly dictionary and incorporating a Local Self-Attention Module (LAM) to focus on local spatial information, resulting in enhanced anomaly reconstruction. Finally, to merge the detection results from both encoders, a nonlinear fusion scheme is employed. Experiments on multiple real and synthetic datasets demonstrate the effectiveness and feasibility of the proposed method.

Identification of dynamic networks community by fusing deep learning and evolutionary clustering

Community detection is a critical component of network analysis and a hot topic in social computing. Detecting community structure in dynamic networks has important theoretical and practical implications for understanding the intrinsic function of networks and predicting network behavior. However, the majority of existing dynamic community detection methods adopt shallow models, which have limited ability to excavate complex non-linear structures and tend to generate undesirable community structures. In order to obtain an accurate and robust community structure in dynamic networks, we are inspired by network representation learning and utilize the deep learning to detect evolving communities in dynamic networks. In this paper, we propose a novel dynamic community detection method by fusing Deep Learning and Evolutionary Clustering (DLEC). This work attempts to combine deep learning and evolutionary clustering into a unified framework. First, we propose a matrix construction strategy to fully reveal the inherent community structures via the underlying community memberships. Then, we develop a novel multi-layer deep autoencoder framework that consists of multiple non-linear functions to extract the latent deep representation of the dynamic network. Based on the evolutionary clustering framework, a graph regularization term is introduced to ensure the smoothness of the community evolution. Finally, we employ the K-means clustering algorithm on the low-dimensional network space to obtain the community structure. Extensive experimental results on synthetic and real-world networks show that the proposed DLEC algorithm can effectively detect high-quality communities in dynamic networks.

Improving Non-Line-of-Sight Identification in Cellular Positioning Systems Using a Deep Autoencoding and Generative Adversarial Network Model

Positioning service is a critical technology that bridges the physical world with digital information, significantly enhancing efficiency and convenience in life and work. The evolution of 5G technology has proven that positioning services are integral components of current and future cellular networks. However, positioning accuracy is hindered by non-line-of-sight (NLoS) propagation, which severely affects the measurements of angles and delays. In this study, we introduced a deep autoencoding channel transform-generative adversarial network model that utilizes line-of-sight (LoS) samples as a singular category training set to fully extract the latent features of LoS, ultimately employing a discriminator as an NLoS identifier. We validated the proposed model in 5G indoor and indoor factory (dense clutter, low base station) scenarios by assessing its generalization capability across different scenarios. The results indicate that, compared to the state-of-the-art method, the proposed model markedly diminished the utilization of device resources and achieved a 2.15% higher area under the curve while reducing computing time by 12.6%. This approach holds promise for deployment in future positioning terminals to achieve superior localization precision, catering to commercial and industrial Internet of Things applications.

Network anomaly detection using Deep Autoencoder and parallel Artificial Bee Colony algorithm-trained neural network

Cyberattacks are increasingly becoming more complex, which makes intrusion detection extremely difficult. Several intrusion detection approaches have been developed in the literature and utilized to tackle computer security intrusions. Implementing machine learning and deep learning models for network intrusion detection has been a topic of active research in cybersecurity. In this study, artificial neural networks (ANNs), a type of machine learning algorithm, are employed to determine optimal network weight sets during the training phase. Conventional training algorithms, such as back-propagation, may encounter challenges in optimization due to being entrapped within local minima during the iterative optimization process; global search strategies can be slow at locating global minima, and they may suffer from a low detection rate. In the ANN training, the Artificial Bee Colony (ABC) algorithm enables the avoidance of local minimum solutions by conducting a high-performance search in the solution space but it needs some modifications. To address these challenges, this work suggests a Deep Autoencoder (DAE)-based, vectorized, and parallelized ABC algorithm for training feed-forward artificial neural networks, which is tested on the UNSW-NB15 and NF-UNSW-NB15-v2 datasets. Our experimental results demonstrate that the proposed DAE-based parallel ABC-ANN outperforms existing metaheuristics, showing notable improvements in network intrusion detection. The experimental results reveal a notable improvement in network intrusion detection through this proposed approach, exhibiting an increase in detection rate (DR) by 0.76 to 0.81 and a reduction in false alarm rate (FAR) by 0.016 to 0.005 compared to the ANN-BP algorithm on the UNSW-NB15 dataset. Furthermore, there is a reduction in FAR by 0.006 to 0.0003 compared to the ANN-BP algorithm on the NF-UNSW-NB15-v2 dataset. These findings underscore the effectiveness of our proposed approach in enhancing network security against network intrusions.

Enhancing Clustering Performance of Failed Test Cases during HIL Simulation: A Study on Deep Auto-Encoder Structures and Hyperparameter Tuning

Over the last decade, hardware-in-the-loop (HIL) simulation has been established as a safe, efficient, reliable, and flexible method for performing real-time simulation. Furthermore, in the automotive sector, the HIL system has been recommended in the ISO 26262 standard as a powerful platform for performing realistic simulation during system integration testing. As a result of performing HIL black-box tests, the results of executing test cases (TCs) are reported as pass/fail without determining the nature and root causes of the underlying failures. The conventional analysis process of the failed TCs relies on expert knowledge. The higher the number of failed TCs, the higher the cost of manual analysis in terms of time and effort. In light of the shortcomings of existing methodologies, this study presents a novel intelligent framework capable of analyzing failed TCs without the need for expert knowledge or code access. To this end, a convolutional auto-encoder-based deep-learning approach is proposed to extract representative features from the textual description of the failed TCs. Furthermore, k-means-based clustering is used to categorize the extracted features according to their respective failure classes. To illustrate the effectiveness and validate the performance of the proposed method, a virtual test drive with real-time HIL simulation is presented as a case study. The proposed model exhibits superior clustering performance compared to other standalone k-means algorithms, as demonstrated by the David Bouldin Index (DBI) and accuracy values of 0.5184 and 94.33%, respectively. Furthermore, the proposed model shows a significant advantage in terms of feature extraction and clustering performance compared to the current state-of-the-art fault-analysis method. The proposed approach not only supports the validation process and improves the safety and reliability of the systems but also reduces the costs of manual analysis in terms of time and effort.

Deep adaptive ensemble learning for imbalanced credit card fraud detection

ABSTRACT Traditional techniques for credit card fraud detection often struggle with highly imbalanced datasets, which compounds the complexity of fraudulent transaction detection. To tackle this issue, the current paper puts forth a novel technique known as deep adaptive ensemble learning (DAEL). Initially, we employ the ensemble-based under-sampling method to bring equilibrium to the data. This is followed by the application of a deep autoencoder for the purpose of feature extraction and dimensionality reduction. Then, a bidirectional long short-term memory model is trained using the features extracted from the autoencoder. Finally, an adaptive weighting technique is applied to unify multiple models. The empirical results demonstrate that the proposed DAEL technique significantly surpasses individual models and conventional ensemble learning techniques in performance.

Deep Learning-Enabled Dynamic Model for Nutrient Status Detection of Aquaponically Grown Plants

Developing models to assess the nutrient status of plants at various growth stages is challenging due to the dynamic nature of plant development. Hence, this study encoded spatiotemporal information of plants within a single time-series model to precisely assess the nutrient status of aquaponically cultivated lettuce. In particular, the long short-term memory (LSTM) and deep autoencoder (DAE) approaches were combined to classify aquaponically grown lettuce plants according to their nutrient status. The proposed approach was validated using extensive sequential hyperspectral reflectance measurements acquired from lettuce leaves at different growth stages across the growing season. A DAE was used to extract distinct features from each sequential spectral dataset time step. These features were used as input to an LSTM model to classify lettuce grown across a gradient of nutrient levels. The results demonstrated that the LSTM outperformed the convolutional neural network (CNN) and multi-class support vector machine (MCSVM) approaches. Also, features selected by the DAE showed better performance compared to features extracted using both genetic algorithms (GAs) and sequential forward selection (SFS). The hybridization of deep autoencoder and long short-term memory (DAE-LSTM) obtained the highest overall classification accuracy of 94%. The suggested methodology presents a pathway to automating the process of nutrient status diagnosis throughout the entire plant life cycle, with the LSTM technique poised to assume a pivotal role in forthcoming time-series analyses for precision agriculture.

Advanced mathematical modeling of mitigating security threats in smart grids through deep ensemble model

A smart grid (SG) is a cutting-edge electrical grid that utilizes digital communication technology and automation to effectively handle electricity consumption, distribution, and generation. It incorporates energy storage systems, smart meters, and renewable energy sources for bidirectional communication and enhanced energy flow between grid modules. Due to their cyberattack vulnerability, SGs need robust safety measures to protect sensitive data, ensure public safety, and maintain a reliable power supply. Robust safety measures, comprising intrusion detection systems (IDSs), are significant to protect against malicious manipulation, unauthorized access, and data breaches in grid operations, confirming the electricity supply chain’s integrity, resilience, and reliability. Deep learning (DL) improves intrusion recognition in SGs by effectually analyzing network data, recognizing complex attack patterns, and adjusting to dynamic threats in real-time, thereby strengthening the reliability and resilience of the grid against cyber-attacks. This study develops a novel Mountain Gazelle Optimization with Deep Ensemble Learning based intrusion detection (MGODEL-ID) technique on SG environment. The MGODEL-ID methodology exploits ensemble learning with metaheuristic approaches to identify intrusions in the SG environment. Primarily, the MGODEL-ID approach utilizes Z-score normalization to convert the input data into a uniform format. Besides, the MGODEL-ID approach employs the MGO model for feature subset selection. Meanwhile, the detection of intrusions is performed by an ensemble of three classifiers such as long short-term memory (LSTM), deep autoencoder (DAE), and extreme learning machine (ELM). Eventually, the dung beetle optimizer (DBO) is utilized to tune the hyperparameter tuning of the classifiers. A widespread simulation outcome is made to demonstrate the improved security outcomes of the MGODEL-ID model. The experimental values implied that the MGODEL-ID model performs better than other models.

Industrial Process Fault Detection Based on Siamese Recurrent Autoencoder

Although deep autoencoders excel at extracting intricate features, their application in process monitoring is limited by the requirement for large sample sizes and interpretability of latent representations. This work presents a special deep learning structure named Siamese network to detect abnormal deviations in nonlinear dynamic processes. By leveraging the capability of Siamese architecture to process multiple inputs simultaneously, the training sample size expands exponentially, which enhances the learning potential of the model. Furthermore, a long short-term memory unit is integrated to enable the capture of long-term process dynamics. To refine the distribution of latent features extracted from diverse data types, a contrastive loss function is proposed, which strengthens the model's fault detection capabilities and enhances its interpretation of latent representations. Then T2 statistic is established on the latent space to perform fault detection. The effectiveness of the method is demonstrated through case studies on simulation processes and an industrial process.

A Deep Cascade Architecture for Stroke Lesion Segmentation and Synthetic Parametric Map Generation over CT St

Stroke is the second leading cause of mortality worldwide. Immediate attention and diagnosis play a crucial role in patient prognosis. Nowadays, computed tomography (CT) is the most utilized diagnostic imaging for early analysis and lesion stroke detection. Nonetheless, acute lesions are not visible on CT, and it is only possible to use this modality in screening analysis, discarding other neurological affectations. Expert radiologists can observe stroke lesions in advanced stages (subacute and chronic), as hypodense regions, but with limited sensibility and remarked subject variability. Computational strategies have been addressed to support lesion segmentation, following deep autoencoders and multimodal inputs. However, these strategies remain limited due to the high variability in the appearance and geometry of stroke lesions. This work introduces a novel deep representation that uses multimodal inputs from CT studies and parametric maps, computed from perfusion (CTP), to retrieve stroke lesions. The architecture follows an autoencoder deep representation, that forces attention on the geometry of stroke through additive cross-attention modules. Besides, a cascade train is herein proposed to generate synthetic perfusion maps that complement multimodal inputs and help with stroke lesion refinement at each stage of processing. The proposed approach brings saliency maps that support observational expert analysis, about lesion localization, but also lead with automatic shape estimation of the stroke. The proposed approach was validated on the ISLES 2018 public dataset with a total of 92 studies that include the annotation of an expert radiologist. The proposed approach achieves a Dice score of 0.66 and a precision of 0.67, outperforming classical autoencoder approximations.

Identifying faults in UHVDC transmission systems using DAE and ensemble learning

Abstract The current fault identification technique for the ultra high voltage direct current transmission system has a low success rate, the process of extracting feature qualities is complex, and it is difficult to identify high resistance grounding. This research provides a defect evaluation method for UHVDC transmission system using Deep Auto Encoder (DAE) and integrated learning. Firstly, the fault voltage signal is feature extracted using DAE, and then, a series of obtained features are feature screened by XGBoost algorithm and combined with an integrated classifier consisting of Multi-Layer Perceptron (MLP), K Nearest Neighbor (KNN), and Random Forest (RF) classifiers to identify the UHVDC faults. Finally, ±800kV UHVDC transmission line models are established in MATLAB/Simulink simulation software to simulate different types of faults. The study’s findings demonstrate that the fault diagnostic method introduced in this research may significantly enhance the poor detection success rate of current fault diagnostics methods and is more resilient to excessive resistance.

DDLA: a double deep latent autoencoder for diabetic retinopathy diagnose based on continuous glucose sensors.

The current diagnosis of diabetic retinopathy is based on fundus images and clinical experience. However, considering the ineffectiveness and non-portability of medical devices, we aimed to develop a diagnostic model for diabetic retinopathy based on glucose series data from the wearable continuous glucose monitoring system. Therefore, this study developed a novel method, i.e., double deep latent autoencoder, for exploring glycemic variability influence from multi-day glucose data for diabetic retinopathy. Specifically, the model proposed in this research could encode continuous glucose sensor data with non-continuous and variable length via the integration of a data reorganization module and a novel encoding module with fragmented-missing-wise objective function. Additionally, the model implements a double deep autoencoder, which integrated convolutional neural network, long short-term memory, to jointly capturing the inter-day and intra-day glucose latent features from glucose series. The effectiveness of the proposed model is evaluated through a cross-validation method to clinical datasets of 765 type 2 diabetes patients. The proposed method achieves the highest accuracy value (0.89), precision value (0.88), and F1 score (0.73). The results suggest that our model can be used to remotely diagnose and screen for diabetic retinopathy by learning potential features of glucose series data collected by wearable continuous glucose monitoring systems.

Design of a turbo-based deep semantic autoencoder for marine Internet of Things

With the rapid growth of the global marine economy and flourishing maritime activities, the marine Internet of Things (IoT) is gaining unprecedented momentum. However, current marine equipment is deficient in data transmission efficiency and semantic comprehension. To address these issues, this paper proposes a novel End-to-End (E2E) coding scheme, namely the Turbo-based Deep Semantic Autoencoder (Turbo-DSA). The Turbo-DSA achieves joint source-channel coding at the semantic level through the E2E design of transmitter and receiver, while learning to adapt to environment changes. The semantic encoder and decoder are composed of transformer technology, which efficiently converts messages into semantic vectors. These vectors are dynamically adjusted during neural network training according to channel characteristics and background knowledge base. The Turbo structure further enhances the semantic vectors. Specifically, the channel encoder utilizes Turbo structure to separate semantic vectors, ensuring precise transmission of meaning, while the channel decoder employs Turbo iterative decoding to optimize the representation of semantic vectors. This deep integration of the transformer and Turbo structure is ensured by the design of the objective function, semantic extraction, and the entire training process. Compared with traditional Turbo coding techniques, the Turbo-DSA shows a faster convergence speed, thanks to its efficient processing of semantic vectors. Simulation results demonstrate that the Turbo-DSA surpasses existing benchmarks in key performance indicators, such as bilingual evaluation understudy scores and sentence similarity. This is particularly evident under low signal-to-noise ratio conditions, where it shows superior text semantic transmission efficiency and adaptability to variable marine channel environments.

A comprehensive bearing prognosis framework based on piecewise function stacking convolution auto-encoder and XGBoost algorithm

Abstract Accurately forecasting the remaining useful life (RUL) stands as a pivotal and formidable task within the realm of prognostics and health management. However, there is limited research that considers integrating early fault diagnosis during the bearing’s lifecycle with the prediction of its RUL. In this article, a comprehensive bearing prognosis framework based on piecewise function stacking convolution auto-encoder (AE) and XGBoost algorithm is proposed. To achieve this, an unsupervised piecewise function-based deep stacked convolutional AE was designed to construct the health indicator (HI) of the bearing for reducing the dependency on prior knowledge and furnishing a dynamic foundation for predicting RUL. The 4σ criterion based on HI’s increment was proposed for determining the fault occurrence time (FOT) of the bearing’s operational process. Subsequently, an XGBoost algorithm model was utilized to predict the RUL of faulty bearings. The efficacy of the bearing prognosis framework was validated by two real bearing test datasets. Results indicate the employed criterion of construct HI can flexibly adjust to various operational conditions and accurately pinpoint the bearing’s FOT. Furthermore, the findings demonstrate the proposed bearing prognosis framework achieves superior performance compared with several conventional anomaly detection methods.

Advancing aquifer vulnerability mapping through integrated deep learning approaches

Groundwater vulnerability maps are crucial for safeguarding groundwater quality. A research gap exists in using advanced data fusion techniques to identify areas subject to seawater intrusion. To address this gap, this research enhances the GALDIT method and applies diverse deep learning models, combined with machine learning techniques, to improve the precision of aquifer vulnerability mapping. The new GALDITMW model incorporates the seawater mixing index and the parameters related to the production well density and aquifer porous medium. For the first time, supervised and unsupervised deep learning models, such as deep neural networks, deep belief networks, deep stacked autoencoders, and convolutional neural networks, are used for vulnerability mapping. In the second stage, the results of various machine learning models are fused to improve performance. The models' effectiveness is evaluated using a vulnerability index based on total dissolved solids (TDS) in an aquifer hydraulically connected with Salt Lake in central Iran, which faces groundwater depletion and salinization. The evaluation of the models based on performance metrics and the confusion matrix demonstrates that initial deep-learning models perform well. Significant improvements were observed in the second stage involving machine learning models, confirming their strong correlation (R2 > 0.985) with observed chloride values. The GPR model achieved an F1 score of 86.92%, an NSE of 0.911, and an RMSE reduction of 0.026 mg/L compared to the first-stage models. The proposed method offers a novel and accurate method for identifying vulnerable areas and provides helpful information for groundwater resource management.

A deep decentralized privacy-preservation framework for online social networks

This paper addresses the critical challenge of privacy in Online Social Networks (OSNs), where centralized designs compromise user privacy. We propose a novel privacy -preservation framework that integrates blockchain technology with deep learning to overcome these vulnerabilities. Our methodology employs a two-tier architecture: the first tier uses an elitism-enhanced Particle Swarm Optimization and Gravitational Search Algorithm (ePSOGSA) for optimizing feature selection, while the second tier employs an enhanced Non-symmetric Deep Autoencoder (e-NDAE) for anomaly detection. Additionally, a blockchain network secures users’ data via smart contracts, ensuring robust data protection. When tested on the NSL-KDD dataset, our framework achieves 98.79% accuracy, a 10% false alarm rate, and a 98.99% detection rate, surpassing existing methods. The integration of blockchain and deep learning not only enhances privacy protection in OSNs but also offers a scalable model for other applications requiring robust security measures.

Lassi onk: a system framework to annotate and classify vehicular honk from road traffic.

Some recent studies highlight that vehicular traffic and honking contribute to more than 50% of noise pollution in urban or sub-urban areas in developing countries, including Indian cities. Frequent honking has an adverse effect on health and hampers road safety, the environment, etc. Therefore, recognizing the various vehicle honks and classifying the honk of different vehicles can provide good insights into environmental noise pollution. Moreover, classifying honks based on vehicle types allows for the inference of contextual information of a location, area, or traffic. So far, the researchers have done outdoor sound classification and honk detection, where vehicular honks are collected in a controlled environment or in theabsence of ambient noise. Such classification models fail to classify honk based on vehicle types. Therefore, it becomes imperative to design a system that can detect and classify honks of different types of vehicles to infer some contextual information. This paper presents a novel framework lassi onk that performs raw vehicular honk sensing, data labeling, and classifies the honk into three major groups, i.e., light-weight vehicles, medium-weight vehicles, and heavy-weight vehicles. Raw audio samples of different vehicular honking are collected based on spatio-temporal characteristics and converted them into spectrogram images. A deep learning-based multi-label autoencoder model (MAE) is proposed for automated labeling of the unlabeled data samples, which provides 97.64% accuracy in contrast to existing deep learning-based data labeling methods. Further, various pre-trained models, namely Inception V3, ResNet50, MobileNet, and ShuffleNet are used and proposed an Ensembled Transfer Learning model (EnTL) for vehicle honks classification and performed comparative analysis. Results reveal that EnTL exhibits the best performance compared to pre-trained models and achieves 96.72% accuracy in our dataset. In addition, context of a location is identified based on these classified honk signatures in a city.

Forecasting dominance of SARS-CoV-2 lineages by anomaly detection using deep AutoEncoders.

The coronavirus disease of 2019 (COVID-19) pandemic is characterized by sequential emergence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) variants, lineages, and sublineages, outcompeting previously circulating ones because of, among other factors, increased transmissibility and immune escape. We propose DeepAutoCoV, an unsupervised deep learning anomaly detection system to predict future dominant lineages (FDLs). We define FDLs as viral (sub)lineages that will constitute more than 10% of all the viral sequences added to the GISAID database on a given week. DeepAutoCoV is trained and validated by assembling global and country-specific data sets from over 16 million Spike protein sequences sampled over a period of about 4 years. DeepAutoCoV successfully flags FDLs at very low frequencies (0.01% - 3%), with median lead times of 4-17 weeks, and predicts FDLs ~5 and ~25 times better than a baseline approach For example, the B.1.617.2 vaccine reference strain was flagged as FDL when its frequency was only 0.01%, more than a year before it was considered for an updated COVID-19 vaccine. Furthermore, DeepAutoCoV outputs interpretable results by pinpointing specific mutations potentially linked to increased fitness, and may provide significant insights for the optimization of public health pre-emptive intervention strategies.

Detection of Intrusions in Internet of Things Based Deep Auto Encoder Using Deepnets

Detection of Intrusions in Internet of Things Based Deep Auto Encoder Using Deepnets

Multi-dimensional weighted deep subspace clustering with feature classification

Deep subspace clustering (DSC) methods based on deep autoencoder (DAE) and self-expression layers have achieved impressive performance. However, traditional DSC method often loses useful information in the feature extraction of DAE, leading to incomplete learning of the self-expression coefficient matrix. Besides, existing deep subspace clustering methods always extract feature information from a single dimension, ignoring the potential information that may exist in the self-expression coefficient matrix across multiple dimensions. In this paper, we propose a novel method called multi-dimensional weighted deep subspace clustering based on feature classification (MWDSC). Specifically, we use the multivariate feature classification module (MFC) to learn the potential information of multiple local feature matrix for local pre-clustering. To further improve the clustering performance, we propose a self-flip weighted fusion strategy (SWF). The SWF strategy obtains a self-expression coefficient matrix with rich features through self-flipping weighting, thereby aiding in achieving global optimal solutions. Extensive experiments conducted on five benchmark datasets validate the superiority and effectiveness of our proposed method over other state-of-the-art approaches.