Denoising Autoencoder Research Articles

An Efficient Intrusion Detection Method Based on LightGBM and Autoencoder

Due to the insidious characteristics of network intrusion behaviors, developing an efficient intrusion detection system is still a big challenge, especially in the era of big data where the number of traffic and the dimension of each traffic feature are high. Because of the shortcomings of traditional common machine learning algorithms in network intrusion detection, such as insufficient accuracy, a network intrusion detection system based on LightGBM and autoencoder (AE) is proposed. The LightGBM-AE model proposed in this paper includes three steps: data preprocessing, feature selection, and classification. The LightGBM-AE model adopts the LightGBM algorithm for feature selection, and then uses an autoencoder for training and detection. When a set of data containing network intrusion behaviors are inputted into an autoencoder, there is a large reconstruction error between the original input data and the reconstructed data obtained by the autoencoder, which provides a basis for intrusion detection. According to the reconstruction error, an appropriate threshold is set to distinguish symmetrically between normal behavior and attack behavior. The experiment is carried out on the NSL-KDD dataset and implemented using Pytorch. In addition to autoencoder, variational autoencoder (VAE) and denoising autoencoder (DAE) are also used for intrusion detection and are compared with existing machine learning algorithms such as Decision Tree, Random Forest, KNN, GBDT, and XGBoost. The evaluation is carried out through classification evaluation indexes such as accuracy, precision, recall, F1-score. The experimental results show that the method can efficiently separate the attack behavior from normal behavior according to the reconstruction error. Compared with other methods, the effectiveness and superiority of this method are verified.

Power Grid Fault Diagnosis Method Based on Stacked Sparse Denoising Auto-Encoder and GRU Network

This research aims to solve the problems of feature extraction, data dimension reduction, gradient dissipation and low training efficiency in power grid fault classification. Aiming at PMU measurement data with high dimensionality and strong real-time characteristics, a fault diagnosis method based on Stacked Sparse Denoising Auto-Encoder and GRU network is proposed. The Stacked Sparse Denoising Auto-Encoder is used to reduce the sequence dimension to obtain the sparse feature expression of the data, and then the time-dependent features of the data extracted by the GRU are used to obtain the fault type. Simulations and experiments show that compared with the traditional neural network algorithm, the proposed method can effectively extract high-dimensional data features, reduce data dimensions, improve the efficiency of GRU network classification, accelerate the convergence speed and reduce the training time, and has better stability and higher accuracy.

Development of an efficient data-driven method to estimate the hydraulic properties of aquifers from groundwater level fluctuation pattern features

A method to develop a data-driven model able to estimate hydraulic properties based on groundwater level (GWL) fluctuation patterns is proposed. In particular, a preprocessing method using a denoising autoencoder (DAE) is incorporated into the proposed method to improve the performance of the developed method. DAE is applied to prepare the input variable of the estimation model by extracting informative low-dimensional features from the original high-dimensional GWL data. Before applying the proposed DAE to this study, the reliability of applying the DAE is validated. First, the ability to reduce the noise of GWL data is validated by observing that an average of 71% of the noise is reduced. Additionally, the performances of the extracted principal characteristics of the GWL data is confirmed by reasonable matches in the extracted features to the corresponding hydraulics of the aquifers. In this case, both synthetic data and actual data acquired over South Korea are applied. Based on the validated DAE results, models to estimate two types of hydraulic properties are constructed. The estimation performances of the models are quantitatively validated using the correlation coefficient between the estimated and actual hydraulic properties. Overall, the constructed models for k and α/n show an appropriate estimation accuracy with a high correlation coefficient between the actual result and estimate (0.8663 and 0.7207, respectively). Therefore, using the proposed method, the hydraulic properties of an un-informed aquifer can be effectively inferred given GWL data without conducting field experiments (e.g., pumping tests). The proposed method is promising for efficient evaluations of the physical hydraulics of un-informed aquifers and, therefore, can be used as an effective tool to manage groundwater resources.

DEEP LEARNING-BASED APPROACH FOR MISSING DATA IMPUTATION

The missing values in the datasets are a problem that will decrease the machine learning performance. New methods are recommended every day to overcome this problem. The methods of statistical, machine learning, evolutionary and deep learning methods are among these methods. Although deep learning is one of the popular subjects of today, there are limited studies in the missing data imputation. Several deep learning techniques have been used to handling missing data, one of them is the auto-encoder and its denoising and stacked variants. In this study, the missing value in three different real-world datasets was estimated by using denoising auto-encoder (DAE), k-nearest neighbor (kNN) and multivariate imputation by chained equations (MICE) methods. The estimation success of the methods was compared according to the root mean square error (RMSE) criterion. It was observed that the DAE method was more successful than other methods in estimating the missing values.

Online dynamic security assessment of wind integrated power system using SDAE with SVM ensemble boosting learner

The increasing trend of decarbonization, particularly high penetration of wind power generation (WPG), has resulted in an increased operational uncertainty of the power system. This emerging trend of wind integration requires novel and effective methods of power system security assessment. This paper proposes a method for dynamic security assessment (DSA) based upon stacked de-noising auto-encoder (SDAE) using support vector machine (SVM) ensemble with boosting learning approach. A two-step feature reduction method is introduced after screening the redundant features among original feature set using mutual information theory. Multi-level features of original input data are extracted using multi-layer SDAE. SVM ensemble classifier is used to perform classification with data from all hidden layers of SDAE. The output of multi-layer SDAE and SVM ensemble are combined together as an input to ensemble boosting learning method which gives the final DSA result. The proposed method is tested on modified New England 39 bus system and extended on simplified AC/DC hybrid power system of Shandong province, China. The effectiveness and accuracy of the proposed method is demonstrated by simulation results and by comparisons with other methods.

Convolutional neural network-based dosimetry evaluation of esophageal radiation treatment planning.

A dosimetry evaluation model for treatment planning of esophageal radiation therapy is developed using a deep learning model. The model predicts dose volume histogram (DVH) from distance to target histogram (DTH) based on stacked de-noise auto-encoder (SDAE) and one-dimensional convolutional network (1D-CN). First, SDAE is used to extract the features from the curves of DTH and DVH. Then 1D-CN model is employed to learn the relationship between the features of DTH and DVH, and later used to predict the features of DVH from the features of DTH. Finally, the curve of DVH is restored from the features of DVH based on SDAE. Two hundred and seventy treatment plans are used for training 1D-CN and another sixty-three treatment plans are used for evaluating this model. This method is also compared with another two popular prediction methods based on support vector machine (SVM) and U-net. Based on the experimental result, the proposed model achieves the lowest dose endpoint error comparing to the other models. The average prediction error on planned target volume, left lung, right lung, heart, and spinal cord is 2.94% for the proposed model, while the average prediction errors are 6.79% and 3.41% for SVM and U-net, respectively. A dosimetry evaluation method based on SDAE and 1D-CN is developed in characterizing the correlation relationship between DTH and DVH of treatment plans. The results show that the model could be trained more efficiently in this framework and the DVH could be predicted with higher accuracy comparing to those existing methods. It provides a useful tool in supporting automated treatment planning of esophageal intensity-modulated radiotherapy.

Low-Order Spherical Harmonic HRTF Restoration Using a Neural Network Approach

Spherical harmonic (SH) interpolation is a commonly used method to spatially up-sample sparse head related transfer function (HRTF) datasets to denser HRTF datasets. However, depending on the number of sparse HRTF measurements and SH order, this process can introduce distortions into high frequency representations of the HRTFs. This paper investigates whether it is possible to restore some of the distorted high frequency HRTF components using machine learning algorithms. A combination of convolutional auto-encoder (CAE) and denoising auto-encoder (DAE) models is proposed to restore the high frequency distortion in SH-interpolated HRTFs. Results were evaluated using both perceptual spectral difference (PSD) and localisation prediction models, both of which demonstrated significant improvement after the restoration process.

A spatiotemporal approach for traffic data imputation with complicated missing patterns

With the advent of intelligent transportation systems (ITS), spatiotemporal traffic data has gained growing importance in real-time monitoring, prediction, and control of traffic. However, in practical implementations, data collection devices are often faced with malfunctions caused by various unpredictable disruptions, thereby resulting in the so-called “missing value problems.” In realistic cases, the disruptions to the data collection devices are often associated with some key events (e.g., power cut and natural disasters), in addition, along with other disruptions the missing value problem could be in a complicated manner with both randomly and completely missing patterns. To perform the imputation task with such complicated missing patterns, we propose a hybrid spatiotemporal method which utilizes the time series properties by “prophet” model and captures the spatial residuals information by iterative random forest model. The spatiotemporal method first applies the temporal part to fill the missing value and then adopts the spatial part to acquire the residual component of the missing values. The results of the two components are integrated into the final imputations. Based on the PeMS freeway dataset (PeMS, 2019) and an urban road dataset under extensive artificially designed scenarios like randomly, clustered non-completely and completely missing patterns, we test our proposed approach with some existing techniques such as K-Nearest Neighbor (KNN), Seasonal-Trend decomposition using Loess (STL), Bayesian tensor decomposition, Denoising AutoEncoder (DAE). The test results indicate that the hybrid method achieves the best imputation quality for most missing patterns, particularly for those with completely or hybrid missing patterns. Furthermore, the hybrid model still performs well under extreme missing rates as high as 0.9, which validates the robustness of the model in extreme situations.

Denoising Autoencoder, A Deep Learning Algorithm, Aids the Identification of A Novel Molecular Signature of Lung Adenocarcinoma

Precise biomarker development is a key step in disease management. However, most of the published biomarkers were derived from a relatively small number of samples with supervised approaches. Recent advances in unsupervised machine learning promise to leverage very large datasets for making better predictions of disease biomarkers. Denoising autoencoder (DA) is one of the unsupervised deep learning algorithms, which is a stochastic version of autoencoder techniques. The principle of DA is to force the hidden layer of autoencoder to capture more robust features by reconstructing a clean input from a corrupted one. Here, a DA model was applied to analyze integrated transcriptomic data from 13 published lung cancer studies, which consisted of 1916 human lung tissue samples. Using DA, we discovered a molecular signature composed of multiple genes for lung adenocarcinoma (ADC). In independent validation cohorts, the proposed molecular signature is proved to be an effective classifier for lung cancer histological subtypes. Also, this signature successfully predicts clinical outcome in lung ADC, which is independent of traditional prognostic factors. More importantly, this signature exhibits a superior prognostic power compared with the other published prognostic genes. Our study suggests that unsupervised learning is helpful for biomarker development in the era of precision medicine.

Automatic EEG eyeblink artefact identification and removal technique using independent component analysis in combination with support vector machines and denoising autoencoder

This study proposes a novel combination of independent component analysis (ICA) in conjunction with support vector machine (SVM) and denoising autoencoder (DA), for the first time, for removal of eyeblink artefacts from the corrupted electroencephalography (EEG). At first the eyeblink corrupted EEG signals are decomposed into independent components (ICs) using ICA, the corrupted-ICs are then identified using SVM as a classifier. From the corrupted-ICs, the artefacted segment is identified with a second SVM classifier and corrected by the pre-trained DA. Finally, inverse-ICA operation is applied on the remaining ICs and the corrected ICs to obtain the artefact-free EEG signal. The proposed methodology modifies only the portion corrupted with artefacts, and does not alter the uncorrupted part, thereby preserving the neural information in the original EEG. The proposed methodology was implemented to remove eyeblinks from the EEG data collected from the publicly available EEGLab data set. The results reveal that the proposed methodology is superior to the other recently reported methods in terms of the mutual information and average correlation coefficient. Further, the proposed method is automatic and does not require any intervention of the operator, whereas the other methods require intervention of the user.

Stability Analysis of Denoising Autoencoders Based on Dynamical Projection System

In this study, we give a stability analysis of denoising autoencoder(DAE) from the novel perspective of dynamical systems when the input density is defined as a distribution on a manifold. We demonstrate the connection between the corrupted distribution and the learned reconstruction function of a nonlinear DAE, which motivates the use of a dynamic projection system (DPS) associated with the learned reconstruction function. Utilizing the constructed DPS, we prove that the high-density region of the corrupted data distribution asymptotically converges to the data manifold. Then, we show that the region is the attracting stable equilibrium manifold of the DPS which is completely stable. These results serve a theoretical basis of the DAE in recognizing the high-density region of the highly corrupted data with large deviations through the DPS. The effectiveness of this analysis is verified by conducting experiments on several toy examples and real image datasets with various types of noise.

A threshold self-setting condition monitoring scheme for wind turbine generator bearings based on deep convolutional generative adversarial networks

Long-term reliable health condition monitoring (HCM) of a wind turbine is an essential method to avoid catastrophic failure results. Existing unsupervised learning methods, such as auto-encoder (AE) and de-noising auto-encoder (DAE) models, are utilized to the condition monitoring of wind turbines. The critical bottleneck of these models for monitoring is to determine a threshold for identifying different health conditions. Unfortunately, the threshold is usually set up with different kinds of calculation methods or even based on experience. Therefore, the uncertainty of the threshold will inevitably influence the accuracy of the monitoring process and may lead to misdiagnoses. To overcome this limitation, this research introduces a threshold self-setting HCM scheme, based on deep convolutional generative adversarial networks (DCGAN) and employed for defining a self-setting threshold to monitor wind turbine generator bearings. A threshold for HCM can be automatically created through the output of the G network in the DCGAN model, and the challenging problem of setting up a threshold can be solved. Besides, the use of Nash Equilibrium for training enables this scheme to become self-defined evaluators with a high level of consistency, without any human intervention and can be treated as a self-defined threshold, and it is a model self-tuning process. Furthermore, a sample discrepancy analysis based on the output of the G network is utilized so that a quantitative indicator of the fault severity in wind turbine generator bearings are provided. By tracking a real wind turbine dataset from the LU NAN wind farm in China, the effectiveness of the proposed scheme is verified.

Enhancing Bitcoin Price Fluctuation Prediction Using Attentive LSTM and Embedding Network

Bitcoin has attracted extensive attention from investors, researchers, regulators, and the media. A well-known and unusual feature is that Bitcoin’s price often fluctuates significantly, which has however received less attention. In this paper, we investigate the Bitcoin price fluctuation prediction problem, which can be described as whether Bitcoin price keeps or reversals after a large fluctuation. In this paper, three kinds of features are presented for the price fluctuation prediction, including basic features, traditional technical trading indicators, and features generated by a Denoising autoencoder. We evaluate these features using an Attentive LSTM network and an Embedding Network (ALEN). In particular, an attentive LSTM network can capture the time dependency representation of Bitcoin price and an embedding network can capture the hidden representations from related cryptocurrencies. Experimental results demonstrate that ALEN achieves superior state-of-the-art performance among all baselines. Furthermore, we investigate the impact of parameters on the Bitcoin price fluctuation prediction problem, which can be further used in a real trading environment by investors.

Combined Denoising and Suppression of Transient Artifacts in Arterial Spin Labeling MRI Using Deep Learning.

Arterial spin labeling (ASL) is a useful tool for measuring cerebral blood flow (CBF). However, due to the low signal-to-noise ratio (SNR) of the technique, multiple repetitions are required, which results in prolonged scan times and increased susceptibility to artifacts. To develop a deep-learning-based algorithm for simultaneous denoising and suppression of transient artifacts in ASL images. Retrospective. 131 pediatric neuro-oncology patients for model training and 11 healthy adult subjects for model evaluation. 3T / pseudo-continuous and pulsed ASL with 3D gradient-and-spin-echo readout. A denoising autoencoder (DAE) model was designed with stacked encoding/decoding convolutional layers. Reference standard images were generated by averaging 10 pairwise ASL subtraction images. The model was trained to produce perfusion images of a similar quality using a single subtraction image. Performance was compared against Gaussian and non-local means (NLM) filters. Evaluation metrics included SNR, peak SNR (PSNR), and structural similarity index (SSIM) of the CBF images, compared to the reference standard. One-way analysis of variance (ANOVA) tests for group comparisons. The DAE model was the only model to produce a significant increase in SNR compared to the raw images (P < 0.05), providing an average SNR gain of 62%. The DAE model was also effective at suppressing transient artifacts, and was the only model to show a significant improvement in accuracy in the generated CBF images, as assessed using PSNR values (P < 0.05). In addition, using data from multiple inflow time acquisitions, the DAE images produced the best fit to the Buxton kinetic model, offering a 75% reduction in the fitting error compared to the raw images. Deep-learning-based algorithms provide superior accuracy when denoising ASL images, due to their ability to simultaneously increase SNR and suppress artifactual signals in raw ASL images. 3 TECHNICAL EFFICACY STAGE: 1.

Network Structural Transformation-Based Community Detection with Autoencoder

In this paper, we proposed a novel community detection method based on the network structure transformation, that utilized deep learning. The probability transfer matrix of the network adjacency matrix was calculated, and the probability transfer matrix was used as the input of the deep learning network. We use a denoising autoencoder to nonlinearly map the probability transfer matrix into a new sub space. The community detection was calculated with the deep learning nonlinear transform of the network structure. The network nodes were clustered in the new space with the K-means clustering algorithm. The division of the community structure was obtained. We conducted extensive experimental tests on the benchmark networks and the standard networks (known as the initial division of communities). We tested the clustering results of the different types, and compared with the three base algorithms. The results showed that the proposed community detection model was effective. We compared the results with other traditional community detection methods. The empirical results on datasets of varying sizes demonstrated that our proposed method outperformed the other community detection methods for this task.

CLPM: A Cooperative Link Prediction Model for Industrial Internet of Things Using Partitioned Stacked Denoising Autoencoder

With the development of Industry 4.0, an increasing number of industrial Internet of Things (IIoT) mobile devices (MD), which constantly transmit data at any time, are working on the production line. However, due to node movement, signal attenuation, or physical obstacles, data must rely on the transmission of relay nodes to finally reach the destination node. Based on this scenario, in this article, we propose a cooperative link prediction model (CLPM) using a stacked denoising autoencoder (SDAE) to predict links of the IIoT-based MDs at the next moment through historical link information. The layer structure of the SDAE model is partitioned so that the local MD and edge servers can cooperatively process the link prediction tasks. Experimental results show that our proposed CLPM outperforms others in terms of prediction performance and execution delay.

EEG Connectivity Analysis Using Denoising Autoencoders for the Detection of Dyslexia.

The Temporal Sampling Framework (TSF) theorizes that the characteristic phonological difficulties of dyslexia are caused by an atypical oscillatory sampling at one or more temporal rates. The LEEDUCA study conducted a series of Electroencephalography (EEG) experiments on children listening to amplitude modulated (AM) noise with slow-rythmic prosodic (0.5-1[Formula: see text]Hz), syllabic (4-8[Formula: see text]Hz) or the phoneme (12-40[Formula: see text]Hz) rates, aimed at detecting differences in perception of oscillatory sampling that could be associated with dyslexia. The purpose of this work is to check whether these differences exist and how they are related to children's performance in different language and cognitive tasks commonly used to detect dyslexia. To this purpose, temporal and spectral inter-channel EEG connectivity was estimated, and a denoising autoencoder (DAE) was trained to learn a low-dimensional representation of the connectivity matrices. This representation was studied via correlation and classification analysis, which revealed ability in detecting dyslexic subjects with an accuracy higher than 0.8, and balanced accuracy around 0.7. Some features of the DAE representation were significantly correlated ([Formula: see text]) with children's performance in language and cognitive tasks of the phonological hypothesis category such as phonological awareness and rapid symbolic naming, as well as reading efficiency and reading comprehension. Finally, a deeper analysis of the adjacency matrix revealed a reduced bilateral connection between electrodes of the temporal lobe (roughly the primary auditory cortex) in DD subjects, as well as an increased connectivity of the F7 electrode, placed roughly on Broca's area. These results pave the way for a complementary assessment of dyslexia using more objective methodologies such as EEG.

SMPLR: Deep learning based SMPL reverse for 3D human pose and shape recovery

In this paper we propose to embed SMPL within a deep-based model to accurately estimate 3D pose and shape from a still RGB image. We use CNN-based 3D joint predictions as an intermediate representation to regress SMPL pose and shape parameters. Later, 3D joints are reconstructed again in the SMPL output. This module can be seen as an autoencoder where the encoder is a deep neural network and the decoder is SMPL model. We refer to this as SMPL reverse (SMPLR). By implementing SMPLR as an encoder-decoder we avoid the need of complex constraints on pose and shape. Furthermore, given that in-the-wild datasets usually lack accurate 3D annotations, it is desirable to lift 2D joints to 3D without pairing 3D annotations with RGB images. Therefore, we also propose a denoising autoencoder (DAE) module between CNN and SMPLR, able to lift 2D joints to 3D and partially recover from structured error. We evaluate our method on SURREAL and Human3.6M datasets, showing improvement over SMPL-based state-of-the-art alternatives by about 4 and 12 mm, respectively.

Combining Denoising Autoencoders and Dynamic Programming for Acoustic Detection and Tracking of Underwater Moving Targets.

Accurate detection and tracking of moving targets in underwater environments pose significant challenges, because noise in acoustic measurements (e.g., SONAR) makes the signal highly stochastic. In continuous marine monitoring a further challenge is related to the computational complexity of the signal processing pipeline—due to energy constraints, in off-shore monitoring platforms algorithms should operate in real time with limited power consumption. In this paper, we present an innovative method that allows to accurately detect and track underwater moving targets from the reflections of an active acoustic emitter. Our system is based on a computationally- and energy-efficient pre-processing stage carried out using a deep convolutional denoising autoencoder (CDA), whose output is then fed to a probabilistic tracking method based on the Viterbi algorithm. The CDA is trained on a large database of more than 20,000 reflection patterns collected during 50 designated sea experiments. System performance is then evaluated on a controlled dataset, for which ground truth information is known, as well as on recordings collected during different sea experiments. Results show that, compared to the benchmark, our method achieves a favorable trade-off between detection and false alarm rate, as well as improved tracking accuracy.

PV-DAE: A hybrid model for deceptive opinion spam based on neural network architectures

Opinion review is of great importance for both customers and organizations. Indeed, it helps customers in buying decisions and represents a valuable feedback for the companies, allowing them to improve their productions. However, numerous greedy companies resort to fake reviews in order to influence the customer and brighten the brand image, or to defame the one of their competitors. Various models are proposed in order to detect deceptive opinion reviews. Most of these models adopt traditional methods focusing on feature extraction and traditional classifiers. Unfortunately, these models do not capture the semantic aspect while ignoring the opinion’s context. In order to tackle this issue, we propose a new approach based on Paragraph Vector Distributed Bag of Words (PV-DBOW) and the Denoising Autoencoder (DAE). The proposed customized model provides a strong representation which is based on a global representation of the opinions while preserving their semantics. Indeed, the embedding vectors capture the semantic meaning of all words in the context of each opinion. The generated review representations are fed into a fully connected neural network in order to detect deceptive opinion spam. The obtained results concerning the deception dataset show that our model is effective and outperforms the existing state-of-the-art methodologies.