Denoising Autoencoder Research Articles

Identification of Cancer Mediating Biomarkers using Stacked Denoising Autoencoder Model - An Application on Human Lung Data

In this work, we form stacked denoising auto encoder model which is recognized few feasible genes mediating human lung adenocarcinoma. At first we have trained the data for feature selection by using Stacked Denoising Auto-encoder (SDAE) and for backpropagation we have used Multilayer Perceptron (MLP) procedure. We said this model is MLP-SDAE. The process include classification of genes according to correlation coefficient value and select few feasible genes. The superiority of the method has been established some present gene selection procedures like Support Vector Machine (SVM), Significance Analysis of Microarry (SAM), Bayesian Regularization (BR), Neighborhood Analysis (NA), and Gaussian Mixture Model (GMM). The MLP-SDAE model has been effectively used to one human lung microarray gene expression data. The result are appropriately verify by preliminary analysis, t-test, and gene expression profile plots. In this method, we have established more number of true positive genes then another present procedures.

Early Warning of Critical Blockage in Coal Mills Based on Stacked Denoising Autoencoders

Coal mills have a significant influence on the reliability, efficiency, and safe operation of a coal-fired power plant. Coal blockage is one of the main reasons for coal mill malfunction. It is highly essential to accurately detect the critical blockage in coal mills to ensure a safe and stable operation of the unit. Taking advantage of unsupervised learning methods and historical process data from distributed control systems (DCS), a stacked denoising autoencoder (SDAE) network–based approach for monitoring critical blockage in a coal mill is proposed in this work. The SDAE model with optimized parameters is applied to reconstruct the operating data during normal operating conditions. The intrinsic relationship between all input variables was captured by training a multilayer network model. The monitoring indicator was obtained from the reconstruction errors, and the threshold for monitoring indicators was obtained using kernel density estimation (KDE) during normal operation. The proposed network is independent of fault data, requires a reduced on-line calculation, and demonstrates a better real-time performance compared to conventional methods. The abnormal variables analysis may provide a theoretical evidence for critical blockage. The effectiveness of the proposed method is validated using operating data collected from an actual coal-fired power plant in China. The results demonstrated that the proposed method can effectively detect critical blockage in a coal mill and issue a timely warning, which allows operators to detect potential faults.

An Attention Guided Semi-Supervised Learning Mechanism to Detect Electricity Frauds in the Distribution Systems

Electricity theft is one of the main causes of non-technical losses and its detection is important for power distribution companies to avoid revenue loss. The advancement of traditional grids to smart grids allows a two-way flow of information and energy that enables real-time energy management, billing and load surveillance. This infrastructure enables power distribution companies to automate electricity theft detection (ETD) by constructing new innovative data-driven solutions. Whereas, the traditional ETD approaches do not provide acceptable theft detection performance due to high-dimensional imbalanced data, loss of data relationships during feature extraction and the requirement of experts’ involvement. Hence, this paper presents a new semi-supervised solution for ETD, which consists of relational denoising autoencoder (RDAE) and attention guided (AG) TripleGAN, named as RDAE-AG-TripleGAN. In this system, RDAE is implemented to derive features and their associations while AG performs feature weighting and dynamically supervises the AG-TripleGAN. As a result, this procedure significantly boosts the ETD. Furthermore, to demonstrate the acceptability of the proposed methodology over conventional approaches, we conducted extensive simulations using the real power consumption data of smart meters. The proposed solution is validated over the most useful and suitable performance indicators: area under the curve, precision, recall, Matthews correlation coefficient, F1-score and precision-recall area under the curve. The simulation results prove that the proposed method efficiently improves the detection of electricity frauds against conventional ETD schemes such as extreme gradient boosting machine and transductive support vector machine. The proposed solution achieves the detection rate of 0.956, which makes it more acceptable for electric utilities than the existing approaches.

A Novel Acoustic Emission Sources Localization and Identification Method in Metallic Plates Based on Stacked Denoising Autoencoders

Nowadays, deep learning could be an alternative approach to crack characterization. However, to the best of the authors’ knowledge, little research exists on a deep learning-based characterization of fatigue-related AE sources occurring in plate-like structures. Consequently, this paper introduces a stacked denoising autoencoders (SDAE)-based framework to localize acoustic emission (AE) sources in common and complex metallic panels. The experimental specimen are respectively a Q235B steel plate and a 316L stainless steel containing a laser cladding layer. Specifically, SDAE is pre-trained and utilized to localize AE sources that are simulated by using the classical pencil lead break (PLB) approach. Meanwhile, the number of layers and hidden nodes of SDAE used for coordinate-based location is optimized according to a Bayesian Information Criteria (BIC) approach. To validate the proposed network and simplify the analysis, experiments are carried out on the surface of plate-like structures, which only one sensor is applied. After identifying AE sources that occur near laser cladding layers, the proposed approach classifies them into four source-to-laser cladding layer distance categories. Particularly, a ten-fold cross-validation method is utilized to improve the accuracy of localization in this paper. Moreover, the effectiveness analysis to the number of sensors and comparison with conventional machine learning methods, including support vector machine (SVM) and artificial neural network (ANN), are also evaluated. In order to validate the performance of the proposed approach in terms of coordinate-based source localization. Ultimately, the results demonstrate that 100% accuracy for zonal localization, and the root mean squared (RMS) localization errors of two metallic panels are 38 mm (1.5”) and 48 mm (1.9”), respectively. Additionally, in comparison with conventional machine learning approaches (i.e. SVM and ANN) which the RMS errors were 78 mm (2.5”) and 67 mm (2.1”), respectively, the coordinates-based localization accuracy is significantly improved using the proposed approach. The results demonstrate the proposed approach is effective in AE-based structural health monitoring of plate-like structures with single-sensor.

Frame Prediction- Noise Removal using Denoising Autoencoders

In the current times, tasks like Object Detection, Object tracking, Gesture prediction, Video prediction in computer vision are being solved effectively with models of deep learning . Video frame prediction involves predicting the next few frames of a video given the previous frame or frames as input. Currently, the challenge in video frame prediction is that the predicted future frames are blurry. This paper focuses on the removal of noise from the predicted image using Denoising Autoencoders, solve the above-addressed issue. The proposed work, trains LSTM model which generates future frames by giving a sequence of input frames. The predicted output is given as an input to the Denoising Autoencoders which tries to remove the blurry predictions. Our approach is implemented on Moving MNIST Dataset. The result of our proposed method improved accuracy and is compared with the accuracy of Denoising Autoencoders, LSTM, and LSTM along with Denoising Autoencoders.

Benchmarking Daily Line Loss Rates of Low Voltage Transformer Regions in Power Grid Based on Robust Neural Network

Line loss is inherent in transmission and distribution stages, which can cause certain impacts on the profits of power-supply corporations. Thus, it is an important indicator and a benchmark value of which is needed to evaluate daily line loss rates in low voltage transformer regions. However, the number of regions is usually very large, and the dataset of line loss rates contains massive outliers. It is critical to develop a regression model with both great robustness and efficiency when trained on big data samples. In this case, a novel method based on robust neural network (RNN) is proposed. It is a multi-path network model with denoising auto-encoder (DAE), which takes the advantages of dropout, L2 regularization and Huber loss function. It can achieve several different outputs, which are utilized to compute benchmark values and reasonable intervals. Based on the comparison results, the proposed RNN possesses both superb robustness and accuracy, which outperforms the testing conventional regression models. According to the benchmark analysis, there are about 13% outliers in the collected dataset and about 45% regions that hold outliers within a month. Hence, the quality of line loss rate data should still be further improved.

Feature extraction and classification of heart sound using 1D convolutional neural networks

We proposed a one-dimensional convolutional neural network (CNN) model, which divides heart sound signals into normal and abnormal directly independent of ECG. The deep features of heart sounds were extracted by the denoising autoencoder (DAE) algorithm as the input feature of 1D CNN. The experimental results showed that the model using deep features has stronger anti-interference ability than using mel-frequency cepstral coefficients, and the proposed 1D CNN model has higher classification accuracy precision, higher F-score, and better classification ability than backpropagation neural network (BP) model. In addition, the improved 1D CNN has a classification accuracy rate of 99.01%.

PERSIANN-CNN: Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks–Convolutional Neural Networks

Abstract Accurate and timely precipitation estimates are critical for monitoring and forecasting natural disasters such as floods. Despite having high-resolution satellite information, precipitation estimation from remotely sensed data still suffers from methodological limitations. State-of-the-art deep learning algorithms, renowned for their skill in learning accurate patterns within large and complex datasets, appear well suited to the task of precipitation estimation, given the ample amount of high-resolution satellite data. In this study, the effectiveness of applying convolutional neural networks (CNNs) together with the infrared (IR) and water vapor (WV) channels from geostationary satellites for estimating precipitation rate is explored. The proposed model performances are evaluated during summer 2012 and 2013 over central CONUS at the spatial resolution of 0.08° and at an hourly time scale. Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks (PERSIANN)–Cloud Classification System (CCS), which is an operational satellite-based product, and PERSIANN–Stacked Denoising Autoencoder (PERSIANN-SDAE) are employed as baseline models. Results demonstrate that the proposed model (PERSIANN-CNN) provides more accurate rainfall estimates compared to the baseline models at various temporal and spatial scales. Specifically, PERSIANN-CNN outperforms PERSIANN-CCS (and PERSIANN-SDAE) by 54% (and 23%) in the critical success index (CSI), demonstrating the detection skills of the model. Furthermore, the root-mean-square error (RMSE) of the rainfall estimates with respect to the National Centers for Environmental Prediction (NCEP) Stage IV gauge–radar data, for PERSIANN-CNN was lower than that of PERSIANN-CCS (PERSIANN-SDAE) by 37% (14%), showing the estimation accuracy of the proposed model.

Examining the Mapping Functions of Denoising Autoencoders in Singing Voice Separation

The goal of this article is to investigate what singing voice separation approaches based on neural networks learn from the data. We examine the mapping functions of neural networks based on the denoising autoencoder (DAE) model that are conditioned on the mixture magnitude spectra. To approximate the mapping functions, we propose an algorithm inspired by the knowledge distillation, denoted the neural couplings algorithm (NCA). The NCA yields a matrix that expresses the mapping of the mixture to the target source magnitude information. Using the NCA, we examine the mapping functions of three fundamental DAE-based models in music source separation; one with single-layer encoder and decoder, one with multi-layer encoder and single-layer decoder, and one using skip-filtering connections (SF) with a single-layer encoding and decoding. We first train these models with realistic data to estimate the singing voice magnitude spectra from the corresponding mixture. We then use the optimized models and test spectral data as input to the NCA. Our experimental findings show that approaches based on the DAE model learn scalar filtering operators, exhibiting a predominant diagonal structure in their corresponding mapping functions, limiting the exploitation of inter-frequency structure of music data. In contrast, skip-filtering connections are shown to assist the DAE model in learning filtering operators that exploit richer inter-frequency structures.

Deep Neural Networks for Recommender Systems

The data available online, helps users to get information about anything of his/her interest. But since the data is huge and complex it is difficult to get useful information from it. Recommender Systems are effective software techniques to overcome this problem. Based on the user’s and item’s information available, these techniques provide recommendations to users in their area of interest. Recommender systems have wide applications like providing suggestive list of items to customers for online shopping, recommending articles or books for online reading, movie or music recommendations, news recommendations etc. In this paper we provide a study of Deep Neural Networks (DNN) approaches that can be used for recommender systems. They have been used widely in last decade in many fields like image processing, video streaming, Natural Language Processing etc. including recommendations to overcome the drawbacks of traditional systems. The paper also provides performance of Denoising AutoEncoders (DAE) which are feed forward neural networks and its comparison with traditional systems. Denoising Autoencoders are a type of autoencoders wherein some part of input is corrupted, i.e., noise is added to the input. While learning to remove noise from input, the DAE also learns to predict unknown values. This property of Denoising Autoencoders can help in recommendation systems to predict unknown values before recommending new items. Experimentation has shown improvement in the performance of recommendation systems with denoising autoencoders. The evaluation is performed on MovieLens-1M dataset with and without additional features of users (age and gender) and items (movie genres) provided in the dataset.

Augmented Autoencoders: Implicit 3D Orientation Learning for 6D Object Detection

We propose a real-time RGB-based pipeline for object detection and 6D pose estimation. Our novel 3D orientation estimation is based on a variant of the Denoising Autoencoder that is trained on simulated views of a 3D model using Domain Randomization. This so-called Augmented Autoencoder has several advantages over existing methods: It does not require real, pose-annotated training data, generalizes to various test sensors and inherently handles object and view symmetries. Instead of learning an explicit mapping from input images to object poses, it provides an implicit representation of object orientations defined by samples in a latent space. Our pipeline achieves state-of-the-art performance on the T-LESS dataset both in the RGB and RGB-D domain. We also evaluate on the LineMOD dataset where we can compete with other synthetically trained approaches. We further increase performance by correcting 3D orientation estimates to account for perspective errors when the object deviates from the image center and show extended results. Our code is available here https://github.com/DLR-RM/AugmentedAutoencoder.

DEVDAN: Deep evolving denoising autoencoder

The Denoising Autoencoder (DAE) enhances the flexibility of data stream method in exploiting unlabeled samples. Nonetheless, the feasibility of DAE for data stream analytic deserves in-depth study because it characterizes a fixed network capacity which cannot adapt to rapidly changing environments. Deep evolving denoising autoencoder (DEVDAN), is proposed in this paper. It features an open structure in the generative phase and the discriminative phase where the hidden units can be automatically added and discarded on the fly. The generative phase refines the predictive performance of discriminative model exploiting unlabeled data. Furthermore, DEVDAN is free of the problem-specific threshold and works fully in the single-pass learning fashion. We show that DEVDAN can find competitive network architecture compared with state-of-the-art methods on the classification task using ten prominent datasets simulated under the prequential test-then-train protocol.

Study of Star/Galaxy Classification Based on the XGBoost Algorithm

Machine learning has achieved great success in many areas today. The lifting algorithm has a strong ability to adapt to various scenarios with a high accuracy, and has played a great role in many fields. But in astronomy, the application of lifting algorithms is still rare. In response to the low classification accuracy of the dark star/galaxy source set in the Sloan Digital Sky Survey (SDSS), a new research result of machine learning, eXtreme Gradient Boosting (XGBoost), has been introduced. The complete photometric data set is obtained from the SDSS-DR7, and divided into a bright source set and a dark source set according to the star magnitude. Firstly, the ten-fold cross-validation method is used for the bright source set and the dark source set respectively, and the XGBoost algorithm is used to establish the star/galaxy classification model. Then, the grid search and other methods are used to adjust the XGBoost parameters. Finally, based on the galaxy classification accuracy and other indicators, the classification results are analyzed, by comparing with the models of function tree (FT), Adaptive boosting (Adaboost), Random Forest (RF), Gradient Boosting Decision Tree (GBDT), Stacked Denoising AutoEncoders (SDAE), and Deep Belief Nets (DBN). The experimental results show that, the XGBoost improves the classification accuracy of galaxies in the dark source classification by nearly 10% as compared to the function tree algorithm, and improves the classification accuracy of sources with the darkest magnitudes in the dark source set by nearly 5% as compared to the function tree algorithm. Compared with other traditional machine learning algorithms and deep neural networks, the XGBoost also has different degrees of improvement.

An Improved Stacked Denoise Autoencoder with Elu Activation Function for Traffic Data Imputation

Traffic data plays a major role in transport related applications. The problem of missing data has greatly impact the performance of Intelligent transportation systems(ITS). In this work impute the missing traffic data with spatio-temporal exploitation for high precision result under various missing rates. Deep learning based stacked denoise autoencoder is proposed with efficient Elu activation function to remove noise and impute the missing value.This imputed value will be used in analyses and prediction of vehicle traffic. Results are discussed that the proposed method outperforms well in state of the art approaches.

Highly undersampled magnetic resonance imaging reconstruction using autoencoding priors.

Although recent deep learning methodologies have shown promising results in fast MR imaging, how to explore it to learn an explicit prior and leverage it into the observation constraint is still desired. A denoising autoencoder (DAE) network is leveraged as an explicit prior to address the highly undersampling MR image reconstruction problem. First, inspired by the observation that the prior information learned from high-dimension signals is more effective than that from the low-dimension counterpart in image restoration tasks, we train the network in a multichannel scenario and apply the learned network to single-channel image reconstruction by a variables augmentation technique. Second, because of the fact that multiple implementations of artificial noise generation in DAE favors a better underlying result, we introduce a 2-sigma rule to complement each other for improving the final reconstruction. The whole algorithm is tackled by proximal gradient descent. Experimental results under varying sampling trajectories and acceleration factors consistently demonstrate the superiority of the enhanced autoencoding priors, in terms of peak signal-to-noise ratio, structural similarity, and high-frequency error norm. A simple and effective way to incorporate the DAE prior into highly undersampling MR reconstruction is proposed. Once the DAE prior is obtained, it can be applied to the reconstruction tasks with different sampling trajectories and acceleration factors, and achieves superior performance in comparison with state-of-the-art methods.

Mitigating the Impacts of Covert Cyber Attacks in Smart Grids Via Reconstruction of Measurement Data Utilizing Deep Denoising Autoencoders

As one of the most diversified cyber-physical systems, the smart grid has become more decumbent to cyber vulnerabilities. An intelligently crafted, covert, data-integrity assault can insert biased values into the measurements collected by a sensor network, to elude the bad data detector in the state estimator, resulting in fallacious control decisions. Thus, such an attack can compromise the secure and reliable operations of smart grids, leading to power network disruptions, economic loss, or a combination of both. To this end, in this paper, we propose a novel idea for the reconstruction of sensor-collected measurement data from power networks, by removing the impacts of the covert data-integrity attack. The proposed reconstruction scheme is based on a latterly developed, unsupervised learning algorithm called a denoising autoencoder, which learns about the robust nonlinear representations from the data to root out the bias added into the sensor measurements by a smart attacker. For a robust, multivariate reconstruction of the attacked measurements from multiple sensors, the denoising autoencoder is used. The proposed scheme was evaluated utilizing standard IEEE 14-bus, 39-bus, 57-bus, and 118-bus systems. Simulation results confirm that the proposed scheme can handle labeled and non-labeled historical measurement data and results in a reasonably good reconstruction of the measurements affected by attacks.

MRI super-resolution reconstruction for MRI-guided adaptive radiotherapy using cascaded deep learning: In the presence of limited training data and unknown translation model.

Deep learning (DL)-based super-resolution (SR) reconstruction for magnetic resonance imaging (MRI) has recently been receiving attention due to the significant improvement in spatial resolution compared to conventional SR techniques. Challenges hindering the widespread implementation of these approaches remain, however. Low-resolution (LR) MRIs captured in the clinic exhibit complex tissue structures obfuscated by noise that are difficult for a simple DL framework to handle. Moreover, training a robust network for a SR task requires abundant, perfectly matched pairs of LR and high-resolution (HR) images that are often unavailable or difficult to collect. The purpose of this study is to develop a novel SR technique for MRI based on the concept of cascaded DL that allows for the reconstruction of high-quality SR images in the presence of insufficient training data, an unknown translation model, and noise. The proposed framework, based on the concept named cascaded deep learning, consists of three components: (a) a denoising autoencoder (DAE) trained using clinical LR noisy MRI scans that have been processed with a nonlocal means filter that generates denoised LR data; (b) a down-sampling network (DSN) trained with a small amount of paired LR/HR data from volunteers that allows for the generation of perfectly paired LR/HR data for the training of a generative model; and (c) the proposed SR generative model (p-SRG) trained with data generated by the DSN that maps from LR inputs to HR outputs. After training, LR clinical images may be fed through the DAE and p-SRG to yield SR reconstructions of the LR input. The application of this framework was explored in two settings: 3D breath-hold MRI axial SR reconstruction from LR axial scans (<3sec/vol) and in the enhancement of the spatial resolution of LR 4D-MRI acquisitions (0.5sec/vol). The DSN produces LR scans from HR inputs with a higher fidelity to true, LR clinical scans compared to conventional k-space down-sampling methods based on the metrics of root mean square error (RMSE) and structural similarity index (SSIM). Furthermore, HR outputs generated by the p-SRG exhibit improved scores in the peak signal-to-noise ratio, normalized RMSE, SSIM, and in the blind/reference-less image spatial quality evaluator assessment compared to conventional approaches to MRI SR. The robust, SR reconstruction method for MRI based on the novel cascaded deep learning framework is an end-to-end method for producing detail-preserving SR reconstructions from noisy, LR clinical MRI scans. Fourfold enhancements in spatial resolution facilitate target delineation and motion management during radiation therapy, enabling precise MRI-guided radiation therapy with 3D LR breath-hold MRI and 4D-MRI in a clinically feasible time frame.

Robust and efficient data transmission over noisy communication channels using stacked and denoising autoencoders

We study the effects of quantization and additive white Gaussian noise (AWGN) in transmitting latent representations of images over a noisy communication channel. The latent representations are obtained using autoencoders (AEs). We analyze image reconstruction and classification performance for different channel noise powers, latent vector sizes, and number of quantization bits used for the latent variables as well as AEs' parameters. The results show that the digital transmission of latent representations using conventional AEs alone is extremely vulnerable to channel noise and quantization effects. We then propose a combination of basic AE and a denoising autoencoder (DAE) to denoise the corrupted latent vectors at the receiver. This approach demonstrates robustness against channel noise and quantization effects and enables a significant improvement in image reconstruction and classification performance particularly in adverse scenarios with high noise powers and significant quantization effects.

Modulation classification based on denoising autoencoder and convolutional neural network with GNU radio

In this article, a machine learning method to classify signal with Gaussian noise based on denoising auto encoder (DAE) and convolutional neural network (CNN) is proposed. We combine denoising autoencoder's denoising ability with CNN's feature extraction capability. First, a six-layer neural network is built, including three CNN layers. Then a dataset containing noiseless signal of 11 modulation is generated. In the simulation, we apply this dataset to train neural network and achieve an accuracy of 94%, which is much higher than performance with noisy signal, meaning that noise can greatly influence the accuracy of neural network. Next, we build a denoising autoencoder and train it with signal of 5 dB signal-to-noise ratio (SNR). Compared with neural network without denoising autoencoder, adding a denoising autoencoder can achieve an accuracy of 84% at signal of 18 dB SNR, improved by 58%.

Robust Anomaly Detection in Videos Using Multilevel Representations

Detecting anomalies in surveillance videos has long been an important but unsolved problem. In particular, many existing solutions are overly sensitive to (often ephemeral) visual artifacts in the raw video data, resulting in false positives and fragmented detection regions. To overcome such sensitivity and to capture true anomalies with semantic significance, one natural idea is to seek validation from abstract representations of the videos. This paper introduces a framework of robust anomaly detection using multilevel representations of both intensity and motion data. The framework consists of three main components: 1) representation learning using Denoising Autoencoders, 2) level-wise representation generation using Conditional Generative Adversarial Networks, and 3) consolidating anomalous regions detected at each representation level. Our proposed multilevel detector shows a significant improvement in pixel-level Equal Error Rate, namely 11.35%, 12.32% and 4.31% improvement in UCSD Ped 1, UCSD Ped 2 and Avenue datasets respectively. In addition, the model allowed us to detect mislabeled anomalies in the UCDS Ped 1.