Deep Denoising Autoencoder Research Articles

Stealthy data integrity attack identification in smart grid networks utilizing deep denoising autoencoder

Smart grids arose as the largest cyber-physical systems with the integration of sophisticated control, computing, and state-of-the-art communications. Like all cyber-physical systems, the smart grids are vulnerable to malicious cyber assaults due to their enormous dependency on communication networks. Various machine learning-based schemes are being investigated in the industry and academia to develop robust defense mechanisms to counter cyber assaults. However, the curse of high dimensionality, which increases with the escalating evolution of an electric power system, infringes upon the efficiency of machine learning models employed to detect such assaults. To this end, this paper proposes a deep denoising autoencoder (DAE)-based framework for dimensionality reduction that learns salient feature representation for high-dimensional, multi-variant smart grid measurement data collected from the smart grids. The latent space apprehended by DAE is then fed to binary support vector machine (SVM) to determine the assaulted data. Various standard IEEE test cases are employed in simulations. The results show that the proposed scheme learns more robust features that reveal the nonlinear properties exhibited in the smart grid measurements, further leading to improved detection accuracy of the classifier as compared to existing approaches.

Research on unsupervised condition monitoring method of pump-type machinery in nuclear power plant

As a typical active equipment, pump machinery is widely used in nuclear power plants. Although the mechanism of pump machinery in nuclear power plants is similar to that of conventional pumps, the safety and reliability requirements of nuclear pumps are higher in complex operating environments. Once there is significant performance degradation or failure, it may cause huge security risks and economic losses. There are many pumps mechanical parameters, and it is very important to explore the correlation between multi-dimensional variables and condition. Therefore, a condition monitoring model based on Deep Denoising Autoencoder (DDAE) is constructed in this paper. This model not only ensures low false positive rate, but also realizes early abnormal monitoring and location. In order to alleviate the influence of parameter time-varying effect on the model in long-term monitoring, this paper combined equidistant sampling strategy and DDAE model to enhance the monitoring efficiency. By using the simulation data of reactor coolant pump and the actual centrifugal pump data, the monitoring and positioning capabilities of the proposed scheme under normal and abnormal conditions were verified. This paper has important reference significance for improving the intelligent operation and maintenance efficiency of nuclear power plants.

Deep learning for exploring ultra-thin ferroelectrics with highly improved sensitivity of piezoresponse force microscopy

Hafnium oxide-based ferroelectrics have been extensively studied because of their existing ferroelectricity, even in ultra-thin film form. However, studying the weak response from ultra-thin film requires improved measurement sensitivity. In general, resonance-enhanced piezoresponse force microscopy (PFM) has been used to characterize ferroelectricity by fitting a simple harmonic oscillation model with the resonance spectrum. However, an iterative approach, such as traditional least squares (LS) fitting, is sensitive to noise and can result in the misunderstanding of weak responses. In this study, we developed the deep neural network (DNN) hybrid with deep denoising autoencoder (DDA) and principal component analysis (PCA) to extract resonance information. The DDA/PCA-DNN improves the PFM sensitivity down to 0.3 pm, allowing measurement of weak piezoresponse with low excitation voltage in 10-nm-thick Hf0.5Zr0.5O2 thin films. Our hybrid approaches could provide more chances to explore the low piezoresponse of the ultra-thin ferroelectrics and could be applied to other microscopic techniques.

Environmental Noise Reduction based on Deep Denoising Autoencoder

Speech enhancement plays an important role in Automatic Speech Recognition (ASR) even though this task remains challenging in real-world scenarios of human-level performance. To cope with this challenge, an explicit denoising framework called Deep Denoising Autoencoder (DDAE) is introduced in this paper. The parameters of DDAE encoder and decoder are optimized based on the backpropagation criterion, where all denoising autoencoders are stacked up instead of recurrent connections. For better speech estimation in real and noisy environments, we include matched and mismatched noisy and clean pairs of speech data to train the DDAE. The DDAE has the ability to achieve optimal results even for a limited amount of training data. Our experimental results show that the proposed DDAE outperformed the baseline algorithms. The DDAE shows superior performances based on three-evaluation metrics in noisy and clean pairs of speech data compared to three baseline algorithms.

Calibration-Free 3D Indoor Positioning Algorithms Based on DNN and DIFF.

The heterogeneity of wireless receiving devices, co-channel interference, and multi-path effect make the received signal strength indication (RSSI) of Wi-Fi fluctuate greatly, which seriously degrades the RSSI-based positioning accuracy. Signal strength difference (DIFF), a calibration-free solution for handling the received signal strength variance between diverse devices, can effectively reduce the negative impact of signal fluctuation. However, DIFF also leads to the explosion of the RSSI data dimension, expanding the number of dimensions from m to , which reduces the positioning efficiency. To this end, we design a data hierarchical processing strategy based on a building-floor-specific location, which effectively improves the efficiency of high-dimensional data processing. Moreover, based on a deep neural network (DNN), we design three different positioning algorithms for multi-building, multi-floor, and specific-location respectively, extending the indoor positioning from the single plane to three dimensions. Specifically, in the stage of data preprocessing, we first create the original RSSI database. Next, we create the optimized RSSI database by identifying and deleting the unavailable data in the RSSI database. Finally, we perform DIFF processing on the optimized RSSI database to create the DIFF database. In the stage of positioning, firstly, we design an improved multi-building positioning algorithm based on a denoising autoencoder (DAE). Secondly, we design an enhanced DNN for multi-floor positioning. Finally, the newly deep denoising autoencoder (DDAE) used for specific location positioning is proposed. The experimental results show that the proposed algorithms have better positioning efficiency and accuracy compared with the traditional machine learning algorithms and the current advanced deep learning algorithms.

Boundary-Preserved Deep Denoising of Stochastic Resonance Enhanced Multiphoton Images.

Objective: With the rapid growth of high-speed deep-tissue imaging in biomedical research, there is an urgent need to develop a robust and effective denoising method to retain morphological features for further texture analysis and segmentation. Conventional denoising filters and models can easily suppress the perturbative noise in high-contrast images; however, for low photon budget multiphoton images, a high detector gain will not only boost the signals but also bring significant background noise. In such a stochastic resonance imaging regime, subthreshold signals may be detectable with the help of noise, meaning that a denoising filter capable of removing noise without sacrificing important cellular features, such as cell boundaries, is desirable. Method: We propose a convolutional neural network-based denoising autoencoder method — a fully convolutional deep denoising autoencoder (DDAE) — to improve the quality of three-photon fluorescence (3PF) and third-harmonic generation (THG) microscopy images. Results: The average of 200 acquired images of a given location served as the low-noise answer for the DDAE training. Compared with other conventional denoising methods, our DDAE model shows a better signal-to-noise ratio (28.86 and 21.66 for 3PF and THG, respectively), structural similarity (0.89 and 0.70 for 3PF and THG, respectively), and preservation of the nuclear or cellular boundaries (F1-score of 0.662 and 0.736 for 3PF and THG, respectively). It shows that DDAE is a better trade-off approach between structural similarity and preserving signal regions. Conclusions: The results of this study validate the effectiveness of the DDAE system in boundary-preserved image denoising. Clinical Impact: The proposed deep denoising system can enhance the quality of microscopic images and effectively support clinical evaluation and assessment.

Evaluation Algorithm of Teaching Work Quality in Colleges and Universities Based on Deep Denoising Autoencoder Network

One of the most significant components of the teaching department’s evaluation of teaching quality is evaluating teachers’ performance. With the acceleration of educational informatization, modern information processing technology can be used effectively to evaluate teachers’ teaching quality in traditional teaching. In this context, combined with some computational intelligence algorithms, it is critical to developing a targeted teaching quality evaluation system. This paper studies teacher teaching evaluation’s characteristics and existing problems and analyzes the fundamental theories and methods of teacher teaching evaluation in colleges and universities. A novel combination of deep denoising autoencoder and support vector machine was proposed for evaluating teacher’s teaching quality. Moreover, support vector regression is used to predict the model’s output layer to achieve supervised assessment prediction. To capture the data’s key properties, the model comprises numerous hidden layers and conducts various feature transformations during unsupervised training to minimize the mean square error between the reconstructed output data and the original input data. As a result, the proposed model achieved the highest recognition accuracy of 85.23% and convergence compared to other models. Thus, the method can be employed to evaluate and forecast the quality of university teaching activity successfully.

Anomaly identification with few labeled data in the distillation process based on semisupervised ladder networks

AbstractHigh repeatability of similar information but a lack of typical fault features, in the monitoring data of distillation processes for continuous production, leads to a small proportion of data with labels. Therefore, the requirement for a large number of labeled samples in conventional deep learning models cannot be met, resulting in significant performance degradation in their anomaly identification. In this paper, an intelligent anomaly identification method for small samples is proposed, based on semisupervised deep learning. Specifically, on the basis of a deep denoising autoencoder (DAE), semisupervised ladder networks (SSLN) is constructed to use a large number of unlabeled, process data to assist the supervised learning process, thus improving the performance of the anomaly identification model. In order to construct the optimum SSLN model, the influences of parameters such as the number of deep network layers, the proportion of labeled samples, and the noise intensity on identification accuracy are analyzed while making the information flow in the network more efficient. Experimental results of anomaly identification in the depropanization distillation process show that compared with the conventional multilayer perception (MLP) and convolutional neural network (CNN)‐DAE models, the proposed method can obtain a higher diagnostic accuracy in the case with limited labeled process data.

Improving Speech Quality for Hearing Aid Applications Based on Wiener Filter and Composite of Deep Denoising Autoencoders

In hearing aid devices, speech enhancement techniques are a critical component to enable users with hearing loss to attain improved speech quality under noisy conditions. Recently, the deep denoising autoencoder (DDAE) was adopted successfully for recovering the desired speech from noisy observations. However, a single DDAE cannot extract contextual information sufficiently due to the poor generalization in an unknown signal-to-noise ratio (SNR), the local minima, and the fact that the enhanced output shows some residual noise and some level of discontinuity. In this paper, we propose a hybrid approach for hearing aid applications based on two stages: (1) the Wiener filter, which attenuates the noise component and generates a clean speech signal; (2) a composite of three DDAEs with different window lengths, each of which is specialized for a specific enhancement task. Two typical high-frequency hearing loss audiograms were used to test the performance of the approach: Audiogram 1 = (0, 0, 0, 60, 80, 90) and Audiogram 2 = (0, 15, 30, 60, 80, 85). The hearing-aid speech perception index, the hearing-aid speech quality index, and the perceptual evaluation of speech quality were used to evaluate the performance. The experimental results show that the proposed method achieved significantly better results compared with the Wiener filter or a single deep denoising autoencoder alone.

A real-world noise removal with wavelet speech feature

Real-world noise signals are non-stationary, and these signals are a mixture of more than one non-stationary noise signal. Most of the conventional speech enhancement algorithms (SEAs) focus primarily on a single noise corrupted speech signal, and it is far from real-world environments. In this article, we discuss speech enhancement in real-world environments with a new speech feature. The novelty of this article is three-fold, (1) The proposed model analyzed in real-world environments. (2) The proposed model uses a discrete wavelet transform (DWT) coefficients as input features. (3) The proposed Deep Denoising Autoencoder (DDAE) designed experimentally. The result of the proposed feature compares with conventional speech features like FFT-Amplitude, Log-Magnitude, Mel frequency cepstral coefficients (MFCCs), and the Gammatone filter cepstral coefficients (GFCCs). The performance of the proposed method compared with conventional speech enhancement methods. The enhanced signal evaluated with speech quality measures, like, perceptual evaluation speech quality (PESQ), weighted spectral slope (WSS), and Log-likelihood ratio (LLR). Similarly, speech intelligibility measured with short-time objective intelligibility (STOI). The results show that the proposed SEA model with the DWT feature improves quality and intelligibility in all real-world environmental Signal-to-Noise ratio (SNR) conditions.

Deep denoising autoencoder for seismic random noise attenuation

Attenuation of seismic random noise is considered an important processing step to enhance the signal-to-noise ratio of seismic data. A new approach is proposed to attenuate random noise based on a deep-denoising autoencoder (DDAE). In this approach, the time-series seismic data are used as an input for the DDAE. The DDAE encodes the input seismic data to multiple levels of abstraction, and then it decodes those levels to reconstruct the seismic signal without noise. The DDAE is pretrained in a supervised way using synthetic data; following this, the pretrained model is used to denoise the field data set in an unsupervised scheme using a new customized loss function. We have assessed the proposed algorithm based on four synthetic data sets and two field examples, and we compare the results with several benchmark algorithms, such as f- x deconvolution ( f- x deconv) and the f- x singular spectrum analysis ( f- x SSA). As a result, our algorithm succeeds in attenuating the random noise in an effective manner.

Estimating gene expression from DNA methylation and copy number variation: A deep learning regression model for multi-omics integration

Gene expression analysis plays a significant role for providing molecular insights in cancer. Various genetic and epigenetic factors (being dealt under multi-omics) affect gene expression giving rise to cancer phenotypes. A recent growth in understanding of multi-omics seems to provide a resource for integration in interdisciplinary biology since they altogether can draw the comprehensive picture of an organism's developmental and disease biology in cancers. Such large scale multi-omics data can be obtained from public consortium like The Cancer Genome Atlas (TCGA) and several other platforms. Integrating these multi-omics data from varied platforms is still challenging due to high noise and sensitivity of the platforms used. Currently, a robust integrative predictive model to estimate gene expression from these genetic and epigenetic data is lacking. In this study, we have developed a deep learning-based predictive model using Deep Denoising Auto-encoder (DDAE) and Multi-layer Perceptron (MLP) that can quantitatively capture how genetic and epigenetic alterations correlate with directionality of gene expression for liver hepatocellular carcinoma (LIHC). The DDAE used in the study has been trained to extract significant features from the input omics data to estimate the gene expression. These features have then been used for back-propagation learning by the multilayer perceptron for the task of regression and classification. We have benchmarked the proposed model against state-of-the-art regression models. Finally, the deep learning-based integration model has been evaluated for its disease classification capability, where an accuracy of 95.1% has been obtained.

Use of a Tracer-Specific Deep Artificial Neural Net to Denoise Dynamic PET Images

Application of kinetic modeling (KM) on a voxel level in dynamic PET images frequently suffers from high levels of noise, drastically reducing the precision of parametric image analysis. In this paper, we investigate the use of machine learning and artificial neural networks to denoise dynamic PET images. We train a deep denoising autoencoder (DAE) using noisy and noise-free spatiotemporal image patches, extracted from the simulated images of [11C]raclopride, a dopamine D2 receptor agonist. The DAE-processed dynamic and corresponding parametric images (simulated and acquired) are compared with those obtained with conventional denoising techniques, including temporal and spatial Gaussian smoothing, iterative spatiotemporal smoothing/deconvolution, and the highly constrained backprojection processing (HYPR). The simulated (acquired) parametric image non-uniformity was 7.75% (19.49%) with temporal and 5.90% (14.50%) with spatial smoothing, 5.82% (16.21%) with smoothing/deconvolution, 5.49% (13.38%) with HYPR, and 3.52% (11.41%) with DAE. The DAE also produced the best results in terms of the coefficient of variation of voxel values and structural similarity index. Denoising-induced bias in the regional mean binding potential was 7.8% with temporal and 26.31% with spatial smoothing, 28.61% with smoothing/deconvolution, 27.63% with HYPR, and 14.8% with DAE. When the test data did not match the training data, erroneous outcomes were obtained. Our results demonstrate that a deep DAE can provide a substantial reduction in the voxel-level noise compared with the conventional spatiotemporal denoising methods while introducing a similar or lower amount of bias. The better DAE performance comes at the cost of lower generality and requiring appropriate training data.

Multi-objective learning based speech enhancement method to increase speech quality and intelligibility for hearing aid device users

Background noise is a critical issue for hearing aid device users; a common solution to address this problem is speech enhancement (SE). In recent times, a novel SE approach based on deep learning technology, called deep denoising autoencoder (DDAE), has been proposed. Previous studies show that the DDAE SE approach provides superior noise suppression capabilities and produces less distortion than any of the classical SE approaches in the case of processed speech. Motivated by the improved results using DDAE shown in previous studies, we propose the multi-objective learning-based DDAE (M-DDAE) SE approach in this study; in addition, we evaluated its speech quality and intelligibility improvements using seven typical hearing loss audiograms. The experimental results of our objective evaluations show that our M-DDAE approach achieved significantly better results than the DDAE approach in most test conditions. Considering this, the proposed M-DDAE SE approach can be potentially used to further improve the listening performance of hearing aid devices in noisy conditions.

Bone-conducted speech enhancement using deep denoising autoencoder

Bone-conduction microphones (BCMs) capture speech signals based on the vibrations of the speaker's skull and exhibit better noise-resistance capabilities than normal air-conduction microphones (ACMs) when transmitting speech signals. Because BCMs only capture the low-frequency portion of speech signals, their frequency response is quite different from that of ACMs. When replacing an ACM with a BCM, we may obtain satisfactory results with respect to noise suppression, but the speech quality and intelligibility may be degraded due to the nature of the solid vibration. The mismatched characteristics of BCM and ACM can also impact the automatic speech recognition (ASR) performance, and it is infeasible to recreate a new ASR system using the voice data from BCMs. In this study, we propose a novel deep-denoising autoencoder (DDAE) approach to bridge BCM and ACM in order to improve speech quality and intelligibility, and the current ASR could be employed directly without recreating a new system. Experimental results first demonstrated that the DDAE approach can effectively improve speech quality and intelligibility based on standardized evaluation metrics. Moreover, our proposed system can significantly improve the ASR performance by a notable 48.28% relative character error rate (CER) reduction (from 14.50% to 7.50%) under quiet conditions. In an actual noisy environment (sound pressure from 61.7 dBA to 73.9 dBA), our proposed system with a BCM outperforms an ACM, yielding an 84.46% reduction in the relative CER (proposed system: 9.13% and ACM: 58.75%).

Deep Learning-Based Noise Reduction Approach to Improve Speech Intelligibility for Cochlear Implant Recipients.

We investigate the clinical effectiveness of a novel deep learning-based noise reduction (NR) approach under noisy conditions with challenging noise types at low signal to noise ratio (SNR) levels for Mandarin-speaking cochlear implant (CI) recipients. The deep learning-based NR approach used in this study consists of two modules: noise classifier (NC) and deep denoising autoencoder (DDAE), thus termed (NC + DDAE). In a series of comprehensive experiments, we conduct qualitative and quantitative analyses on the NC module and the overall NC + DDAE approach. Moreover, we evaluate the speech recognition performance of the NC + DDAE NR and classical single-microphone NR approaches for Mandarin-speaking CI recipients under different noisy conditions. The testing set contains Mandarin sentences corrupted by two types of maskers, two-talker babble noise, and a construction jackhammer noise, at 0 and 5 dB SNR levels. Two conventional NR techniques and the proposed deep learning-based approach are used to process the noisy utterances. We qualitatively compare the NR approaches by the amplitude envelope and spectrogram plots of the processed utterances. Quantitative objective measures include (1) normalized covariance measure to test the intelligibility of the utterances processed by each of the NR approaches; and (2) speech recognition tests conducted by nine Mandarin-speaking CI recipients. These nine CI recipients use their own clinical speech processors during testing. The experimental results of objective evaluation and listening test indicate that under challenging listening conditions, the proposed NC + DDAE NR approach yields higher intelligibility scores than the two compared classical NR techniques, under both matched and mismatched training-testing conditions. When compared to the two well-known conventional NR techniques under challenging listening condition, the proposed NC + DDAE NR approach has superior noise suppression capabilities and gives less distortion for the key speech envelope information, thus, improving speech recognition more effectively for Mandarin CI recipients. The results suggest that the proposed deep learning-based NR approach can potentially be integrated into existing CI signal processors to overcome the degradation of speech perception caused by noise.

Improving the performance of hearing aids in noisy environments based on deep learning technology.

The performance of a deep-learning-based speech enhancement (SE) technology for hearing aid users, called a deep denoising autoencoder (DDAE), was investigated. The hearing-aid speech perception index (HASPI) and the hearing- aid sound quality index (HASQI), which are two well-known evaluation metrics for speech intelligibility and quality, were used to evaluate the performance of the DDAE SE approach in two typical high-frequency hearing loss (HFHL) audiograms. Our experimental results show that the DDAE SE approach yields higher intelligibility and quality scores than two classical SE approaches. These results suggest that a deep-learning-based SE method could be used to improve speech intelligibility and quality for hearing aid users in noisy environments.

Industrial Big Data Analytics for Prediction of Remaining Useful Life Based on Deep Learning

Due to the recent development of cyber-physical systems, big data, cloud computing, and industrial wireless networks, a new era of industrial big data is introduced. Deep learning, which brought a revolutionary change in computer vision, natural language processing, and a variety of other applications, has significant potential for solutions providing in sophisticated industrial applications. In this paper, a concept of device electrocardiogram (DECG) is presented, and an algorithm based on deep denoising autoencoder (DDA) and regression operation is proposed for the prediction of the remaining useful life of industrial equipment. First, the concept of electrocardiogram is explained. Then, a problem statement based on manufacturing scenario is presented. Subsequently, the architecture of the proposed algorithm called integrated DDA and the algorithm workflow are provided. Moreover, DECG is compared with traditional factory information system, and the feasibility and effectiveness of the proposed algorithm are validated experimentally. The proposed concept and algorithm combine typical industrial scenario and advance artificial intelligence, which has great potential to accelerate the implementation of industry 4.0.

A Pythagorean-Type Fuzzy Deep Denoising Autoencoder for Industrial Accident Early Warning

Early warning is crucial for preventing industrial accidents and mitigating damage, but current methods are often time-consuming, error-prone, and incompetent to deal with uncertainty. This paper presents a fuzzy deep neural network for early warning of industrial accidents, which equips the classical deep denoising autoencoder (DDAE) model with Pythagorean-type fuzzy parameters in order to enhance the model's representation ability and robustness. To efficiently train the fuzzy deep model, we propose a hybrid algorithm combining Hessian-free optimization and biogeography-based optimization metaheuristic to balance global search and local search. Experiments on datasets from several industrial zones in China show that the proposed Pythagorean-type fuzzy DDAE (PFDDAE) can achieve much higher accuracy of accident risk classification than the classical DDAE and the fuzzy DDAE using regular fuzzy parameters, and the proposed hybrid learning algorithm exhibits significant performance advantage over some other learning algorithms in training PFDDAE. In particular, a test on the 2014 Kunshan aluminum dust explosion accident shows that the deep learning model would be very likely to prevent the accident if it was adopted in advance.

Whispered Speech Recognition Using Deep Denoising Autoencoder and Inverse Filtering

Due to the profound differences between acoustic characteristics of neutral and whispered speech, the performance of traditional automatic speech recognition (ASR) systems trained on neutral speech degrades significantly when whisper is applied. In order to deeply analyze this mismatched train/test situation and to develop an efficient way for whisper recognition, this study first analyzes acoustic characteristics of whispered speech, addresses the problems of whispered speech recognition in mismatched conditions, and then proposes a new robust cepstral features and preprocessing approach based on deep denoising autoencoder (DDAE) that enhance whisper recognition. The experimental results confirm that Teager-energy-based cepstral features, especially TECCs, are more robust and better whisper descriptors than traditional Mel-frequency cepstral coefficients (MFCC). Further detailed analysis of cepstral distances, distributions of cepstral coefficients, confusion matrices, and experiments with inverse filtering, prove that voicing in speech stimuli is the main cause of word misclassification in mismatched train/test scenarios. The new framework based on DDAE and TECC feature, significantly improves whisper recognition accuracy and outperforms traditional MFCC and GMM-HMM (Gaussian mixture density—Hidden Markov model) baseline, resulting in an absolute 31% improvement of whisper recognition accuracy. The achieved word recognition rate in neutral/whisper scenario is 92.81%.