Denoising Autoencoder Research Articles

Monitoring trends in road surface impact on rolling noise emissions

Road surfaces degrade over time due to heavy traffic and weather conditions, which negatively influences both driving comfort and acoustic properties. In addition, the lifetime of a road surface can be increased by performing cost-effective incremental maintenance and this maintenance becomes more expensive when the damages are more severe (cracks, potholes). Current methods such as CPX are performed in a standardized way (using designated equipment and tightly controlled measurement conditions), however budget constraints limit frequent monitoring of surfaces. Therefore, continuous monitoring using ordinary passenger vehicles could be helpful to observe trends in rolling noise emissions and road evenness. Hence, we deployed designated sensor boxes in a number of vehicles that are on the road for other purposes. In addition, advances in calibration of different devices using de-noising autoencoders alleviate the effect of various measurement conditions such as driving speed, braking, accelerating, and temperature. As our innovative methodology has now been on the road for several years, trend analysis becomes possible.

Evaluation Algorithm of Teaching Work Quality in Colleges and Universities Based on Deep Denoising Autoencoder Network

One of the most significant components of the teaching department’s evaluation of teaching quality is evaluating teachers’ performance. With the acceleration of educational informatization, modern information processing technology can be used effectively to evaluate teachers’ teaching quality in traditional teaching. In this context, combined with some computational intelligence algorithms, it is critical to developing a targeted teaching quality evaluation system. This paper studies teacher teaching evaluation’s characteristics and existing problems and analyzes the fundamental theories and methods of teacher teaching evaluation in colleges and universities. A novel combination of deep denoising autoencoder and support vector machine was proposed for evaluating teacher’s teaching quality. Moreover, support vector regression is used to predict the model’s output layer to achieve supervised assessment prediction. To capture the data’s key properties, the model comprises numerous hidden layers and conducts various feature transformations during unsupervised training to minimize the mean square error between the reconstructed output data and the original input data. As a result, the proposed model achieved the highest recognition accuracy of 85.23% and convergence compared to other models. Thus, the method can be employed to evaluate and forecast the quality of university teaching activity successfully.

Intelligent fault diagnosis of machinery using digital twin-assisted deep transfer learning

Digital twin (DT) is emerging as a key technology for smart manufacturing. The high fidelity DT model of the physical assets can produce system performance data that is close to reality, which provides remarkable opportunities for machine fault diagnosis when the measured fault condition data are insufficient. This paper presents an intelligent fault diagnosis framework for machinery based on DT and deep transfer learning. First, the DT model of the machine is built by establishing the simulation model and with further updating through continuously measured data from the physical asset. Second, all important machine conditions can be simulated from the built DT. Third, a new-type deep structure based on novel sparse de-noising auto-encoder (NSDAE) is developed and pre-trained with condition data from the source domain, as generated from the DT. Then, to achieve accurate machine fault diagnosis with possible variations in working conditions and system characteristics, the pre-trained NSDAE is fine-tuned using parameter transfer with only one sample from the target domain. The presented method is validated through a case study of triplex pump fault diagnosis. The experimental results demonstrate that the proposed method achieves intelligent fault diagnosis with a limited amount of measured data and outperforms other state-of-the-art data-driven methods.

Attention based multi-agent intrusion detection systems using reinforcement learning

Designing an effective network intrusion system (IDS) is a challenging problem because of the emergence of a large number of novel attacks and heterogeneous network applications. The existing IDSs fail to adapt to the changing attack patterns and unseen attacks that lead to inaccurate detection of network vulnerabilities and system performance degradation. Therefore, there is a need to design robust, scalable, efficient, and adaptive IDS for networks. This paper presents a novel deep reinforcement learning-based IDS that employs Deep Q-Network logic in multiple distributed agents and uses attention mechanisms to efficiently detect and classify advanced network attacks. Our proposed multi-agent IDS is designed as a distributed attack detection platform where agents work in a coordinated manner to provide scalable, fault-tolerant, multi-view architecture guided security system. We have tested our model with extensive experimentation on two benchmark datasets: NSL-KDD and CICIDS2017. It shows improved performance in terms of higher accuracy, precision, recall, F1-Score, and low false-positive rate (FPR) in comparison to the state-of-the-art IDS works. On the other hand, many machine learning systems are found vulnerable to adversarial attacks. Thus, we evaluated our model’s robustness against a practical black-box adversarial attack and observed only a little degradation in performance. We integrated the concept of denoising autoencoder (DAE) with our model to further improve its robustness. Finally, we discuss the usability of our system in real-life applications against zero-day attack patterns.

Damage Identification Based on the Loaded Frequency Changes and Deep Learning for Bridge

Frequency is widely used in damage identification as the dynamic characteristic parameter with the simplest measurement method and the highest measurement accuracy, but the frequency changes are nonunique parameters of damage location for the symmetrical structure. To improve the method of damage location identification based on frequency changes for bridge, the bridge damage identification network based on loaded frequency changes and deep learning theory is proposed by introducing the moving vehicle detection method. A continuous beam example is used for testing, and the test results show that the identification network based on the bridge load frequency changes and the Stacked Denoising Auto-Encoder(SDAE) network can effectively locate the damage of bridge, which enables the frequency changes data to overcome the nonunique limitation of the symmetrical structure, enhances the expression ability of the parameters and improves the effectiveness and the anti-noise property of bridge damage location identification.

Automatic diagnosis of COVID-19 disease using deep convolutional neural network with multi-feature channel from respiratory sound data: Cough, voice, and breath

The problem of respiratory sound classification has received good attention from the clinical scientists and medical researcher’s community in the last year to the diagnosis of COVID-19 disease. The Artificial Intelligence (AI) based models deployed into the real-world to identify the COVID-19 disease from human-generated sounds such as voice/speech, dry cough, and breath. The CNN (Convolutional Neural Network) is used to solve many real-world problems with Artificial Intelligence (AI) based machines. We have proposed and implemented a multi-channeled Deep Convolutional Neural Network (DCNN) for automatic diagnosis of COVID-19 disease from human respiratory sounds like a voice, dry cough, and breath, and it will give better accuracy and performance than previous models. We have applied multi-feature channels such as the data De-noising Auto Encoder (DAE) technique, GFCC (Gamma-tone Frequency Cepstral Coefficients), and IMFCC (Improved Multi-frequency Cepstral Coefficients) methods on augmented data to extract the deep features for the input of the CNN. The proposed approach improves system performance to the diagnosis of COVID-19 disease and provides better results on the COVID-19 respiratory sound dataset.

Likelihood-free Cosmological Constraints with Artificial Neural Networks: An Application on Hubble Parameters and SNe Ia

Abstract The errors of cosmological data generated from complex processes, such as the observational Hubble parameter data (OHD) and the Type Ia supernova (SN Ia) data, cannot be accurately modeled by simple analytical probability distributions, e.g., a Gaussian distribution. To constrain cosmological parameters from these data, likelihood-free inference is usually used to bypass the direct calculation of the likelihood. In this paper, we propose a new procedure to perform likelihood-free cosmological inference using two artificial neural networks (ANNs), the masked autoregressive flow (MAF) and the denoising autoencoder (DAE). Our procedure is the first to use DAE to extract features from data, in order to simplify the structure of MAF needed to estimate the posterior. Tested on simulated Hubble parameter data with a simple Gaussian likelihood, the procedure shows the capability of extracting features from data and estimating posterior distributions without the need of tractable likelihood. We demonstrate that it can accurately approximate the real posterior, achieve performance comparable to the traditional Markov chain Monte Carlo method, and MAF obtains better training results for a small number of simulation when the DAE is added. We also discuss the application of the proposed procedure to OHD and Pantheon SN Ia data, and use them to constrain cosmological parameters from the non-flat ΛCDM model. For SNe Ia, we use fitted light-curve parameters to find constraints on H 0, Ω m , and ΩΛ similar to relevant work, using less empirical distributions. In addition, this work is also the first to use a Gaussian process in the procedure of OHD simulation.

Research on Evaluation of Tower Vibration State Based on SCADA

Tower is very critical to the safe operation of wind turbines. In this paper, SCADA data is used to evaluate the vibration state of the tower. A tower vibration correlation analysis method based on denoising autoencoder (DAE) is proposed, which evaluates the impact of state parameters on tower vibration based on the reconstruction residual. The tower vibration is predicted based on the Long Short-Term Memory (LSTM) network, and then the tower vibration state is evaluated based on the Wasserstein distance. The actual SCADA data is used to verify the proposed method. The results show that the method accurately predicts the tower vibration trend and quantitatively evaluates the tower vibration state.

Software defect prediction based on stacked sparse denoising autoencoders and enhanced extreme learning machine

Software defect prediction is an important software quality assurance technique. Nevertheless, the prediction performance of the constructed model is easily susceptible to irrelevant or redundant features in the software projects and is not predominant enough. To address these two issues, a novel defect prediction model called SSEPG based on Stacked Sparse Denoising AutoEncoders (SSDAE) and Extreme Learning Maching (ELM) optimised by Particle Swarm Optimisation (PSO) and another complementary Gravitational Search Algorithm (GSA) are proposed in this paper, which has two main merits: (1) employ a novel deep neural network – SSDAE to extract new combined features, which can effectively learn the robust deep semantic feature representation. (2) integrate strong exploitation capacity of PSO with strong exploration capability of GSA to optimise the input weights and hidden layer biases of ELM, and utilise the superior discriminability of the enhanced ELM to predict the defective modules. The SSDAE is compared with eleven state-of-the-art feature extraction methods in effect and efficiency, and the SSEPG model is compared with multiple baseline models that contain five classic defect predictors and three variants across 24 software defect projects. The experimental results exhibit the superiority of the SSDAE and the SSEPG on six evaluation metrics.

Temporal convolutional denoising autoencoder network for air pollution prediction with missing values

In recent years, people are paying more attention to improve air quality levels to mitigate its negative impact on human health. So, effective air pollution control has become one of the hottest environmental issues. Accurate pollution prediction plays a vital role in air pollution control. However, air quality modeling faces challenges like long-term pollutant variations due to meteorological variables impact and missing values due to natural disaster or sensor shutdown. These issues make air quality models more complex and challenging to introspect. So, we developed Temporal Convolutional Denoising Autoencoder (TCDA) network, a hybrid PM2.5 prediction framework that can perform rapid extraction of complex dataset's features, handle missing values and improve PM2.5 prediction results. This research paper includes Temporal Convolutional Network (TCN) and Denoising Autoencoder (DAE) network to handle nonlinear multivariate massive datasets. The former utilizes parallel feature processing of Convolutional Neural Network (CNN) and temporal component modeling ability of Recurrent Neural Network (RNN), which helps for features extraction from complex dataset. The latter is to reconstruct the error to fine-tune the prediction results and handle missing values. The proposed model's prediction capability is evaluated by comparing its performance with other baseline models, which shows its performance superiority over other models.

A Two-Stage Neural Network-Based Cold Start Item Recommender

Nowadays, recommendation systems have been successfully adopted in variant online services such as e-commerce, news, and social media. The recommenders provide users a convenient and efficient way to find their exciting items and increase service providers’ revenue. However, it is found that many recommenders suffered from the cold start (CS) problem where only a small number of ratings are available for some new items. To conquer the difficulties, this research proposes a two-stage neural network-based CS item recommendation system. The proposed system includes two major components, which are the denoising autoencoder (DAE)-based CS item rating (DACR) generator and the neural network-based collaborative filtering (NNCF) predictor. In the DACR generator, a textual description of an item is used as auxiliary content information to represent the item. Then, the DAE is applied to extract the content features from high-dimensional textual vectors. With the compact content features, a CS item’s rating can be efficiently derived based on the ratings of similar non-CS items. Second, the NNCF predictor is developed to predict the ratings in the sparse user–item matrix. In the predictor, both spare binary user and item vectors are projected to dense latent vectors in the embedding layer. Next, latent vectors are fed into multilayer perceptron (MLP) layers for user–item matrix learning. Finally, appropriate item suggestions can be accurately obtained. The extensive experiments show that the DAE can significantly reduce the computational time for item similarity evaluations while keeping the original features’ characteristics. Besides, the experiments show that the proposed NNCF predictor outperforms several popular recommendation algorithms. We also demonstrate that the proposed CS item recommender can achieve up to 8% MAE improvement compared to adding no CS item rating.

Autoencoder-Based Extrasystole Detection and Modification of RRI Data for Precise Heart Rate Variability Analysis.

Heart rate variability, which is the fluctuation of the R-R interval (RRI) in electrocardiograms (ECG), has been widely adopted for autonomous evaluation. Since the HRV features that are extracted from RRI data easily fluctuate when arrhythmia occurs, RRI data with arrhythmia need to be modified appropriately before HRV analysis. In this study, we consider two types of extrasystoles—premature ventricular contraction (PVC) and premature atrial contraction (PAC)—which are types of extrasystoles that occur every day, even in healthy persons who have no cardiovascular diseases. A unified framework for ectopic RRI detection and a modification algorithm that utilizes an autoencoder (AE) type of neural network is proposed. The proposed framework consists of extrasystole occurrence detection from the RRI data and modification, whose targets are PVC and PAC. The RRI data are monitored by means of the AE in real time in the detection phase, and a denoising autoencoder (DAE) modifies the ectopic RRI caused by the detected extrasystole. These are referred to as AE-based extrasystole detection (AED) and DAE-based extrasystole modification (DAEM), respectively. The proposed framework was applied to real RRI data with PVC and PAC. The result showed that AED achieved a sensitivity of 93% and a false positive rate of 0.08 times per hour. The root mean squared error of the modified RRI decreased to 31% in PVC and 73% in PAC from the original RRI data by DAEM. In addition, the proposed framework was validated through application to a clinical epileptic seizure problem, which showed that it correctly suppressed the false positives caused by PVC. Thus, the proposed framework can contribute to realizing accurate HRV-based health monitoring and medical sensing systems.

Rolling Bearing Fault Classification Based on Stacked Denoising Auto Encoders

Aiming at the characteristics of large capacity and diversity of rolling bearing fault data, an intelligent rolling bearing fault diagnosis method based on stacked denoising auto encoders was proposed. Firstly, Principal Component Analysis was used to reduce the dimension of the original data, and the redundant information is deleted. Then, three de-noising auto-encoders are created to train the bearing data. Then, a stack de-noising auto-encoder with three hidden layers is stacked with the trained DAE for reverse optimization. Finally, the features are input into soft-max classifier to realize rolling bearing fault diagnosis. The experimental results show that SDAE network can extract fault features effectively, and it is better than back propagation neural network in generalization and fault classification accuracy.

Prediction of a single Gaussian shape of spectral line measured with low-dispersion spectrometer by using machine learning.

We have developed a denoising autoencoder based neural network (NN) method to determine a spectral line intensity with an uncertainty lower than the uncertainty determined by fitting the spectral line. The NN method processes the measured raw spectral line shape, providing a single Gaussian shape based on the training dataset, which consists of synthetically prepared Doppler shift and broadening free spectral lines in the present work. It is found that the uncertainty reduction level significantly depends on the training dataset. Limitations originating from the training dataset are also discussed.

A novel gradient foster shared-representation convolutional network optimization for multi-modalities

Significant growth has been made with multi-modal data as its entrance in the field of deep learning; whereas, Convolutional Neural Network (CNN) provides sufficient training data to develop a representative encrusted image. Yet, the multi-modality approach in CNN affect the performance by slowly converge the variance along with high-dimensionality, heterogeneity and non-aconvex optimization problems. To abridge these issues, a novel Gradient Foster Shared-representation Convolutional Network (GFSCN) framework is proposed, which improve and optimize the performance interms of accuracy and dimensionality reduction. Initially, the framework incorporates a multiple scant weighted de-noising autoencoder to solve the heterogeneity problem and reduces the dimensionality of data by transforming shared feature representation. Consequently, the work integrated enhanced stochastic variance reduced ascension approach. This approach diminishes the non-convex optimization problem through integrating two gradients consuming mini-batches, which reduced the loss function thereby achieves faster convergence even with the usage of larger dataset. Thus, the proposed framework achieves better performance in terms of achieving utmost accuracy with faster convergence and reduced variance.

Multi-Scale Convolutional Recurrent Neural Network for Bearing Fault Detection in Noisy Manufacturing Environments

The failure of a facility to produce a product can have significant impacts on the quality of the product. Most equipment failures occur in rotating equipment, with bearing damage being the biggest cause of failure in rotating equipment. In this paper, we propose a denoising autoencoder (DAE) and multi-scale convolution recurrent neural network (MS-CRNN), wherein the DAE accurately inspects bearing defects in the same environment as bearing vibration signals in the field, and the MS-CRNN inspects and classifies defects. We experimented with adding random noise to create a dataset that resembled noisy manufacturing installations in the field. From the results of the experiment, the accuracy of the proposed method was more than 90%, proving that it is an algorithm that can be applied in the field.

An end-to-end denoising autoencoder-based deep neural network approach for fault diagnosis of analog circuit

Fault diagnosis of analog circuit is critical to improve safety and reliability in electrical systems and reduce losses. Traditional fault diagnosis methods of analog circuit usually rely on the hand design feature extractor and can not generalize well to other diagnosis domains. To address these issues, an end-to-end denoising autoencoder (EEDAE)-based fault diagnosis approach is proposed. The proposed approach includes denoising autoencoder (DAE) and a softmax classifier. The DAE is designed to automatically extract fault features from the raw time series signals without any signal processing techniques and diagnostic expertise, and then the softmax classifier is used to classify the fault mode of analog circuits. Specifically, we design a novel loss function by jointly minimizing reconstruction loss and classification loss to improve training efficiency. The proposed approach just has one training stage, in which the encoder, decoder, and classifier are trained simultaneously. The experimental results demonstrate that compared with traditional methods, the proposed method has higher accuracy and lower requirements on data.

RETRACTED: A Novel Day- ahead Electricity Price Forecasting Using multi-modal combined Integration via Stacked Pruning Sparse Denoising Auto Encoder

One of the best options for managing energy markets and power system performance optimization is precise Electricity Price Forecasting (EPF). In detecting the prices of energy, some factors are of stochastic behavior which has made an unwieldy task. In this study, carrying out Tensor Canonical Correlation Analysis (TCCA) is the first activity to choose the operative parameters in EPF issues and removing redundant factors with low correlation ranks. The second activity is making use of a Deep Neural Network (DNN) with a novel Stacked Pruning Sparse Denoising Autoencoder (SPSDAE) to decrease the noise of data sets with different sources individually. The third activity is that a novel method which is called the Maximum Separation Subspace (MASES) in Sufficient Dimension Reduction (SDR) with Categorical Response. This method is used to detect the outstanding properties of the input data and removing the uncertainty characteristics. The new multi-modal combined (MMC) method is the final step that needs to be taken to predict the day-ahead (one day in advance) cost of electricity. The proposed method is investigated based on the data that were collected from Australia The combination of various predicting models and suggested methods might develop the function of predicting possibility by supplying divergent types of compactness performance. The combined model comprised of three predicting possibilities called beta circulation fitting, scattered Bayesian learning, core compactness prediction. Predicting data respectively are used as a yardstick to compare the efficiency of forecasted results and consequently assess the proposed method. Results of this study confirmed a decrease of effectual error criterion and accuracy enhancement in EPF field as the advantageous effects of this proposed SDR structure.

Heterogeneous bi-directional recurrent neural network combining fusion health indicator for predictive analytics of rotating machinery

Data-driven intelligent methods arise the increasing demand for predictive analytics to evaluate the operational reliability and natural degradation of rotating machinery. Nevertheless, accurate and timely predictive analytics is still regarded as an extremely challenging mission, because the quality of predictive maintenance depends not only on the capability of intelligent model, but also on the construction of effective health indicators To overcome this issue, a novel heterogeneous bi-directional gated recurrent unit (GRU) model combining with fusion health indicator (Fusion-HI) is proposed for predictive analytics in this paper. First, the support evidence space is constructed to reflect the operating state of mechanical equipment. Then the evidence features from multiple domains are integrated to obtain the optimal Fusion-HI by the modified de-noising auto-encoder (MDAE). Finally, a hybrid prediction network is designed combining with the gate attention algorithm, which consists of multi-scale convolution layers, bi-directional GRU layers, smoothed and de-noised layers, and regression layers. Three experimental whole lifetime data and one industrial entire life cycle data are analyzed to validate the feasibility of the proposed approach in two case studies respectively. Relevant experimental results indicate that the Fusion-HI is capable to sensitively characterize the degradation state of equipment, while the prediction accuracy of presented heterogeneous model is superior to that of conventional prediction approaches.

A system for detection of moving caption text in videos: a news use case

Extraction of news text captions aims at a digital understanding of what is happening in a specific region during a certain period that helps in better communication between different nations because we can easily translate the plain text from one language to another. Moving text captions causes blurry effects that are a significant cause of text quality impairments in the news channels. Most of the existing text caption detection models do not address this problem in a way that captures the different dynamic motion of captions, gathers a full news story among several frames in the sequence, resolves the blurring effect of text motion, offers a language-independent model, or provides it as an end-to-end solution for the community to use. We process the frames coming in sequence and extract edge features using either the Hough transform or our color-based technique. We verify text existence using a Convolutional Neural Network (CNN) text detection pre-trained model. We analyze the caption motion status using hybrid pre-trained Recurrent Neural Network (RNN) of Long Short-Term Memory (LSTM) type model and correlation-based model. In case the motion is determined to be horizontal rotation, there are two problems. First, it means that text keeps rotating with no stop resulting in a high blurring effect that affects the text quality and consequently resulting in low character recognition accuracy. Second, there are successive news stories which are separated by the channel logo or long spaces. We managed to solve the first problem by deblurring the text image using either Bicubic Spline Interpolation (BSI) technique or the Denoising Autoencoder Neural Network (DANN). We solved the second problem using a Point Feature Matching (PFM) technique to match the existing channel logo with the channels’ logo database (ground truth). We evaluate our framework using Abbyy® SDK as a standalone tool used for text recognition supporting different languages.