Stacked Denoising Autoencoder Model Research Articles

Traffic Image Analysis Based on Stacked Denoising Autoencoder Neural Network

This study aims to explore major neural network models - Stacked Denoising Autoencoder (SDAE), Deep Belief Network (DBN), Backpropagation - that have recently garnered attention and propose the most suitable and reliable artificial neural network model for real-time road traffic information collection. In this study, to enhance the reliability of experimental results, numerous experiments were conducted under identical conditions (such as parameter values and network configuration) by setting different initial values for the weight vector. The results of the experiments were statistically validated to draw conclusions. The research results showed that the SDAE model exhibited the most superior performance, while the accuracy of the DBN was somewhat lower compared to the SDAE model. On the other hand, the Backpropagation model demonstrated a relatively low predictive accuracy compared to both models, particularly showing a significant influence of the initial values

Passenger flow prediction and management method of urban public transport based on SDAE model and improved Bi-LSTM neural network

In the era of big data, the exponentially increasing data volume and emerging technical tools have put forward new requirements for enterprise information management. Therefore, it is of great significance to enhance the core competitiveness of enterprises to explore how big data can empower the innovation of enterprise information management. Intelligent transportation system combines a variety of technologies and applies them to a large-scale transportation management system, so as to make a reasonable dispatch of traffic conditions. Aiming at the problem of the relatively low accuracy of bus passenger flow forecasting with the existing models, a short-term passenger flow prediction model combining Stacked Denoising Auto Encoder (SDAE) and improved bidirectional Long-short Term Memory network (Bi-LSTM) is proposed. First, the SDAE model is used to fill in the missing bus passenger flow data, the characteristics of the bus passenger flow data are effectively utilized, and the data with rich information is used to predict the missing values with high accuracy. Second, Bi-LSTM model combined with attention mechanism is used for short-term bus passenger flow prediction. Considering that the data sequence of bus passenger flow is relatively long and there is a two-way information flow, the BiLSTM neural network is used for prediction tasks, and the influence of key factors is highlighted through attention weights to mine the internal laws of passenger flow data. The experimental results show that the proposed method achieves the lowest prediction error among all the comparison methods in the task of short-term bus passenger flow prediction on the public transportation dataset, with MAE, MRE, and RMSE values of 6.014, 0.052, and 9.874, respectively. These findings confirmed the effectiveness of the new model in the passenger flow prediction field.

RDERL: Reliable deep ensemble reinforcement learning-based recommender system

Recommender systems (RSs) have been employed for many real-world applications including search engines, social networks, and information retrieval systems as powerful intelligent techniques for creating personalized recommendations. RSs mainly utilize different data resources such as user-item ratings, tag data, and social relationships in their recommendation process to overcome the data sparsity issue. One of the main challenges is how to integrate the results obtained by these data resources. Moreover, most of the existing RSs focus on the recommendation accuracy, and do not consider the reliability of the recommendations. To address these challenges, in this study, a reliable recommendation method is developed, which employs deep neural networks and reinforcement learning. The proposed method firstly uses a stacked denoising autoencoder model to obtain the latent features from the rating, tag, and trust data. The similarity value between each pair of users is then calculated in three different aspects according to the extracted latent features. An ensemble approach is developed by integrating the three similarities using reinforcement learning to determine the final similarity value. The ensemble similarity values are used in the rating prediction process, and the reliability of the predicted ratings is evaluated using an effective measure. Finally, a recommendation strategy is developed based on the integration of the predicted ratings and their reliability values. We evaluated the effectiveness of the proposed method by conducting numerous experiments on three datasets where the results show that the proposed method is far superior to other recommendation models in making more accurate recommendations.

Visualized Stacked Denoising Auto-Encoder Model for Extracting and Evaluating the State Features of Rolling Bearings

Extracting intuitive operating state features from vibration signals without prior knowledge is a prospective requirement for health monitoring and fault diagnosis in bearings. In this paper, a visualized stacked denoising auto-encoder (VSDAE) model is proposed for the unsupervised extraction and quantitative evaluation of bearings’ state features. First, the stacked denoising auto-encoder (SDAE) was used to reconstruct vibration signals. The intermediate vector of the SDAE, which is a high-information-density representation of vibration signals, was regarded as the pending state feature. Then, the dimension of the intermediate vector was reduced by the t-distributed stochastic neighbor embedding (t-SNE) method to the two-dimensional visualization space. Finally, the silhouette coefficient of feature distribution was calculated to quantitatively evaluate the extracted features. The proposed model was evaluated using experimental bearing signals simulating various operating states. The results proved that the features, extracted and evaluated by the VSDAE, allowed the recognition of the operating states of the examined bearings.

Research on the Identification of Pilots’ Fatigue Status Based on Functional Near-Infrared Spectroscopy

Fatigue can lead to sluggish responses, misjudgments, flight illusions and other problems for pilots, which could easily bring about serious flight accidents. In this paper, a wearable functional near-infrared spectroscopy (fNIRS) device was used to record the changes of hemoglobin concentration of pilots during flight missions. The data was pre-processed, and 1080 valid samples were determined. Then, mean value, variance, standard deviation, kurtosis, skewness, coefficient of variation, peak value, and range of oxyhemoglobin (HbO2) in each channel were extracted. These indexes were regarded as the input of a stacked denoising autoencoder (SDAE) and were used to train the identification model of pilots’ fatigue state. The identification model of pilots’ fatigue status was established. The identification accuracy of the SDAE model was 91.32%, which was 23.26% and 15.97% higher than that of linear discriminant analysis (LDA) models and support vector machines (SVM) models, respectively. Results show that the SDAE model established in our study has high identification accuracy, which can accurately identify different fatigue states of pilots. Identification of pilots’ fatigue status based on fNIRS has important practical significance for reducing flight accidents caused by pilot fatigue.

Daily Activity Recognition and Tremor Quantification from Accelerometer Data for Patients with Essential Tremor Using Stacked Denoising Autoencoders

Human activity recognition (HAR) has received more and more attention, which is able to play an important role in many fields, such as healthcare and intelligent home. Thus, we have discussed an application of activity recognition in the healthcare field in this paper. Essential tremor (ET) is a common neurological disorder that can make people with this disease rise involuntary tremor. Nowadays, the disease is easy to be misdiagnosed as other diseases. We have combined the essential tremor and activity recognition to recognize ET patients’ activities and evaluate the degree of ET for providing an auxiliary analysis toward disease diagnosis by utilizing stacked denoising autoencoder (SDAE) model. Meanwhile, it is difficult for model to learn enough useful features due to the small behavior dataset from ET patients. Thus, resampling techniques are proposed to alleviate small sample size and imbalanced samples problems. In our experiment, 20 patients with ET and 5 healthy people have been chosen to collect their acceleration data for activity recognition. The experimental results show the significant result on ET patients activity recognition and the SDAE model has achieved an overall accuracy of 93.33%. What’s more, this model is also used to evaluate the degree of ET and has achieved the accuracy of 95.74%. According to a set of experiments, the model we used is able to acquire significant performance on ET patients activity recognition and degree of tremor assessment.

A Multimodal Approach for Identifying Autism Spectrum Disorders in Children.

Identification of autism spectrum disorder (ASD) in children is challenging due to the complexity and heterogeneity of ASD. Currently, most existing methods mainly rely on a single modality with limited information and often cannot achieve satisfactory performance. To address this issue, this paper investigates from internal neurophysiological and external behavior perspectives simultaneously and proposes a new multimodal diagnosis framework for identifying ASD in children with fusion of electroencephalogram (EEG) and eye-tracking (ET) data. Specifically, we designed a two-step multimodal feature learning and fusion model based on a typical deep learning algorithm, stacked denoising autoencoder (SDAE). In the first step, two SDAE models are designed for feature learning for EEG and ET modality, respectively. Then, a third SDAE model in the second step is designed to perform multimodal fusion with learned EEG and ET features in a concatenated way. Our designed multimodal identification model can automatically capture correlations and complementarity from behavior modality and neurophysiological modality in a latent feature space, and generate informative feature representations with better discriminability and generalization for enhanced identification performance. We collected a multimodal dataset containing 40 ASD children and 50 typically developing (TD) children to evaluate our proposed method. Experimental results showed that our proposed method achieved superior performance compared with two unimodal methods and a simple feature-level fusion method, which has promising potential to provide an objective and accurate diagnosis to assist clinicians.

CLPM: A Cooperative Link Prediction Model for Industrial Internet of Things Using Partitioned Stacked Denoising Autoencoder

With the development of Industry 4.0, an increasing number of industrial Internet of Things (IIoT) mobile devices (MD), which constantly transmit data at any time, are working on the production line. However, due to node movement, signal attenuation, or physical obstacles, data must rely on the transmission of relay nodes to finally reach the destination node. Based on this scenario, in this article, we propose a cooperative link prediction model (CLPM) using a stacked denoising autoencoder (SDAE) to predict links of the IIoT-based MDs at the next moment through historical link information. The layer structure of the SDAE model is partitioned so that the local MD and edge servers can cooperatively process the link prediction tasks. Experimental results show that our proposed CLPM outperforms others in terms of prediction performance and execution delay.

Urban built-up areas extraction by the multiscale stacked denoising autoencoder technique

Stacked denoising autoencoder (SDAE) model has a strong feature learning ability and has shown great success in the classification of remote sensing images. However, built-up area (BUA) information is easily interfered with by broken rocks, bare land, and other features with similar spectral features. SDAEs are vulnerable to broken and similar features in the image. We propose a multiscale SDAE model to overcome these problems, which can extract BUA features in different scales and recognize the type of land object from multiple scales. The model effectively improves the recognition rate of BUA. The experimental results show that our algorithm can resist the disturbance information, and the classification accuracies are better than support vector machine, backpropagation, random forests, and SDAE. Then we investigate an application in Wuhan (China) metropolitan area analysis with the classification results of our algorithm. The range of the metropolitan area is 1.5-h isochronous circle calculated by Tencent map big data and is divided into three layers: core metropolitan area, subcore metropolitan area, and daily metropolitan. Finally, from the comprehensive statistical data and traffic data, we know that the Wuhan metropolitan area has a “target-shaped” distribution structure radiating outward from the core metropolitan area. It includes five metropolitan development corridors: Wuhan–Huanggang, Wuhan–Xiaogan–Suizhou, Wuhan–Ezhou–Huangshi, Wuhan–Xiantao–Tianmen, and Wuhan–Xianan–Chibi. The corridor is of great significance to the development of metropolitan areas.

Early Warning of Critical Blockage in Coal Mills Based on Stacked Denoising Autoencoders

Coal mills have a significant influence on the reliability, efficiency, and safe operation of a coal-fired power plant. Coal blockage is one of the main reasons for coal mill malfunction. It is highly essential to accurately detect the critical blockage in coal mills to ensure a safe and stable operation of the unit. Taking advantage of unsupervised learning methods and historical process data from distributed control systems (DCS), a stacked denoising autoencoder (SDAE) network–based approach for monitoring critical blockage in a coal mill is proposed in this work. The SDAE model with optimized parameters is applied to reconstruct the operating data during normal operating conditions. The intrinsic relationship between all input variables was captured by training a multilayer network model. The monitoring indicator was obtained from the reconstruction errors, and the threshold for monitoring indicators was obtained using kernel density estimation (KDE) during normal operation. The proposed network is independent of fault data, requires a reduced on-line calculation, and demonstrates a better real-time performance compared to conventional methods. The abnormal variables analysis may provide a theoretical evidence for critical blockage. The effectiveness of the proposed method is validated using operating data collected from an actual coal-fired power plant in China. The results demonstrated that the proposed method can effectively detect critical blockage in a coal mill and issue a timely warning, which allows operators to detect potential faults.

Health condition identification for rolling bearing using a multi-domain indicator-based optimized stacked denoising autoencoder

Stacked denoising autoencoder is one of the most classic models of deep learning. However, there are two problems in the traditional stacked denoising autoencoder: (1) the parameter selection of stacked denoising autoencoder mainly depends on expert experience and (2) stacked denoising autoencoder is mainly restricted to learn automatically single-domain features from raw vibration signals while identifying the fault type, which implies that no linear mapping relationship located in other domains of vibration data is neglected, which may lead to the imperfect diagnostic results. Consequently, to address these issues, learn the well-rounded feature representation, and improve recognition accuracy, this article presents a novel approach called multi-domain indicator-based optimized stacked denoising autoencoder for automatic fault identification of rolling bearing. First, multi-domain indicator of the original vibration signal is constructed through calculating the expression of different domains (e.g. time frequency domain, and time frequency domain). Second, the constructed multi-domain indicator is regarded as the input dataset to train stacked denoising autoencoder architecture containing three hidden layers, and a recently reported nature-inspired algorithm named grasshopper optimization algorithm is employed to synchronously determine the model parameters of stacked denoising autoencoder, which is aimed at learning more robust and reliable feature representation. Finally, the feature representation learned in the testing set is fed into the trained stacked denoising autoencoder model containing softmax classifier for identifying bearing health conditions. The presented method is evaluated using two bearing vibration datasets. Experimental results indicate that our approach can provide high-accuracy recognition over 99% for bearing health condition, and it achieves more decent and precise classification results compared with some shallow learning model and standard deep learning architecture.

A deep autoencoder feature learning method for process pattern recognition

Recognition of various defect patterns exhibited in discrete manufacturing processes can significantly reduce the diagnostic processes, and increase manufacturing process stability and quality. Thus the effective recognizers are in great demand to improve the performance of process pattern recognition (PPR). Deep learning has been widely applied in image and visual analysis with great successes. However, the application of deep learning in feature learning for process control is still few. This paper presents an effective and reliable deep learning method known as stacked denoising autoencoder (SDAE) for PPR in manufacturing processes. This paper will concentrate on developing an SDAE model to learn effective features from the process signals and then implementing an effective PPR through a deep network architecture. Feature visualization is also performed to explicitly present the feature representation of the proposed SDAE model. The effectiveness of the proposed PPR method is verified through a big simulation dataset and Tennessee Eastman process. The result shows that the proposed method obtains good feature learning and PPR performance.

A New Online Detection Approach for Rolling Bearing Incipient Fault via Self-Adaptive Deep Feature Matching

This paper presents a new online detection approach for rolling bearing’s incipient fault based on self-adaptive deep feature matching (SDFM). This approach includes offline and online stages. At the offline stage, a new health state assessment algorithm is first proposed based on singular value decomposition (SVD) and Kurtosis criterion. Based on the assessment results, a kind of deep learning algorithm, i.e., stacked denoising autoencoder (SDAE), is introduced to extract the common deep features of normal state and early fault state. Support vector data description (SVDD) is applied to establish the offline detection model using the obtained features. At the online stage, a self-adaptive matching strategy with 1-Dimensional anchor is proposed. By utilizing the SDAE model established at offline stage, this strategy can extract more representative deep features of the target bearing via generating various proposal fragments and then determining the fault occurrence time in a self-adaptive way by feeding the online features into the SVDD model. Experiments run on the bearing data set of IEEE prognostic and health management (PHM) Challenge 2012. The results show the proposed approach has good detection performance in real time and much lower false alarm rate, with no need to acquire fault characteristic frequency in advance.

Stacked denoising autoencoder‐based feature learning for out‐of‐control source recognition in multivariate manufacturing process

AbstractIn multivariate statistical process control (MSPC), regular multivariate control charts (eg, T2) are shown to be effective in detecting out‐of‐control signals based upon an overall statistic. But these charts do not relieve the need for multivariate process pattern recognition (MPPR). MPPR would be very useful for quality operators to locate the assignable causes that give rise to out‐of‐control situation in multivariate manufacturing process. Deep learning has been widely applied and obtained many successes in image and visual analysis. This paper presents an effective and reliable deep learning method known as stacked denoising autoencoder (SDAE) for MPPR in manufacturing processes. This study will concentrate on developing a SDAE model to learn effective discriminative features from the process signals through deep network architectures. Feature visualization is performed to explicitly present feature representations of the proposed SDAE model. The experimental results illustrate that the proposed SDAE model is capable of implementing detection and recognition of various process patterns in complicated multivariate processes. Analysis from this study provides the guideline in developing deep learning‐based MSPC systems.

Traffic State Estimation of Signalized Intersections Based on Stacked Denoising Auto-Encoder Model

To solve the problem of fuzziness and uncertainty of traffic states in signalized intersection, a method was proposed for estimating traffic condition based on stacked de-noising auto-encoder model. The simulation data and empirical data were used to train the model, and the K-means clustering method was used to determine the traffic state thresholds and the data were divided into three categories based on the threshold values. Relevant features based on the reconstruction theory of de-noising auto-encoder were automatically extracted, and unsupervised greedy layer-wise pre-training and supervised fine-tuning were utilized to train the deep auto-encoder network, so that it had good robust performance on obtaining the traffic state characters with low quality in the complex environment. From the experimental results, the proposed method obtains an accuracy of 91.5% in simulation data and 88% in empirical data, which is better by 7.1% than using decision tree model.

Short-Term Electricity Price Forecasting With Stacked Denoising Autoencoders

A short-term forecasting of the electricity price with data-driven algorithms is studied in this research. A stacked denoising autoencoder (SDA) model, a class of deep neural networks, and its extended version are utilized to forecast the electricity price hourly. Data collected in Nebraska, Arkansas, Louisiana, Texas, and Indiana hubs in U.S. are utilized. Two types of forecasting, the online hourly forecasting and day-ahead hourly forecasting, are examined. In online forecasting, SDA models are compared with data-driven approaches including the classical neural networks, support vector machine, multivariate adaptive regression splines, and least absolute shrinkage and selection operator. In the day-ahead forecasting, the effectiveness of SDA models is further validated through comparing with industrial results and a recently reported method. Computational results demonstrate that SDA models are capable to accurately forecast electricity prices and the extended SDA model further improves the forecasting performance.

Stacked Denoising Autoencoders Applied to Star/Galaxy Classification

In recent years, the deep learning algorithm, with the characteristics of strong adaptability, high accuracy, and structural complexity, has become more and more popular, but it has not yet been used in astronomy. In order to solve the problem that the star/galaxy classification accuracy is high for the bright source set, but low for the faint source set of the Sloan Digital Sky Survey (SDSS) data, we introduced the new deep learning algorithm, namely the SDA (stacked denoising autoencoder) neural network and the dropout fine-tuning technique, which can greatly improve the robustness and antinoise performance. We randomly selected respectively the bright source sets and faint source sets from the SDSS DR12 and DR7 data with spectroscopic measurements, and made preprocessing on them. Then, we randomly selected respectively the training sets and testing sets without replacement from the bright source sets and faint source sets. At last, using these training sets we made the training to obtain the SDA models of the bright sources and faint sources in the SDSS DR7 and DR12, respectively. We compared the test result of the SDA model on the DR12 testing set with the test results of the Library for Support Vector Machines (LibSVM), J48 decision tree, Logistic Model Tree (LMT), Support Vector Machine (SVM), Logistic Regression, and Decision Stump algorithm, and compared the test result of the SDA model on the DR7 testing set with the test results of six kinds of decision trees. The experiments show that the SDA has a better classification accuracy than other machine learning algorithms for the faint source sets of DR7 and DR12. Especially, when the completeness function is used as the evaluation index, compared with the decision tree algorithms, the correctness rate of SDA has improved about 15% for the faint source set of SDSS-DR7.

A Deep Learning Model for Robust Wafer Fault Monitoring With Sensor Measurement Noise

Standard fault detection and classification (FDC) models detect wafer faults by extracting features useful for fault detection from time-indexed measurements of the equipment recorded by in situ sensors (sensor signals) and feeding the extracted information into a classifier. However, the preprocessing-and-classification approach often results in the loss of information in the sensor signals that is important for detecting wafer faults. Furthermore, the sensor signals usually contain noise induced by mechanical and electrical disturbances. In this paper, we propose the use of a stacked denoising autoencoder (SdA), which is a deep learning algorithm, to establish an FDC model for simultaneous feature extraction and classification. The SdA model can identify global and invariant features in the sensor signals for fault monitoring and is robust against measurement noise. Through experiments using wafer samples collected from a work-site photolithography tool, we confirmed that as the sensor measurement noise severity increased, the SdA's classification accuracy could be as much as 14% higher than those of the twelve models considered for comparison, each of which employed one of three feature extractors and one of four classifiers.

Cleaning Method for Status Monitoring Data of Power Equipment Based on Stacked Denoising Autoencoders

Currently, the cleaning process for power equipment monitoring data is cumbersome and often leads to loss of information. To address these problems, a data cleaning method based on stacked denoising autoencoder (SDAE) networks is proposed in this paper. SDAE networks have a strong ability to denoise and restore corrupted data and have a strong feature extraction capability. The status monitoring data of equipment under normal conditions are trained by SDAE to obtain the cleaning parameters and the reconstruction errors. An upper threshold of the reconstruction errors obtained from training samples is determined through Kernel density estimation. A tolerance window width is added to achieve rapid anomaly detection. The abnormal data are classified as outliers, missing data, or fault status data according to the relationship between the reconstruction error and the threshold and between the duration of abnormal data and the tolerance window. To verify the effectiveness of the proposed method, the SDAE model is used to process the data for the dissolved gas concentration in transformer oil and the temperature of the transmission line. The results show that the proposed method can effectively identify and repair outliers and missing information. The model can perform rapid anomaly detection when the equipment is running abnormally.

Fault diagnosis of rotary machinery components using a stacked denoising autoencoder-based health state identification

Effective fault diagnosis has long been a research topic in the prognosis and health management of rotary machinery engineered systems due to the benefits such as safety guarantees, reliability improvements, and economical efficiency. This paper investigates an effective and reliable deep learning method known as stacked denoising autoencoder (SDA), which is shown to be suitable for certain health state identifications for signals containing ambient noise and working condition fluctuations. SDA has become a popular approach to achieve the promised advantages of deep architecture-based robust feature representations. In this paper, the SDA-based fault diagnosis method contains three successive steps: health states are first divided into training and testing groups for the SDA model, a deep hierarchical structure is then established with a transmitting rule of greedy training, layer by layer, where sparsity representation and data destruction are applied to obtain high-order characteristics with better robustness in the iteration learning. Validation data are finally employed to confirm the fault diagnosis results of the SDA, where existing health state identification methods are used for comparison. Rotating machinery datasets are employed to demonstrate the effectiveness of the proposed method.