Sparse Denoising Autoencoder Research Articles

Classification model of electricity consumption behavior based on sparse denoising autoencoder feature dimensionality reduction and spectral clustering

The development of electrical measurement technology has brought high latitude residential electricity consumption data to power companies, which contains the characteristics of users' electricity consumption behavior and provides data support for the behavior classification. In order to improve the efficiency of data feature extraction and the accuracy of electricity consumption behavior identification, a classification model based on sparse denoising autoencoder feature dimensionality reduction and spectral clustering is proposed in this paper. Firstly, the sparse denoising autoencoder (SDAE) and the manually defining electricity consumption characteristic indicators are deployed to extract features from the residential daily electricity consumption data, and then the spectral clustering is employed to classify the extracted electricity consumption characteristics. Secondly, the t-distributed stochastic neighbor embedding (t-SNE) is applied to visualize and analyze the classification results, and on this basis, the secondary classification is implemented to fix the issue of the confused electricity consumption behaviors. Finally, the typical consumption behavior curves are calculated by Gaussian distance weighting method, and the characteristics of power consumption behavior are analyzed and summarized. The proposed approach is evaluated and verified by using the electricity dataset in Fujian, China.

Comparing the Performance of Deep Denoising Sparse Autoencoder with Other Defense Methods Against Adversarial Attacks for Arabic letters

The aim of this paper is to compare how effectively the Deep Denoising Sparse Autoencoder (DDSA) method performs compared to other defense strategies - like adversarial training, defensive distillation and feature squeezing - in dealing with adversarial attacks for Arabic letters. We strive to evaluate both the accuracy and robustness as well as efficiency of these methods by examining a test set from the Arabic Handwritten Characters Dataset while considering adversarial attacks. Fast Gradient Sign Method (FGSM), Projected Gradient Descent (PGD), and Carlini and Wagner (C&W) are all part of this. Our research findings demonstrate that DDSA surpasses the rest of the defense methods in terms of classification accuracy and robustness. This exceptional performance is due to the distinctive attributes of DDSA, which concentrate on acquiring distinguishing features and integrating spatial information to improve defense against adversarial perturbations. While it necessitates more computational resources, DDSA's superior performance validates the additional expenses, particularly in critical applications where misclassification may have severe implications.

Incipient fault detection and process monitoring of thermal power plant pulverizing system based on deep representation learning

Process data in contemporary thermal power plants shows the characteristics of large capacity, strong coupling, and high-order nonlinearity, which brings great challenges to process monitoring and fault detection. A deep representation learning fault detection scheme based on stacked sparse denoising autoencoder (SSDAE) is proposed in this paper. Specifically, to enhance the capabilities of noise reduction and feature representation for complex industrial data, the sparse denoising autoencoder (SDAE) is proposed by considering both noise and sparsity constraints. Then, a deep learning architecture is constructed by stacking multiple SDAEs layer by layer to achieve a highly nonlinear representation capability. Based on the low-dimensional representation and residual distance of SSDAE, three monitoring indicators, RE2, MD2, and ZD2, are designed by different distance metrics and the k-nearest neighbor (KNN) discriminant rule. The effectiveness of the proposed method is validated by studying a nonlinear numerical case and a practical power plant pulverizing system. The experimental results demonstrate that the proposed method can effectively detect incipient and slight faults that are difficult to detect with traditional methods.

Robust Deep Learning for Uplink Channel Estimation in Cellular Network Under Inter-Cell Interference

Deep learning (DL)-based channel estimation has achieved remarkable success. However, most existing works focus on the white Gaussian noise which are inapplicable for cell-edge users under inter-cell interference (ICI). In this paper, we address this issue by proposing a novel DL-based channel estimation solution with a cascaded model-based and model-free deep neural network (DNN) structure. Specifically, the model-based module is designed by the variational Bayesian inference (VBI) technique to suppress the time-varying ICI, and the model-free module is designed by the Denoising Sparse Autoencoder (DSAE) structure to further refine the channel estimation. The proposed DNN is firstly pre-trained by offline supervised training, and various channel statistics are encapsulated in the DNN weights with the assist of a hyper-prior net modelling different sparse priors for different training samples. Then, an online Bayesian learning algorithm is proposed to train the model-based VBI module based on real-time pilot samples to track the online channel statistics. Simulation results verify that the proposed solution outperforms various state-of-the-art baseline schemes in a large SINR range with comparable performance to the estimator with genie-aided channel statistics.

Internet of things with nanomaterials-based predictive model for wastewater treatment using stacked sparse denoising auto-encoder

Abstract Wastewater is a serious concern for the environment. There is a substantial amount of toxins that are discharged continuously from several pharmacological companies that lead to serious damage to public health and the ecosystem. Present wastewater treatment technologies include primary, tertiary, and secondary treatments that remove numerous contaminants; but pollutants in the nanoscale range were hard to remove with these steps. Some of these include inorganic and organic pollutants, pathogens, pharmaceuticals, and pollutants of developing concern. The utility of nanoparticles was a promising solution to this issue. Nanoparticles have exclusive properties permitting them to potentially eliminate residual pollutants but being eco-friendly and inexpensive. This study develops a new Archimedes optimization algorithm (AOA) with Stacked Sparse Denoising Auto-Encoder (SSDAE) model, named AOA-SSDAE for wastewater management in the IoT environment. The presented AOA-SSDAE technique aims to predict wastewater treatment depending on the influent indicators. In the presented AOA-SSDAE technique, the IoT devices are initially employed for the data collection process and then data normalization is performed to transform the collected data into a uniform format. For the predictive process, the SSDAE model is employed in this paper. To improve the SSDAE model's prediction capability, the AOA-based hyperparameter tuning process is involved.

Toward Robust Fault Identification of Complex Industrial Processes Using Stacked Sparse-Denoising Autoencoder With Softmax Classifier.

This article proposes a robust end-to-end deep learning-induced fault recognition scheme by stacking multiple sparse-denoising autoencoders with a Softmax classifier, called stacked spare-denoising autoencoder (SSDAE)-Softmax, for the fault identification of complex industrial processes (CIPs). Specifically, sparse denoising autoencoder (SDAE) is established by integrating a sparse AE (SAE) with a denoising AE (DAE) for the low-dimensional but intrinsic feature representation of the CIP monitoring data (CIPMD) with possible noise contamination. SSDAE-Softmax is established by stacking multiple SDAEs with a layerwise pretraining procedure, and a Softmax classifier with a global fine-tuning strategy. Furthermore, SSDAE-Softmax hyperparameters are optimized by a relatively new global optimization algorithm, referred to as the state transition algorithm (STA). Benefiting from the deep learning-based feature representation scheme with the STA-based hyperparameter optimization, the underlying intrinsic characteristics of CIPMD can be learned automatically and adaptively for accurate fault identification. A numeric simulation system, the benchmark Tennessee Eastman process (TEP), and a real industrial process, that is, the continuous casting process (CCP) from a top steel plant of China, are used to validate the performance of the proposed method. Experimental results show that the proposed SSDAE-Softmax model can effectively identify various process faults, and has stronger robustness and adaptability against the noise interference in CIPMD for the process monitoring of CIPs.

Defending Adversarial Examples by a Clipped Residual U-Net Model

Deep learning-based systems have succeeded in many computer vision tasks. However, it is found that the latest study indicates that these systems are in danger in the presence of adversarial attacks. These attacks can quickly spoil deep learning models, e.g., different convolutional neural networks (CNNs), used in various computer vision tasks from image classification to object detection. The adversarial examples are carefully designed by injecting a slight perturbation into the clean images. The proposed CRU-Net defense model is inspired by state-of-the-art defense mechanisms such as MagNet defense, Generative Adversarial Network Defense, Deep Regret Analytic Generative Adversarial Networks Defense, Deep Denoising Sparse Autoencoder Defense, and Condtional Generattive Adversarial Network Defense. We have experimentally proved that our approach is better than previous defensive techniques. Our proposed CRU-Net model maps the adversarial image examples into clean images by eliminating the adversarial perturbation. The proposed defensive approach is based on residual and U-Net learning. Many experiments are done on the datasets MNIST and CIFAR10 to prove that our proposed CRU-Net defense model prevents adversarial example attacks in WhiteBox and BlackBox settings and improves the robustness of the deep learning algorithms especially in the computer vision field. We have also reported similarity (SSIM and PSNR) between the original and restored clean image examples by the proposed CRU-Net defense model.

Leakage aperture identification of natural gas pipeline based on compressed acquisition and DSAE

Traditional gas pipeline leak detection methods usually face challenges in that there are redundant data collected on account of the time-domain sampling theory and the valuable leakage information is hidden in the complex vibration signals. This paper put forward an intelligent aperture identification method for natural gas pipeline leakage. This method integrated the compression of monitoring data based on compressed sensing (CS) transformation and automatic feature extraction and identification based on deep neural networks, namely, denoising sparse autoencoder (DSAE). The compressed acquisition can greatly reduce the volume to be processed and effectively extract the aperture information. DSAE combined the merits of sparse auto-encoder (SAE), denoising coding and dropout to achieve robust feature extraction and aperture classification. Experimental results validate the effectiveness and good performance, and the proposed method can realize faster and more accurate identification of different leak sizes compared with the traditional methods.

Toward robust process monitoring of complex process industries based on denoising sparse auto-encoder

Robust industrial process monitoring is crucial to ensure production safety and product quality stability. However, the process monitoring of high-dimensional, nonlinear, complex industrial processes is still challenging due to their inherent complexities, such as multi-phase, multi-field, and tight coupling of multiple sub-processes. In this article, a simple yet robust nonlinear process monitoring scheme based on a kind of denoising sparse auto-encoder (DSAE) is proposed. Specifically, a novel hybrid auto-encoder, namely DSAE, is established, to address the strong redundancy, nonlinearity and noise interference in process variables by integrating a sparse auto-encoder (SAE) and a denoising auto-encoder (DAE). Successively, an online process monitoring model is established by introducing two new process monitoring statistics computed on the DSAE-based feature representation space and residual space, respectively. Moreover, the Kernel density estimation (KDE) approach is adopted to determine the corresponding control limits of the monitoring statistics, which can avoid the conventional empirical assumption of F or Chi-square distribution on monitoring statistics. Extensive confirmative and comparative experiments conducted for the fault monitoring of a nonlinear numerical simulation system, the benchmark Tennessee Eastman process (TEP) and ventilation exhaust fans (VEFs) in the continuous casting process (CCP) from a top steel plant in China show that the proposed method is promising. Specifically, the proposed method performs favorably against the representative process monitoring methods, and it has especially a stronger robustness against noise and uncertain interference in nonlinear process systems.

Ship Shaft Frequency Extraction Based on Improved Stacked Sparse Denoising Auto-Encoder Network

The modulation spectrum of ship radiated noise contains information on shaft frequency, which is an important feature used to identify ships and a key parameter involved in calculating the number of propeller blades. To improve the shaft frequency extraction accuracy, a ship shaft frequency extraction method based on an improved stacked sparse denoising auto-encoder network (SSDAE) is proposed. Firstly, the mathematical model of the ship radiated noise modulation spectrum is built and data simulation is carried out based on this model, combined with the actual ship parameters. Secondly, we trained the SSDAE model using the simulation data and made slight adjustments to this model by using both simulation and measured data to improve it. Finally, the experimental ship modulation spectrum information was input to the SSDAE model for denoising, enhancement, and regression estimation. Accordingly, the shaft frequency was extracted. The simulation and experimental results show that the shaft frequency extraction method based on the improved SSDAE model has high accuracy and good robustness, especially under the conditions of both missing line spectra and noise interference.

Modified metaheuristics with stacked sparse denoising autoencoder model for cervical cancer classification

Cervical cancer is the most commonly diagnosed cancer among women globally, with high mortality rate. For early diagnosis, automated and accurate cervical cancer classification approaches can be developed through effective classification of Pap smear cell images. The current study introduces a novel Modified Firefly Optimization Algorithm with Deep Learning-enabled cervical cancer classification (MFFOA-DL3) model for the classification of Pap Smear Images (PSI). The proposed MFFOA-DL3 model examines the PSI for the existence of cervical cancer cells. To accomplish this, the proposed MFFOA-DL3 model primarily applies Bilateral Filtering (BF)-based noise removal approach to get rid of the noise. Then, Kapur's entropy-based image segmentation technique is applied to determine the affected regions. Moreover, EfficientNet technique is also applied to generate the feature vectors. Finally, MFFOA with Stacked Sparse Denoising Autoencoder (SSDA) model is exploited to classify the PSI. In current study, MFFOA is utilized to appropriately modify the parameters related to SSDA model. The proposed MFFOA-DL3 model was experimentally validated using benchmark dataset. The results attained from extensive comparative analysis highlighted the better performance of MFFOA-DL3 model over other recent approaches.

M-ary Phase Position Shift Keying Demodulation Using Stacked Denoising Sparse Autoencoders

A deep-learning based detector for M-ary phase position shift keying (MPPSK) systems is proposed in this paper. The major components of this detector include a special impact filter, a stacked denoising sparse autoencoder (DSAE), which was trained in unsupervised learning to extract features from the modulation signals, and a softmax classifier. The features learned by the stacked DSAE were then used to train the softmax classifier to demodulate the received signals into M classes. The architecture presented herein was trained and tested on a simple dataset extended by adding Gaussian noise only. The results from the theoretical analysis and simulation show that the detection performance of the proposed scheme is superior to that of existing detectors.

A Double-Weighted Deterministic Extreme Learning Machine Based on Sparse Denoising Autoencoder and Its Applications

Extreme learning machine (ELM) is a feedforward neural network-based machine learning method that has the benefits of short training times, strong generalization capabilities, and will not fall into local minima. However, due to the traditional ELM shallow architecture, it requires a large number of hidden nodes when dealing with high-dimensional data sets to ensure its classification performance. The other aspect, it is easy to degrade the classification performance in the face of noise interference from noisy data. To improve the above problem, this paper proposes a double pseudo-inverse extreme learning machine (DPELM) based on Sparse Denoising AutoEncoder (SDAE) namely, SDAE-DPELM. The algorithm can directly determine the input weight and output weight of the network by using the pseudo-inverse method. As a result, the algorithm only requires a few hidden layer nodes to produce superior classification results when classifying data. And its combination with SDAE can effectively improve the classification performance and noise resistance. Extensive numerical experiments show that the algorithm has high classification accuracy and good robustness when dealing with high-dimensional noisy data and high-dimensional noiseless data. Furthermore, applying such an algorithm to Miao character recognition substantiates its excellent performance, which further illustrates the practicability of the algorithm.

Intelligent fault diagnosis of machinery using digital twin-assisted deep transfer learning

Digital twin (DT) is emerging as a key technology for smart manufacturing. The high fidelity DT model of the physical assets can produce system performance data that is close to reality, which provides remarkable opportunities for machine fault diagnosis when the measured fault condition data are insufficient. This paper presents an intelligent fault diagnosis framework for machinery based on DT and deep transfer learning. First, the DT model of the machine is built by establishing the simulation model and with further updating through continuously measured data from the physical asset. Second, all important machine conditions can be simulated from the built DT. Third, a new-type deep structure based on novel sparse de-noising auto-encoder (NSDAE) is developed and pre-trained with condition data from the source domain, as generated from the DT. Then, to achieve accurate machine fault diagnosis with possible variations in working conditions and system characteristics, the pre-trained NSDAE is fine-tuned using parameter transfer with only one sample from the target domain. The presented method is validated through a case study of triplex pump fault diagnosis. The experimental results demonstrate that the proposed method achieves intelligent fault diagnosis with a limited amount of measured data and outperforms other state-of-the-art data-driven methods.

Towards robust and understandable fault detection and diagnosis using denoising sparse autoencoder and smooth integrated gradients

Industrial applications of fault detection and diagnosis face great challenges as they require not only accurate identification of faulty statuses but also the effective understandability of the results. In this paper, a two-step robust and understandable fault detection and diagnosis framework is developed to address this challenge by exploiting denoising sparse autoencoder and smooth integrated gradients. Specifically, denoising sparse autoencoder(DSAE) is utilized to detect faults in the first step. DSAE is more robust to noise corruption and has better generalization performance compared to the existing autoencoder-based methods. In the second step, smooth integrated gradients(SIG) is used to diagnose the root-cause variables of the faults detected. Smooth integrated gradients can provide a denoising effect on the feature importance. The proposed framework is evaluated through an application to the Tennessee Eastman process. As proved in the experiments, the presented DSAE-SIG method not only achieves higher diagnosis accuracy but also successfully identifies the potential root-cause variables of process disturbances.

RETRACTED: A Novel Day- ahead Electricity Price Forecasting Using multi-modal combined Integration via Stacked Pruning Sparse Denoising Auto Encoder

One of the best options for managing energy markets and power system performance optimization is precise Electricity Price Forecasting (EPF). In detecting the prices of energy, some factors are of stochastic behavior which has made an unwieldy task. In this study, carrying out Tensor Canonical Correlation Analysis (TCCA) is the first activity to choose the operative parameters in EPF issues and removing redundant factors with low correlation ranks. The second activity is making use of a Deep Neural Network (DNN) with a novel Stacked Pruning Sparse Denoising Autoencoder (SPSDAE) to decrease the noise of data sets with different sources individually. The third activity is that a novel method which is called the Maximum Separation Subspace (MASES) in Sufficient Dimension Reduction (SDR) with Categorical Response. This method is used to detect the outstanding properties of the input data and removing the uncertainty characteristics. The new multi-modal combined (MMC) method is the final step that needs to be taken to predict the day-ahead (one day in advance) cost of electricity. The proposed method is investigated based on the data that were collected from Australia The combination of various predicting models and suggested methods might develop the function of predicting possibility by supplying divergent types of compactness performance. The combined model comprised of three predicting possibilities called beta circulation fitting, scattered Bayesian learning, core compactness prediction. Predicting data respectively are used as a yardstick to compare the efficiency of forecasted results and consequently assess the proposed method. Results of this study confirmed a decrease of effectual error criterion and accuracy enhancement in EPF field as the advantageous effects of this proposed SDR structure.

An improved pollution forecasting model with meteorological impact using multiple imputation and fine-tuning approach

Air pollution forecasting is a significant step for air quality pollution management to mitigate pollution’s negative impact on the environment and people’s health. The data-driven forecasting model can help a better understanding of environmental air quality. The existing data-driven forecasting models usually ignore missing values, the correlations between the pollutant and meteorological factors and fail to perform temporal modeling effectively, affecting prediction accuracy. In response to these issues, we present a deep learning-based Convolutional LSTM–SDAE (CLS) model to forecast the particulate matter level, revealing the correlation between particulate matter and meteorological factors. In the proposed architecture, the k nearest neighbor (KNN) imputation technique is employed to recover the air quality dataset’s missing values. The Convolutional Long Short Term Memory (CNN–LSTM) unit identifies the vast dataset’s hidden features and performs pollutants’ temporal modeling. In addition, Bidirectional Gatted Recurrent Unit (BIGRU) is implemented as both encoder and decoder in Sparse Denoising Autoencoder, which reconstructs the CNN–LSTM model’s output in the dynamic fine-tuning layer to get robust prediction results. The experimental results in Talcher, India, and Beijing, China indicate that the model can improve forecasting accuracy and outperforms the other state of art and baseline models.

Machine Learning Applied to Problem-Solving in Medical Applications

Physical health plays an important role in overall well-being of the human beings. It is the most observed dimension of health among others such as social, intellectual, emotional, spiritual and environmental dimensions. Due to exponential increase in the development of wireless communication techniques, Internet of Things (IoT) has effectively penetrated different aspects of human lives. Healthcare is one of the dynamic domains with ever-growing demands which can be met by IoT applications. IoT can be leveraged through several health service offerings such as remote health and monitoring services, aided living, personalized treatment, and so on. In this scenario, Deep Learning (DL) models are employed in proficient disease diagnosis. The current research work presents a new IoT-based physical health monitoring and management method using optimal Stacked Sparse Denoising Autoencoder (SSDA) technique i.e., OSSDA. The proposed model utilizes a set of IoT devices to collect the data from patients. Imbalanced class problem poses serious challenges during disease diagnosis process. So, the OSSDA model includes Synthetic Minority Over-Sampling Technique (SMOTE) to generate artificial minority class instances to balance the class distribution. Further, the hyperparameter settings of the OSSDA model exhibit heavy influence upon the classification performance of SSDA technique. The number of hidden layers, sparsity, and noise count are determined by Sailfish Optimizer (SFO). In order to validate the effectiveness and performance of the proposed OSSDA technique, a set of experiments was conducted on diabetes and heart disease datasets. The simulation results portrayed a proficient diagnostic outcome from OSSDA technique over other methods. The proposed method achieved the highest accuracy values i.e., 0.9604 and 0.9548 on the applied heart disease and diabetes datasets respectively.

Tool Wear Status Recognition and Prediction Model of Milling Cutter Based on Deep Learning

In order to ensure the reliability and stability of the manufacturing process, tool wear state should be realized real-time and accurate monitoring. This paper proposes a tool wear state recognize and predictive framework model based on Stacking Sparse De-noising Auto-Encoder (SSDAE), the Particle Swarm Optimization (PSO) and the Least Squares Support Vector Machine (LSSVM). The Stacking Sparse De-noising Auto-Encoder (SSDAE) technique is utilized to realize multi-feature signal dimension reduction with the aim of improving the prediction accuracy, which reduces the dependence on the prior knowledge of feature selection and greatly improves the modeling efficiency. PSO technique is helpful for adaptive optimization of kernel parameters, which greatly improved computing power and LSSVM model prediction accuracy. A dataset from a real machining process is utilized to verify the effectiveness of proposed model in improving the prediction accuracy. The experimental results show that a high correlation coefficient greater than 0.95 is used to extract feature vector from time domain, frequency domain and time-frequency domain three directions, and the proposed SSDAE-PSO-LSSVM model performs better than Partial Least Squares Regression (PLSR), Back Propagation Neural Network (BPNN) and Extreme Learning Machine (ELM) in terms of prediction accuracy.

The fault diagnosis method of rolling bearing under variable working conditions based on deep transfer learning

The vibration signals of rolling bearing obtained under variable working conditions do not obey the same independent distribution so that the traditional method of bearing fault diagnosis has low accuracy, a fault diagnosis method about rolling bearing based on sparse denoising autoencoder (SDAE) for deep feature extraction combining transfer learning is proposed. First, the bearing vibration signal in the time domain is transformed for frequency domain signal via Fourier transform, which is input into the SDAE for adaptive deep feature extraction. Then, the joint geometrical and statistical alignment is introduced to deal with the deep feature samples for reducing the domain discrepancy both statistically and geometrically. Finally, the k-nearest neighbor classification algorithm is used for completing the fault diagnosis of rolling bearing under variable working conditions. The experimental results show that the method presented in the paper improves the accuracy rate of fault diagnosis about rolling bearing under variable working conditions, verifies its feasibility and effectiveness.