Sparse Denoising Autoencoder Research Articles

Power Grid Fault Diagnosis Method Based on Stacked Sparse Denoising Auto-Encoder and GRU Network

This research aims to solve the problems of feature extraction, data dimension reduction, gradient dissipation and low training efficiency in power grid fault classification. Aiming at PMU measurement data with high dimensionality and strong real-time characteristics, a fault diagnosis method based on Stacked Sparse Denoising Auto-Encoder and GRU network is proposed. The Stacked Sparse Denoising Auto-Encoder is used to reduce the sequence dimension to obtain the sparse feature expression of the data, and then the time-dependent features of the data extracted by the GRU are used to obtain the fault type. Simulations and experiments show that compared with the traditional neural network algorithm, the proposed method can effectively extract high-dimensional data features, reduce data dimensions, improve the efficiency of GRU network classification, accelerate the convergence speed and reduce the training time, and has better stability and higher accuracy.

A stacked convolutional sparse denoising autoencoder model for underwater heterogeneous information data

Deep learning denoising models can automatically extract underwater heterogeneous information data features to improve denoising performance through an internal network structure. In the present study, a novel stacked convolutional sparse denoising autoencoder (SCSDA) model was proposed in this paper to complete the blind denoising task of underwater heterogeneous information data. Specifically, for the first time, the stacked sparse denoising autoencoder (SSDA) was constructed by three sparse denoising autoencoders (SDA) to extract overcomplete sparse features. Then, the output of the last encoding layer of the SSDA was used as the input of the convolutional neural network (CNN) to further extract the deep features. Finally, in order to solve the lack of the pure underwater heterogenous information data during the acquisition and transmission process, the unrelated dataset was developed to simulate the underwater heterogeneous information data as the training set in proposed SCSDA model. Compared with the existing other algorithms, the experiment results demonstrate that the proposed SCSDA model combines the advantages of SSDA and CNN, which has great blind denoising ability. It can process faster and preserves more edge features of underwater heterogeneous information data. Also, it has a certain degree of robustness and effectiveness.

Monitoring Depth of Anesthesia Based on Hybrid Features and Recurrent Neural Network.

Electroencephalogram (EEG) signals contain valuable information about the different physiological states of the brain, with a variety of linear and nonlinear features that can be used to investigate brain activity. Monitoring the depth of anesthesia (DoA) with EEG is an ongoing challenge in anesthesia research. In this paper, we propose a novel method based on Long Short-Term Memory (LSTM) and a sparse denoising autoencoder (SDAE) to combine the hybrid features of EEG to monitor the DoA. The EEG signals were preprocessed using filtering, etc., and then more than ten features including sample entropy, permutation entropy, spectra, and alpha-ratio were extracted from the EEG signal. We then integrated the optional features such as permutation entropy and alpha-ratio to extract the essential structure and learn the most efficient temporal model for monitoring the DoA. Compared with using a single feature, the proposed model could accurately estimate the depth of anesthesia with higher prediction probability (Pk). Experimental results evaluated on the datasets demonstrated that our proposed method provided better performance than the methods using permutation entropy, alpha-ratio, LSTM, and other traditional indices.

Unsupervised abnormality detection through mixed structure regularization (MSR) in deep sparse autoencoders.

The purpose of this study is to introduce and evaluate the mixed structure regularization (MSR) approach for a deep sparse autoencoder aimed at unsupervised abnormality detection in medical images. Unsupervised abnormality detection based on identifying outliers using deep sparse autoencoders is a very appealing approach for computer-aided detection systems as it requires only healthy data for training rather than expert annotated abnormality. However, regularization is required to avoid overfitting of the network to the training data. We used coronary computed tomography angiography (CCTA) datasets of 90 subjects with expert annotated centerlines. We segmented coronary lumen and wall using an automatic algorithm with manual corrections where required. We defined normal coronary cross section as cross sections with a ratio between lumen and wall areas larger than 0.8. We divided the datasets into training, validation, and testing groups in a tenfold cross-validation scheme. We trained a deep sparse overcomplete autoencoder model for normality modeling with random structure and noise augmentation. We assessed the performance of our deep sparse autoencoder with MSR without denoising (SAE-MSR) and with denoising (SDAE-MSR) in comparison to deep sparse autoencoder (SAE), and deep sparse denoising autoencoder (SDAE) models in the task of detecting coronary artery disease from CCTA data on the test group. The SDAE-MSR achieved the best aggregated area under the curve (AUC) with a 20% improvement and the best aggregated Average Precision (AP) with a 30% improvement upon the SAE and SDAE (AUC: 0.78 to 0.94, AP: 0.66 to 0.86) in distinguishing between coronary cross sections with mild stenosis (stenosis grade<0.3) and coronary cross sections with severe stenosis (stenosis grade>0.7). The improvements were statistically significant (Mann-Whitney U-test, P<0.001). Similarly, The SDAE-MSR achieved the best aggregated AUC (AP) with an 18% (18%) improvement upon the SAE and SDAE (AUC: 0.71 to 0.84, AP: 0.68 to 0.80). The improvements were statistically significant (Mann-Whitney U-test, P<0.05). Deep sparse autoencoders with MSR in addition to explicit sparsity regularization term and stochastic corruption of the input data with Gaussian noise have the potential to improve unsupervised abnormality detection using deep-learning compared to common deep autoencoders.

DDSA: A Defense Against Adversarial Attacks Using Deep Denoising Sparse Autoencoder

Given their outstanding performance, the Deep Neural Networks (DNNs) models have been deployed in many real-world applications. However, recent studies have demonstrated that they are vulnerable to small carefully crafted perturbations, i.e., adversarial examples, which considerably decrease their performance and can lead to devastating consequences, especially for safety-critical applications, such as autonomous vehicles, healthcare and face recognition. Therefore, it is of paramount importance to offer defense solutions that increase the robustness of DNNs against adversarial attacks. In this paper, we propose a novel defense solution based on a Deep Denoising Sparse Autoencoder (DDSA). The proposed method is performed as a pre-processing step, where the adversarial noise of the input samples is removed before feeding the classifier. The pre-processing defense block can be associated with any classifier, without any change to their architecture or training procedure. In addition, the proposed method is a universal defense, since it does not require any knowledge about the attack, making it usable against any type of attack. The experimental results on MNIST and CIFAR-10 datasets have shown that the proposed DDSA defense provides a high robustness against a set of prominent attacks under white-, gray- and black-box settings, and outperforms state-of-the-art defense methods.

Application of stack marginalised sparse denoising auto‐encoder in fault diagnosis of rolling bearing

When a fracturing vehicle is working, it generally needs to bear high loads, media corrosion and erosion. For this special working environment, this study proposes a rolling bearing fault diagnosis method based on stack marginalised sparse denoising auto-encoder (SDAE). This method combines the sparse auto-encoder (SAE) and the denoising auto-encoder (DAE) and combines the characteristics of dimensionality reduction and robustness. The method adds marginalisation to optimise the SDAE. Finally, it uses a two-layer stacking method. The output results of the second marginalised SDAE are used as input to the softmax classifier for learning training and classification testing. This improved method (stack SDAE) improves the denoising ability, reduces the computational complexity, solves the problems of difficult parameter adjustment and slows training convergence. The experimental tests were carried out on the failure of pitting corrosion of the outer ring of the bearing, pitting failure of the inner ring, and cracking of the rolling element. The results show that the algorithm can effectively improve the accuracy of fault diagnosis of rolling bearings, and it has greatly improved than the algorithms of SAEs and DAE.

Denoising Sparse Autoencoder-Based Ictal EEG Classification.

Automatic seizure detection technology can automatically mark the EEG by using the epileptic detection algorithm, which is helpful to the diagnosis and treatment of epileptic diseases. This paper presents an EEG classification framework based on the denoising sparse autoencoder. The denoising sparse autoencoder (DSAE) is an improved unsupervised deep neural network over sparse autoencoder and denoising autoencoder, which can learn the closest representation of the data. The sparsity constraint applied in the hidden layer of the network makes the expression of data as sparse as possible so as to obtain a more efficient representation of EEG signals. In addition, corrupting operation used in input data help to enhance the robustness of the system and make it suitable for the analysis of non-stationary epileptic EEG signals. In this paper, we first imported the pre-processed training data to the DSAE network and trained the network. A logistic regression classifier was connected to the top of the DSAE. Then, put the test data into the system for classification. Finally, the output results of the overall network were post-processed to obtain the final epilepsy detection results. In the two-class (nonseizure and seizure EEGs) problem, the system has achieved effective results with the average sensitivity of 100%, specificity of 100%, and recognition of 100%, showing that the proposed framework can be efficient for the classification of epileptic EEGs.

Deep Marginalized Sparse Denoising Auto-Encoder for Image Denoising

Stacked Sparse Denoising Auto-Encoder (SSDA) has been successfully applied to image denoising. As a deep network, the SSDA network with powerful data feature learning ability is superior to the traditional image denoising algorithms. However, the algorithm has high computational complexity and slow convergence rate in the training. To address this limitation, we present a method of image denoising based on Deep Marginalized Sparse Denoising Auto-Encoder (DMSDA). The loss function of Sparse Denoising Auto-Encoder is marginalized so that it satisfies both sparseness and marginality. The experimental results show that the proposed algorithm can not only outperform SSDA in the convergence speed and training time, but also has better denoising performance than the current excellent denoising algorithms, including both the subjective and objective evaluation of image denoising.

基于改进栈式稀疏去噪自编码器的自适应图像去噪

基于改进栈式稀疏去噪自编码器的自适应图像去噪

Infrared image enhancement with learned features

Due to the variation of imaging environment and limitations of infrared imaging sensors, infrared images usually have some drawbacks: low contrast, few details and indistinct edges. Hence, to promote the applications of infrared imaging technology, it is essential to improve the qualities of infrared images. To enhance image details and edges adaptively, we propose an infrared image enhancement method under the proposed image enhancement scheme. On the one hand, on the assumption of high-quality image taking more evident structure singularities than low-quality images, we propose an image enhancement scheme that depends on the extractions of structure features. On the other hand, different from the current image enhancement algorithms based on deep learning networks that try to train and build the end-to-end mappings on improving image quality, we analyze the significance of first layer in Stacked Sparse Denoising Auto-encoder and propose a novel feature extraction for the proposed image enhancement scheme. Experiment results prove that the novel feature extraction is free from some artifacts on the edges such as blocking artifacts, “gradient reversal”, and pseudo contours. Compared with other enhancement methods, the proposed method achieves the best performance in infrared image enhancement.

Deep belief network-based drug identification using near infrared spectroscopy

Near infrared spectroscopy (NIRS) analysis technology, combined with chemometrics, can be effectively used in quick and nondestructive analysis of quality and category. In this paper, an effective drug identification method by using deep belief network (DBN) with dropout mechanism (dropout-DBN) to model NIRS is introduced, in which dropout is employed to overcome the overfitting problem coming from the small sample. This paper tests proposed method under datasets of different sizes with the example of near infrared diffuse reflectance spectroscopy of erythromycin ethylsuccinate drugs and other drugs, aluminum and nonaluminum packaged. Meanwhile, it gives experiments to compare the proposed method’s performance with back propagation (BP) neural network, support vector machines (SVMs) and sparse denoising auto-encoder (SDAE). The results show that for both binary classification and multi-classification, dropout mechanism can improve the classification accuracy, and dropout-DBN can achieve best classification accuracy in almost all cases. SDAE is similar to dropout-DBN in the aspects of classification accuracy and algorithm stability, which are higher than that of BP neural network and SVM methods. In terms of training time, dropout-DBN model is superior to SDAE model, but inferior to BP neural network and SVM methods. Therefore, dropout-DBN can be used as a modeling tool with effective binary and multi-class classification performance on a spectrum sample set of small size.

Image Classification Based on Convolutional Denoising Sparse Autoencoder

Image classification aims to group images into corresponding semantic categories. Due to the difficulties of interclass similarity and intraclass variability, it is a challenging issue in computer vision. In this paper, an unsupervised feature learning approach called convolutional denoising sparse autoencoder (CDSAE) is proposed based on the theory of visual attention mechanism and deep learning methods. Firstly, saliency detection method is utilized to get training samples for unsupervised feature learning. Next, these samples are sent to the denoising sparse autoencoder (DSAE), followed by convolutional layer and local contrast normalization layer. Generally, prior in a specific task is helpful for the task solution. Therefore, a new pooling strategy—spatial pyramid pooling (SPP) fused with center-bias prior—is introduced into our approach. Experimental results on the common two image datasets (STL-10 and CIFAR-10) demonstrate that our approach is effective in image classification. They also demonstrate that none of these three components: local contrast normalization, SPP fused with center-prior, and l2 vector normalization can be excluded from our proposed approach. They jointly improve image representation and classification performance.

Nonlinear feature transformation and deep fusion for Alzheimer's Disease staging analysis

In this study, we develop a novel nonlinear metric learning method to improve biomarker identification for Alzheimer's Disease (AD) and Mild Cognitive Impairment (MCI). Formulated under a constrained optimization framework, the proposed method learns a smooth nonlinear feature space transformation that makes the mapped data more linearly separable for SVMs. The thin-plate spline (TPS) is chosen as the geometric model due to its remarkable versatility and representation power in generating sophisticated yet smooth deformations. In addition, a deep network based feature fusion strategy through stacked denoising sparse auto-encoder (DSAE) is adopted to integrate cross-sectional and longitudinal features estimated from MR brain images. Using the ADNI dataset, we evaluate the effectiveness of the proposed feature transformation and feature fusion strategies and demonstrate the improvements over the state-of-the-art solutions within the same category.