Stacked Sparse Autoencoder Research Articles

Diagnosis of Interturn Short-Circuit Faults in Permanent Magnet Synchronous Motors Based on Few-Shot Learning Under a Federated Learning Framework

A large amount of labeled data are important to enhance the performance of deep-learning-based methods in the area of fault diagnosis. Because it is difficult to obtain high-quality samples in real industrial applications, federated learning is an effective framework for solving the problem of sparse samples by using the distributed data. Its global model is updated by the local client without sharing data at each round. Considering computing resources and communication loss of multiple clients, an efficient method based on stacked sparse autoencoders (SSAEs) and Siamese networks is proposed to detect interturn short-circuit (ITSC) faults in permanent magnet synchronous motors. In this article, to achieve an accurate ITSC fault detection, an SSAE was employed to extract sparse features in a limited number of samples, and Siamese networks were used to determine the similarity between the given samples. The problem of fault diagnosis is transformed into a classification problem under few-shot learning. Furthermore, the proposed method is trained successfully in the frameworks of centralized learning and decentralized structure. The experimental results indicate that the proposed method achieved high fault diagnosis accuracy. Moreover, it is suitable for deployment in smart manufacturing systems.

Emotion Recognition and Regulation Based on Stacked Sparse Auto-Encoder Network and Personalized Reconfigurable Music

Music can regulate and improve the emotions of the brain. Traditional emotional regulation approaches often adopt complete music. As is well-known, complete music may vary in pitch, volume, and other ups and downs. An individual’s emotions may also adopt multiple states, and music preference varies from person to person. Therefore, traditional music regulation methods have problems, such as long duration, variable emotional states, and poor adaptability. In view of these problems, we use different music processing methods and stacked sparse auto-encoder neural networks to identify and regulate the emotional state of the brain in this paper. We construct a multi-channel EEG sensor network, divide brainwave signals and the corresponding music separately, and build a personalized reconfigurable music-EEG library. The 17 features in the EEG signal are extracted as joint features, and the stacked sparse auto-encoder neural network is used to classify the emotions, in order to establish a music emotion evaluation index. According to the goal of emotional regulation, music fragments are selected from the personalized reconfigurable music-EEG library, then reconstructed and combined for emotional adjustment. The results show that, compared with complete music, the reconfigurable combined music was less time-consuming for emotional regulation (76.29% less), and the number of irrelevant emotional states was reduced by 69.92%. In terms of adaptability to different participants, the reconfigurable music improved the recognition rate of emotional states by 31.32%.

Multi-scale deep learning and optimal combination ensemble approach for AQI forecasting using big data with meteorological conditions

Faced with the rapid update of nonlinear and irregular big data from the environmental monitoring system, both the public and managers urgently need reliable methods to predict possible air pollutions in the future. Therefore, a multi-scale deep learning (MDL) and optimal combination ensemble (OCE) approach for hourly air quality index (AQI) forecasting is proposed in this paper, named MDL-OCE model. Before normal modeling, all original data are preprocessed through missing data filling and outlier testing to ensure smooth computation. Due to the complexity of such big data, slope-based ensemble empirical mode decomposition (EEMD) is adopted to decompose the time series of AQI and meteorological conditions into a finite number of simple intrinsic mode function (IMF) components and one residue component. Then, to unify the number of components of different variables, the fine-to-coarse (FC) technique is used to reconstruct all components into high frequency component (HF), low frequency component (LF), and trend component (TC). For purpose of extracting the underlying relationship between AQI and meteorological conditions, the three components are respectively trained and predicted by different deep learning architectures (stacked sparse autoencoder (SSAE)) with a multilayer perceptron (MLP). The corresponding forecasting results of three components are merged by OCE method to better achieve the ultimate AQI forecasting outputs. The empirical results clearly demonstrate that our proposed MDL-OCE model outperforms other advanced benchmark models in terms of forecasting performances in all cases.

Breath analysis based early gastric cancer classification from deep stacked sparse autoencoder neural network

Deep learning is an emerging tool, which is regularly used for disease diagnosis in the medical field. A new research direction has been developed for the detection of early-stage gastric cancer. The computer-aided diagnosis (CAD) systems reduce the mortality rate due to their effectiveness. In this study, we proposed a new method for feature extraction using a stacked sparse autoencoder to extract the discriminative features from the unlabeled data of breath samples. A Softmax classifier was then integrated to the proposed method of feature extraction, to classify gastric cancer from the breath samples. Precisely, we identified fifty peaks in each spectrum to distinguish the EGC, AGC, and healthy persons. This CAD system reduces the distance between the input and output by learning the features and preserve the structure of the input data set of breath samples. The features were extracted from the unlabeled data of the breath samples. After the completion of unsupervised training, autoencoders with Softmax classifier were cascaded to develop a deep stacked sparse autoencoder neural network. In last, fine-tuning of the developed neural network was carried out with labeled training data to make the model more reliable and repeatable. The proposed deep stacked sparse autoencoder neural network architecture exhibits excellent results, with an overall accuracy of 98.7% for advanced gastric cancer classification and 97.3% for early gastric cancer detection using breath analysis. Moreover, the developed model produces an excellent result for recall, precision, and f score value, making it suitable for clinical application.

Gradient based invasive weed optimization algorithm for the training of deep neural network

Stacked Sparse Auto-Encoder (SSAE) is well known hierarchical deep neural networks for simulating the deep architecture of mammal brain. SSAE can be trained in a greedy layer-wise manner by using methods based on gradient such as Limited memory BFGS (LBFGS). However, methods based on gradient have many disadvantages. The main disadvantage is that they are sensitive to the initial value. In this paper, a meta-heuristic algorithm based on gradient, referred to GCIWOSS, is used to optimize the weights and biases of SSAE. Chaos strategy is firstly used to initial the population of IWO and then a new selection strategy is adopted with the purpose of improving the diversity of population and increasing the global exploration ability. The improved IWO is preparing for the following exploitation based on gradient to avoid falling into local optimal values. In the experiments, the proposed algorithm is proven to be effective in extracting features from different image datasets, compared with the LBFGS and several other feature learning models.

A high-precision feature extraction network of fatigue speech from air traffic controller radiotelephony based on improved deep learning

Air traffic controller (ATC) fatigue is receiving considerable attention in recent studies because it represents a major cause of air traffic incidences. Research has revealed that the presence of fatigue can be detected by analysing speech utterances. However, constructing a complete labelled fatigue data set is very time-consuming. Moreover, a manually constructed speech collection will often contain only little key information to be used effectively in fatigue recognition, while multilevel deep models based on such speech materials often have overfitting problems due to an explosive increase of model parameters. To address these problems, a novel deep learning framework is proposed in this study to integrate active learning (AL) into complex speech features selected from a large set of unlabelled speech data in order to overcome the loss of information. A shallow feature set is first extracted using stacked sparse autoencoder networks, in which fatigue state challenge features from a manually selected speaker set of are exploited as the input vector. A densely connected convolutional autoencoder (DCAE) is then proposed to learn advanced features automatically from spectrograms of the selected data to supplement the fatigue features. The network can be effectively trained using a relatively small number of labelled samples with the help of AL sampling strategies, and the addition of a dense block to the convolutional automatic encoder can decrease the number of parameters and make the model easier to fit. Finally, the two above-mentioned features are combined using multiple kernel learning with a support-vector-machine classifier. A series of comparative experiments using the Civil Aviation Administration of China radiotelephony corpus demonstrates that the proposed method provides a significant improvement in the detection precision compared to current state-of-the-art approaches.

Fault Detection based on Deep Learning for Digital VLSI Circuits

As growing complexity of digital VLSI circuits, fault detection and correction processes have been the most crucial phases during IC design. Many CAD tools and formal approaches have been used for debugging and localizing different kinds of design bugs. However, the search space explosion problem remains the main problem for IC designers. Recently, Artificial intelligence and machine learning models have been expanded in feature extraction and reduction models. In this paper, we introduce a new fault detection model based on deep learning for extracting features and detecting faults from large-sized digital circuits. The main goal of the proposed model is to avoid the search space using stacked sparse autoencoder, a specific type of artificial neural network. The model consists of three phases: test pattern generation using ATALANTA software, feature reduction using SSAE and classification for fault detection. Test vectors are utilized in SSAE as a training data for unsupervised learning phase. The performance of feature extraction is tested by changing the architecture of SSAE network and sparsity constraint. The proposed algorithm has been implemented using eight combinational digital circuits from ISCAS’85. From experimental results, the maximum fault coverage using ATALANTA tool delivers around 99.2% using ISCAS’85. In addition, the maximum validation accuracy of proposed SSAE model delivers around 99.7% in feature reduction phase.

Integrating Stacked Sparse Auto-Encoder Into Matrix Factorization for Rating Prediction

Currently, collaborative filtering technology has been widely used in personalized recommender systems. The problem of data sparsity is a severe challenge faced by traditional collaborative filtering methods based on matrix factorization techniques. A lot of improved collaborative filtering methods have been proposed to alleviate the data sparsity problem; However, due to the sparsity of the user rating matrix, the latent factor learned by these improved methods may be not efficient. In this paper, we propose a novel recommendation algorithm named SSAERec by integrating stacked sparse auto-encoder into matrix factorization for rating prediction, which can learn effective representation from user-item rating matrix. Extensive experiments on three real-world datasets demonstrate the proposed method outperforms other baselines in the rating prediction task.

Deep network approach with stacked sparse autoencoders in detection of DDoS attacks on SDN‐based VANET

Software-defined network (SDN)-based vehicular ad hoc network (VANET) is an outstanding technology for smart transportation as it increases traffic safety, efficiency, comfort, and manageability. However, despite all its benefits and good performance, SDN-based VANET is vulnerable to attack threats such as distributed denial of service (DDoS). When SDN-based VANET systems are exposed to DDoS attacks, this may affect traffic safety, causing traffic accidents and deaths. Therefore, the relevant security threats need to be addressed before integrating the SDN-based VANETs into smart transportation systems. In this study, the stacked sparse autoencoder (SSAE) + Softmax classifier deep network model is proposed to detect DDoS attacks targeting SDN-based VANETs. The features in the dataset obtained from the SDN-based VANET were reduced dimensionally utilising SSAE, and the most significant features were obtained. Then, these features were used as input into the Softmax classifier. According to the experimental results, the best accuracy scores were calculated as 96.9% using the four-layer SSAE + Softmax classifier deep network model proposed. When compared, the results demonstrate the SSAE + Softmax classifier deep network model proposed can obtain better results in the classification of DDoS attacks and is more successful than the other machine learning classifiers.

Psychological stimulation for anxious states detection based on EEG-related features

Anxiety affects human capabilities and behavior as much as it affects productivity and quality of life. It is considered to be the main cause of depression and suicide. Anxious states are detectable by specialists by virtue of their acquired cognition and skills. There is a need for non-invasive reliable techniques that performs the complex task of anxiety detection. In our study, we investigate the impact of different parameters, notably: trial duration, feature type, feature combination and anxiety levels number. The system is evaluated using our own database containing recorded Electroencephalogram (EEG) signals from 23 participants during anxiety elicitation by means of face-to-face psychological stimuli. EEG signals were captured with an Emotiv Epoc headset as a cost-efficient wireless wearable equipment. Two labeling methods are used and results are presented accordingly. Our findings showed that anxiety is well elicited in 1 s. For Self Assessment Manikan SAM-based detection, Stacked Sparse Autoencoder (SSAE) with different type of features achieves 83.50% and 74.60% for 2 and 4 anxiety levels detection, respectively. Results are improved using the Hamilton-based method. We obtained a rate of 86.7% for 4 levels detection using SSAE. The presented results prove the benefits of the use of a low-cost EEG headset instead of medical non-wireless devices and create a starting point for new researches in the field of anxiety detection.

Gearbox Fault Diagnosis Using a Deep Learning Model With Limited Data Sample

Massive volumes of data are needed for deep learning (DL) models to provide accurate diagnosis results. Numerous studies of fault diagnosis systems have demonstrated the effectiveness of DL models over shallow machine learning (SL) in terms of feature extraction, feature dimensional reduction and diagnosis performance. Occasionally, during data acquisition, a problem with a sensor renders some of the data potentially unsuitable for further analysis, leaving only a small data sample. To compensate for this deficiency, a DL model based on a stacked sparse autoencoder (SSAE) model is designed to deal with limited sample data. In this article, the fault diagnosis system is developed based on time-frequency image pattern recognition. Therefore, two gearbox datasets are used to evaluate the proposed diagnosis system. The results from the experiments prove that the proposed system is capable of achieving high diagnostic accuracy even with limited sample data. The proposed fault diagnosis system achieved 100% and 99% diagnosis performance on experimental gearbox and wind turbine gearbox datasets, respectively. The proposed diagnosis system increased diagnosis performance between 10% and 20% over the standard SSAE model. In addition, the proposed model achieved higher diagnosis performance compared to deep neural network and convolutional neural networks models.

A Sleep Apnea Detection Method Based on Unsupervised Feature Learning and Single-Lead Electrocardiogram

Sleep apnea (SA) is a harmful respiratory disorder that has caused widespread concern around the world. Considering that electrocardiogram (ECG)-based SA diagnostic methods were effective and human-friendly, many machine learning or deep learning methods based on ECG have been proposed by prior works. However, these methods are based on feature engineering or supervised and semisupervised learning techniques, and the feature sets are always incomplete, subjective, and highly dependent on labeled data. In addition, some related studies ignored the data imbalance problem which leads to poor performance of classifier on minority classes. In this study, an SA detection model based on frequential stacked sparse auto-encoder (FSSAE) and time-dependent cost-sensitive (TDCS) classification model was proposed. The FSSAE extracts feature set automatically with unsupervised learning technique, and the TDCS classification model is proposed by combining the hidden Markov model (HMM) and the MetaCost algorithm to improve the performance of the classifier by considering temporal dependence and the imbalance problem. In the test set, the result of per-segment classification achieved 85.1%, 86.2%, and 84.4% for accuracy, sensitivity, and specificity, respectively, proving that our method is helpful for SA detection.

Translation Invariance-Based Deep Learning for Rotating Machinery Diagnosis

Discriminative feature extraction is a challenge for data-driven fault diagnosis. Although deep learning algorithms can automatically learn a good set of features without manual intervention, the lack of domain knowledge greatly limits the performance improvement, especially for nonstationary and nonlinear signals. This paper develops a multiscale information fusion-based stacked sparse autoencoder fault diagnosis method. The autoencoder takes advantage of the multiscale normalized frequency spectrum information obtained by dual-tree complex wavelet transform as input. Accordingly, the multiscale normalized features guarantee the translational invariance for signal characteristics, and the stacked sparse autoencoder benefits the unsupervised feature learning and ensures accurate and stable diagnosis performance. The developed method is performed on motor bearing vibration signals and worm gearbox vibration signals, respectively. The results confirm that the developed method can accommodate changing working conditions, be free of manual feature extraction, and perform better than the existing intelligent diagnosis methods.

Early determination of mildew status in storage maize kernels using hyperspectral imaging combined with the stacked sparse auto-encoder algorithm

The early detection the mildew status of maize kernels caused by fungal infection during storage is essential. Hyperspectral imaging (HSI) combined with the deep stacked sparse auto-encoders (SSAE) algorithm was employed to recognize the early mildewed degree of kernels. HSI data of kernels were collected in the spectral range of 400–1000 nm and the mildew status was classified into four grades (health, mild mildew, moderate mildew, and heavy mildew) using the measured fungal spores as the reference. The nonlinear features of the spectra were represented using the SSAE network, and the key wavelengths were selected by variable combination population analysis and random frog methods, which were combined with four classifiers-Softmax, kernel extreme learning machine (KELM), ELM, and support vector machine-to establish models for identifying the mildew grades of kernels. The comparative results indicated that SSAE combined with KELM classifier to build the SSAE-KELM model had better identification ability than other models with high total correct recognition rate of 97.36% and 96.84% for the training and testing sets, respectively. The sensitivity and specificity were 0.92–1. The different grades were visualized on the images of maize kernels based on pixels-level and object-level. The current results demonstrated that HSI with SSAE-KELM model performed well in the early detection of mildew status of maize kernel, and can be applied as the reference method for batch and online detection of mildew kernels to improve grain storage security.

Deep Learning for Detecting Sleep Apnea from ECG Signals

Sleep apnea (SA) is a common sleep disorders affecting the sleep quality. Therefore the automatic SA detection has far-reaching implications for patients and physicians. In this paper, a novel approach is developed based on deep neural network (DNN) for automatic diagnosis SA. To this end, five features are extracted from electrocardiogram (ECG) signals through wavelet decomposition and sample entropy. The deep neural network is constructed by two-layer stacked sparse autoencoder (SSAE) network and one softmax layer. The softmax layer is added at the top of the SSAE network for diagnosing SA. Afterwards, the SSAE network can get more effective high-level features from raw features. The experimental results reveal that the performance of deep neural network can accomplish an accuracy of 96.66%, a sensitivity of 96.25%, and a specificity of 97%. In addition, the performance of deep neural network outperforms the comparison models including support vector machine (SVM), random forest (RF), and extreme learning machine (ELM). Finally, the experimental results reveal that the proposed method can be valid applied to automatic SA event detection.

Automated gleason grading on prostate biopsy slides by statistical representations of homology profile

Background and Objective:Gleason grading system is currently the clinical gold standard for determining prostate cancer aggressiveness. Prostate cancer is typically classified into one of 5 different categories with 1 representing the most indolent disease and 5 reflecting the most aggressive disease. Grades 3 and 4 are the most common and difficult patterns to be discriminated in clinical practice. Even though the degree of gland differentiation is the strongest determinant of Gleason grade, manual grading is subjective and is hampered by substantial inter-reader disagreement, especially with regard to intermediate grade groups.Methods:To capture the topological characteristics and the degree of connectivity between nuclei around the gland, the concept of Homology Profile (HP) for prostate cancer grading is presented in this paper. HP is an algebraic tool, whereby, certain algebraic invariants are computed based on the structure of a topological space. We utilized the Statistical Representation of Homology Profile (SRHP) features to quantify the extent of glandular differentiation. The quantitative characteristics which represent the image patch are fed into a supervised classifier model for discrimination of grade patterns 3 and 4.Results:On the basis of the novel homology profile, we evaluated 43 digitized images of prostate biopsy slides annotated for regions corresponding to Grades 3 and 4. The quantitative patch-level evaluation results showed that our approach achieved an Area Under Curve (AUC) of 0.96 and an accuracy of 0.89 in terms of discriminating Grade 3 and 4 patches. Our approach was found to be superior to comparative methods including handcrafted cellular features, Stacked Sparse Autoencoder (SSAE) algorithm and end-to-end supervised learning method (DLGg). Also, slide-level quantitative and qualitative evaluation results reflect the ability of our approach in discriminating Gleason Grade 3 from 4 patterns on H&E tissue images.Conclusions:We presented a novel Statistical Representation of Homology Profile (SRHP) approach for automated Gleason grading on prostate biopsy slides. The most discriminating topological descriptions of cancerous regions for grade 3 and 4 in prostate cancer were identified. Moreover, these characteristics of homology profile are interpretable, visually meaningful and highly consistent with the rubric employed by pathologists for the task of Gleason grading.

A Novel Order Analysis and Stacked Sparse Auto-Encoder Feature Learning Method for Milling Tool Wear Condition Monitoring.

Milling is a main processing mode of the modern manufacturing industry, which seriously affects the quality and precision of the machined workpiece. However, it is difficult to monitor the tool wear condition in the continuous cutting process, especially under a variable speed condition. The existing tool wear condition monitoring methods only carry out analysis with a constant engine speed. Different from the general monitoring methods, this paper put forward a milling cutter wear condition monitoring method based on order analysis (OA) and stacked sparse autoencoder (SSAE). The methodology in the research include signals feature extraction and tool wear state monitoring and were designed to analyze the three-phase spindle current signals instead of the traditional force signals and vibration signals. The variable speed signals were transformed into angle domain stationary signals by order analysis, and the SSAE neural network was used to monitor the tool wear state. The proposed method was verified on the laboratory signals and the results showed a better performance than the other methods and a better applicability in actual industrial manufacturing.

Latent Relationship Guided Stacked Sparse Autoencoder for Hyperspectral Imagery Classification

Classification is an important application of hyperspectral image (HSI). However, it is also a challenging research topic due to the spatial variability of spectral signature and limited training samples. To address these problems, a novel unsupervised feature learning method called latent relationship guided the stacked sparse autoencoder (LRSSAE) is developed in this article, which can effectively exploit the latent relationship under feature space to improve the ability of feature learning. Moreover, the superpixels constraint is employed on the feature representation to avoid the “salt-and-pepper” problem, and it is enforced on the latent relationship to enhance the latent relationship learning additionally. In LRSSAE, combining the stacked sparse autoencoder (SSAE) with the graph regularizations of latent relationship in each hidden layer and the superpixel constraints in the top layer, we extract feature representation in an unsupervised manner. And then, we present a customized iterative algorithm to optimize the LRSSAE. We evaluate the proposed method on three widely used HSI data sets comprehensively. The results demonstrate that our method achieves promising classification performance on these data sets and obtains improvements of 5.06%, 5.77%, and 2.11% in overall accuracy compared to the best SSAE method.

Deep neural network for semi-automatic classification of term and preterm uterine recordings.

Pregnancy is a complex process, and the prediction of premature birth is uncertain. Many researchers are exploring non-invasive approaches to enhance its predictability. Currently, the ElectroHysteroGram (EHG) and Tocography (TOCO) signal are a real-time and non-invasive technology which can be employed to predict preterm birth. For this purpose, sparse autoencoder (SAE) based deep neural network (SAE-based DNN) is developed. The deep neural network has three layers including a stacked sparse autoencoder (SSAE) network with two hidden layers and one final softmax layer. To this end, the bursts of all 26 recordings of the publicly available TPEHGT DS database corresponding to uterine contraction intervals and non-contraction intervals (dummy intervals) were manually segmented. 20 features were extracted by two feature extraction algorithms including sample entropy and wavelet entropy. Afterwards, the SSAE network is adopted to learn high-level features from raw features by unsupervised learning. The softmax layer is added at the top of the SSAE network for classification. In order to verify the effectiveness of the proposed method, this study used 10-fold cross-validation and four indicators to evaluate classification performance. Experimental research results display that the performance of deep neural network can achieve Sensitivity of 98.2%, Specificity of 97.74%, and Accuracy of 97.9% in the publicly TPEHGT DS database. The performance of deep neural network outperforms the comparison models including deep belief networks (DBN) and hierarchical extreme learning machine (H-ELM). Finally, experimental research results reveal that the proposed method could be valid applied to semi-automatic identification of term and preterm uterine recordings.

A Hybrid Wi-Fi Fingerprint-Based Localization Scheme Achieved by Combining Fisher Score and Stacked Sparse Autoencoder Algorithms

Along with the advancement of wireless technology, indoor localization technology based on Wi-Fi has received considerable attention from academia and industry. The fingerprint-based method is the mainstream approach for Wi-Fi indoor localization and can be easily implemented without additional hardware. However, signal fluctuations constitute a critical issue pertaining to the extraction of robust features to achieve the required localization performance. This study presents a fingerprint feature extraction method commonly referred to as the Fisher score–stacked sparse autoencoder (Fisher–SSAE) method. Some features with low Fisher scores were eliminated, and the representative features were then extracted by the SSAE. Furthermore, this study establishes a hybrid localization model constructed with the use of the global model and the submodel to avoid significant coordinate localization errors attributed to subregional localization errors. Combined with three accessible fingerprint-based positioning methods, namely, the support vector regression, random forest regression, and the multiplayer perceptron classification, the experimental results demonstrate that the proposed methods improve the localization accuracy and response time compared to other feature extraction methods and the single localization model. Compared with some state-of-the-art methods, the proposed methods have better localization performances when large number of features are used.