Sparse Autoencoder Model Research Articles

An automated intrusion detection system in IoT system using Attention based Deep Bidirectional Sparse Auto Encoder model

Nowadays, the Internet of Things (IoT) is a smart network connected to the Internet for transmitting gathered data with verified protocols. Attackers frequently use communication protocol defects as the basis for their attacks. Better protection measures are required since attacks affect the reputations of service providers. Both machine learning (ML) and deep learning (DL) methods have been developed in a number of research works to detect network intrusions. However, the system's security is limited by the rising number of new threats. Critical problems in IoT platforms, cyber-physical systems, wireless networks, and fog computing are caused by such attacks. The development of various cyber-security attacks reinforces the need for a strong intrusion detection system (IDS) in the IoT platform. The proposed study introduced a robust deep-feature learning mechanism for automatically detecting network intruders in the IoT platform. Initially, input data are gathered from the given dataset. Pre-processing helps reduce any noise in the data and improves the data quality using cleaning, outlier removal, and min-max normalization. The proposed Attention-based Deep Bidirectional Sparse Auto Encoder (AD-BiSA) model is the most important feature retrieved using the attention-based deep Bi-LSTM model. The different IoT network threats are categorized using a sparse Autoencoder approach. The chaotic Seagull Optimization (CSGO) algorithm decreases the loss and enhances the weight in the proposed DL technique. The UNSW NB15_IDS and NSL-KDD datasets achieve accuracy rates of 99.71% and 98.97%, respectively, for the proposed technique. The proposed method achieves better performance than existing approaches.

Control Moment Gyroscope Anomaly Detection Based on Sparse Autoencoder

The Control Moment Gyroscope (CMG) is an important executing part of spacecraft, and its anomaly detection has a key influence on the reliability of spacecraft. With the rapid growth of the number of spacecraft and telemetry data, data-driven spacecraft anomaly detection methods have received extensive attention, and such methods may become an important means of spacecraft anomaly detection. However, at present, the spacecraft is faced with the problem of unknown anomalies and lack of abnormal data during orbit. To solve this problem, a sparse autoencoder (SAE) based CMG anomaly detection method was proposed, which takes the telemetry signal of CMG as the object and lacks the sample of anomalous telemetry signal in training data. Firstly, data sample selection and processing were carried out to normalize the data of each telemetry value. Secondly, the SAE model was constructed, and the model was trained with the training set data. Then the sample reconstruction error of the test set was calculated to complete the anomaly detection of the CMG. Finally, it was compared with the traditional artificial threshold monitoring method. The results indicated that the CMG anomaly detection method based on SAE can effectively detect the anomaly samples of the CMG telemetry data, and has better anomaly detection performance than the traditional artificial threshold monitoring method.

Baby cry recognition based on SLGAN model data generation and deep feature fusion

Deep learning models have been applied in baby cry recognition to enhance the recognition accuracy. However, the current research still suffers from data imbalance problem, which leads to bias in model learning. Sparse Autoencoder Long Short-Term Memory based Generative Adversarial Network (SLGAN) is proposed to solve the data imbalance problem. The proposed SLGAN model generates new baby cry data to ensure the number of samples for every cry class is equal. Speech features are extracted using Mel-spectrograms and Short-Time Fourier Transform (STFT). Two deep learning models, i.e. VGG16 and VGG19 are used to extract the deep features. The deep features are then dimensionally reduced by using Principal Component Analysis (PCA). A sparse autoencoder model is used to fuse the deep features. Finally, the fused features are trained and classified using the Deep Neural Network. The experimental results show that the proposed method outperforms the existing methods.

Remain useful life forecasting for roller bearings using sparse auto-encoder

A method based on sparse auto-encoder (SAE) in deep learning (DL) for roller bearings remain useful life (RUL) prediction is presented in this paper. Firstly, the roller bearings vibration signals were calculated by different time and frequency domain factors, in which reflect the vibration signals information well. Therefore, the time and frequency domain features were regarded as the input of SAE, then the SAE model in deep learning was used to extract the features through several hidden layers and the sigmoid function was selected as the output function for calculate the prediction value. Finally, compared with other different prediction methods, such as support vector machine (SVM), back propagation (BP) neural network and random forest (RF), the performance of SAE is better than that those models by using mean absolute error (MAE) and root mean square error (RMSE) these two indicators.

Modified arithmetic optimization algorithm with Deep Learning based data analytics for depression detection

<abstract><p>Depression detection is the procedure of recognizing the individuals exhibiting depression symptoms, which is a mental illness that is characterized by hopelessness, feelings of sadness, persistence and loss of interest in day-to-day activities. Depression detection in Social Networking Sites (SNS) is a challenging task due to the huge volume of data and its complicated variations. However, it is feasible to detect the depression of the individuals by examining the user-generated content utilizing Deep Learning (DL), Machine Learning (ML) and Natural Language Processing (NLP) approaches. These techniques demonstrate optimum outcomes in early and accurate detection of depression, which in turn can support in enhancing the treatment outcomes and avoid more complications related to depression. In order to provide more insights, both ML and DL approaches possibly offer unique features. These features support the evaluation of unique patterns that are hidden in online interactions and address them to expose the mental state amongst the SNS users. In the current study, we develop the Modified Arithmetic Optimization Algorithm with Deep Learning for Depression Detection in Twitter Data (MAOADL-DDTD) technique. The presented MAOADL-DDTD technique focuses on identification and classification of the depression sentiments in Twitter data. In the presented MAOADL-DDTD technique, the noise in the tweets is pre-processed in different ways. In addition to this, the Glove word embedding technique is used to extract the features from the preprocessed data. For depression detection, the Sparse Autoencoder (SAE) model is applied. The MAOA is used for optimum hyperparameter tuning of the SAE approach so as to optimize the performance of the SAE model, which helps in accomplishing better detection performance. The MAOADL-DDTD algorithm is simulated using the benchmark database and experimentally validated. The experimental values of the MAOADL-DDTD methodology establish its promising performance over another recent state-of-the-art approaches.</p></abstract>

Visual Motion Segmentation in Crowd Videos Based on Spatial-Angular Stacked Sparse Autoencoders

Visual motion segmentation (VMS) is an important and key part of many intelligent crowd systems. It can be used to figure out the flow behavior through a crowd and to spot unusual life-threatening incidents like crowd stampedes and crashes, which pose a serious risk to public safety and have resulted in numerous fatalities over the past few decades. Trajectory clustering has become one of the most popular methods in VMS. However, complex data, such as a large number of samples and parameters, makes it difficult for trajectory clustering to work well with accurate motion segmentation results. This study introduces a spatial-angular stacked sparse autoencoder model (SA-SSAE) with <i>l2</i>-regularization and softmax, a powerful deep learning method for visual motion segmentation to cluster similar motion patterns that belong to the same cluster. The proposed model can extract meaningful high-level features using only spatial-angular features obtained from refined tracklets (a.k.a ‘trajectories’). We adopt <i>l2</i>-regularization and sparsity regularization, which can learn sparse representations of features, to guarantee the sparsity of the autoencoders. We employ the softmax layer to map the data points into accurate cluster representations. One of the best advantages of the SA-SSAE framework is it can manage VMS even when individuals move around randomly. This framework helps cluster the motion patterns effectively with higher accuracy. We put forward a new dataset with its manual ground truth, including 21 crowd videos. Experiments conducted on two crowd benchmarks demonstrate that the proposed model can more accurately group trajectories than the traditional clustering approaches used in previous studies. The proposed SA-SSAE framework achieved a 0.11 improvement in accuracy and a 0.13 improvement in the F-measure compared with the best current method using the CUHK dataset.

Adaptive slow feature analysis - sparse autoencoder based fault detection for time-varying processes

BackgroundFault detection and diagnosis technology is of great significance for practical industrial processes. Industrial process characteristics change with time due to various reasons such as changing working conditions. This will cause false alarm or missing alarm of process monitoring. MethodsIn this paper, an adaptive slow feature analysis (SFA) - sparse autoencoder (SAE) algorithm is proposed to establish an adaptive model for time-varying process monitoring. Model update index is built based on time-varying characteristics extracted using SFA model. Process monitoring index is built based on sparse characteristics extracted using SAE model. Through online adaptive update strategy, updated monitoring model is realized to adapt to the time-varying characteristics of the process. Significant findingsThe proposed algorithm has good performance on penicillin fermentation process data set and can realize the task of adaptive process monitoring.

An intelligent fault diagnosis framework based on piecewise aggregate approximation, statistical moments, and sparse autoencoder

Rotating machines (RMs) have vast applicability in almost all the industries in mechanical domain. Rolling element bearings (RBs) are the key elements to ensure that the RMs perform efficiently. RBs are highly prone to wear and tear which could have devastating consequences such as massive economic losses and accidents. In the past, many time-domain based condition-indicators such as root mean square (RMS), skewness and kurtosis, etc. have been proposed by researchers to diagnose the bearing faults and prevent RM failures. However, they are often insensitive to early stage faults, affected by outliers and possess poor degradation tracking characteristics. To overcome these shortcomings, this paper proposes a novel statistical feature extraction technique called as multiscale statistical moment (MSM) analysis, in combination with sparse autoencoder to detect the incipient faults as well as track the progression of wear. Firstly, the vibration signal are acquired from the bearings to be monitored. Secondly, the MSM features are extracted from the vibration signals. Thirdly, the MSM features corresponding to normal conditions are utilized to train the sparse autoencoder network. Fourthly, the MSM features corresponding to test conditions are supplied to the pre-trained sparse autoencoder model. The MSM technique offers the advantage that it extracts the fault properties contained in multiple time-scales of the vibration signals instead of a single time-scale only. Finally, the dissimilarity between the actual and predicted output is measured to obtain the bearing health indicator (BHI). The experimental results demonstrate that the suggested BHI detects the faults at early stages, possess better sensitivity and trends the bearing degradation more accurately as compared to the traditional techniques such as RMS, kurtosis, and BHI obtained with statistical moment features at single-scale only.

A deep learning based malicious module identification using stacked sparse autoencoder network for VLSI circuit reliability

The rate of development of technology and its associated security issues are increasing in integrated circuit industry. This provides a space for implanting dormant nature of threats, named Hardware Trojan (HT) in various process of integrated circuits (IC) design. The impact of HT’s emanates the encrypted signals, privacy disruption, performance degradation or denial of service. The threat models are unimaginable and it can be intruded at any stage which complicates the HT detection process. Therefore, a deep learning-based malicious module identification method is proposed in this work by implementing stacked autoencoder and stacked sparse autoencoder model. The simulation results shows that the proposed stacked sparse autoencoder outperforms the best in detecting the malicious modifications with an average accuracy of 97.53%, true positive rate of 93% and moreover the true negative rate achieved is 98.14% which proves the effectiveness of sparsity nature in extracting suitable features in the proposed schemes.

Emotion Recognition with Short-Period Physiological Signals Using Bimodal Sparse Autoencoders

With the advancement of human-computer interaction and artificial intelligence, emotion recognition has received significant research attention. The most commonly used technique for emotion recognition is EEG, which is directly associated with the central nervous system and contains strong emotional features. However, there are some disadvantages to using EEG signals. They require high dimensionality, diverse and complex processing procedures which make real-time computation difficult. In addition, there are problems in data acquisition and interpretation due to body movement or reduced concentration of the experimenter. In this paper, we used photoplethysmography (PPG) and electromyography (EMG) to record signals. Firstly, we segmented the emotion data into 10-pulses during preprocessing to identify emotions with short period signals. These segmented data were input to the proposed bimodal stacked sparse auto-encoder model. To enhance recognition performance, we adopted a bimodal structure to extract shared PPG and EMG representations. This approach provided more detailed arousal-valence mapping compared with the current high/low binary classification. We created a dataset of PPG and EMG signals, called the emotion dataset dividing into four classes to help understand emotion levels. We achieved high performance of 80.18% and 75.86% for arousal and valence, respectively, despite more class classification. Experimental results validated that the proposed method significantly enhanced emotion recognition performance.

SSAE‐MLP: Stacked sparse autoencoders‐based multi‐layer perceptron for main bearing temperature prediction of large‐scale wind turbines

SummaryCondition monitoring and fault diagnosis of main bearings of large‐scale wind turbines is critical for improving its reliability and reducing operating and maintenance costs, especially in the early stages. To achieve the goal, this paper proposes a novel deep learning approach named stacked sparse autoencoder multi‐layer perceptron (SSAE‐MLP) with a new framework by utilizing supervisory control and data acquisition (SCADA) data for wind turbine main bearing temperature prediction. After the SCADA parameter variables related to the temperature change of the main bearing are extracted, the input characteristic vector is constructed. Then, the multiple sparse autoencoders are stacked to learn the deep features inside the input data by applying the greedy layerwise unsupervised learning algorithm. Finally, a regression predictor is added to the top layer of the stacked sparse autoencoder model for supervised learning to fine‐tune the overall network. Comparative experiments show that the proposed approach has superior performance for wind turbine main bearing temperature prediction.

K sparse autoencoder-based accelerated reconstruction of magnetic resonance imaging

Owing to the sequential collection of phase encoded data in k-space, magnetic resonance (MR) imaging suffers from long acquisition time. One possible measure to reduce the long acquisition time is to reconstruct MR image using a subset of k-space MR data rather than the complete set. In this work, we propose to implement a K sparse autoencoder model for reconstruction of MR image from undersampled k-space data. Autoencoder models, which have shown great ability in capturing the complex features of input data, can be used to reconstruct high-quality MR image. The reconstruction process involved solving an optimization problem whose solution was expected to satisfy the data consistency and also lie in close proximity to the output space of trained K sparse autoencoder. Observing the effect of sparsity value enforced by K sparse autoencoder on the reconstructed output, we implemented the cascaded form of reconstruction, incorporating three K sparse autoencoders with three different K values. Using MR-PD and MR-T1 images, reconstruction performance of the proposed approach was compared with those of the conventional reconstruction approaches. The quantitative as well as the qualitative analysis of the reconstructed images, obtained using the proposed approach, validates the efficiency of the proposed approach.

A Hybrid Method for Remaining Useful Life Prediction of Proton Exchange Membrane Fuel Cell Stack

Proton exchange membrane fuel cell (PEMFC) is a clean and efficient alternative technology for transport applications. The degradation analysis of the PEFMC stack plays a vital role in electric vehicles. We propose a hybrid method based on a deep neural network model, which uses the Monte Carlo dropout approach called MC-DNN and a sparse autoencoder model to analyze the power degradation trend of the PEMFC stack. The sparse autoencoder can map high-dimensional data space to low-dimensional latent space and significantly reduce noise data. Under static and dynamic operating conditions, using two experimental PEMFC stack datasets the predictive performance of our proposed model is compared with some published models. The results show that the MC-DNN model is better than other models. Regarding the remaining useful life (RUL) prediction, the proposed model can obtain more accurate results under different training lengths, and the relative error between 0.19% and 1.82%. In addition, the prediction interval of the predicted RUL is derived by using the MC dropout approach.

Radiomics Analysis Combining Unsupervised Learning and Handcrafted Features: A Multiple-Disease Transfer Study

Background: To investigate the performance of incorporating features from unsupervised learning and handcrafted methods for radiomics study in clinical multiple-disease identification. Method: In this retrospective study, we collected a total of 666 patients with four different types of tumor and four heterogeneous clinical end points (pancreatic neuroendocrine tumors (pNETs) with different pathological grades, high-grade osteosarcoma (HOS) with different 5-year survival status, intrahepatic cholangiocarcinoma (ICC) with different early recurrence results and gastric cancer (GC) with different lymph node (LN) status). A total of 67 handcrafted features and 900 sparse autoencoder (SAE) features were extracted by handcrafted and SAE method respectively. Prediction models developed by handcrafted, SAE and hybrid features respectively. Pearson correlation method was performed to assess the correlation between handcrafted features and SAE features. Findings: In the four clinical tasks, the SAE model yielded a better prediction performance than the handcrafted model in identifying the pNETs, HOS and ICC datasets. In GC dataset, the handcrafted model showed a higher performance than the SAE model. The predictive capability of the hybrid model was superior to the SAE and handcrafted models in all four datasets. Correlation analysis showed that only 12.75% (pNETs), 7.11% (HOS), 14.78% (ICC), and 12.79% (GC) features had significant correlation for each disease. Interpretation: This study demonstrated SAE features had potential as radiomics features which could fused with handcrafted features to improve the performance of model in multiple-disease. Funding:This work was supported by the National Key R&D Program of China (2018YFE0114800), Natural Science Foundation of China (NSFC Grant No. 81871351, 81950410632), Zhejiang Provincial Key Projects of Technology Research (WKJ-ZJ-2033). Declaration of Interests: The authors declare no competing financial interests. Ethics Approval Statement: This study was approved by the institutional review boards of all participant centers and the requirement of informed consent was waived.

Transient stability assessment model with improved cost‐sensitive method based on the fault severity

For power system transient stability assessment (TSA), the consequences of misclassifications of unstable samples are more serious than those of stable samples. Therefore, the cost-sensitive method is introduced into TSA. A constant weight is set for all unstable training samples by the conventional cost-sensitive method. However, the fault severities of the training samples are different. Thus, the impacts of the samples with different fault severities on assessment rules are also different. Therefore, an improved cost-sensitive method based on the fault severity is proposed. Firstly, the fault severity of each sample is calculated by its fault clearing time. On this basis, different cost-sensitive coefficients are assigned to the samples with different fault severities. The weights of critical samples are larger than those of non-critical ones. Then, a stacked sparse auto-encoder model based on the improved cost-sensitive method is built through modifying the loss function. Finally, the proposed method is tested in an actual system. Compared with the traditional cost-sensitive method, the results show that the improved cost-sensitive method has better performance. Also, it has the advantages of the low false rate of unstable samples, high-assessment accuracy, and strong anti-noise ability.

Segmentation of the Common Carotid Intima-Media Complex in Ultrasound Images Using 2-D Continuous Max-Flow and Stacked Sparse Auto-encoder

The intima-media thickness (IMT) of a common carotid artery in an ultrasound image is considered an important indicator of the onset of atherosclerosis. However, it is challenging to segment the intima-media complex (IMC) directly in ultrasound images. This study proposes a fully automatic method to segment the IMC on longitudinal B-mode ultrasound images. Our method consists of two stages: (i) extraction of the region of interest with a continuous max-flow algorithm and region-of-interest reconstruction using a stacked sparse auto-encoder model, and (ii) IMC segmentation using a trained random forest classifier. The proposed method has been tested on three databases from three different imaging centres, comprising a total of 228 ultrasound images of the common carotid artery. On the three databases, our method yields mean absolute errors of 0.028 ± 0.016 mm, 0.579 ± 0.288 pixel and 0.582 ± 0.341 pixel; polyline distance (PD) measures of 0.026 ± 0.017 mm, 0.657 ± 0.275 pixel and 0.731 ± 0:282 pixel; Hausdorff distance measures of 0.249 ± 0.101 mm, 4.760 ± 1.085 pixels and 5.825 ± 2.059 pixels; and correlation coefficients of 95.19%, 93.79%, and 98.96%, respectively. These results indicate that the proposed method performs well in segmentation of the IMC and measurement of the IMT.

An unsupervised online monitoring method for tool wear using a sparse auto-encoder

Tool wear, and its online monitoring, plays an important role in increasing productivity and improving product quality. We describe an unsupervised method to monitor the wear state of milling cutter by tracking an error sequence generated by reconstructing monitoring signals from a sparse auto-encoder (SAE). The monitoring signals consist of the force and vibration signals collected during the cutting process. We establish a well-structured SAE model, which can adaptively extract the characteristics of the signal and complete the training of the model without supervision of the empirical label and investigate the reconstruction performance of the model for cutting signal. On this basis, an automatic online tool wear state identification strategy is designed to monitor the milling process. The mean reconstruction error (MRE) sequence associated with tool wear is recorded in real time by reconstructing the next signal segment from the SAE model, which is trained and updated using the current signal segment. Monitoring criteria and thresholds are recommended to automate the identification of tool wear conditions based on the filtered MRE curve. Five experiments with two different milling environments are run to confirm the feasibility of tool wear monitoring using this method, and the results show that the method can be used to monitor tool wear conditions online under different milling conditions without being supervised by any empirical labels.

Combustion stability monitoring through flame imaging and stacked sparse autoencoder based deep neural network

Combustion instability is a well-known problem in the combustion processes and closely linked to lower combustion efficiency and higher pollutant emissions. Therefore, it is important to monitor combustion stability for optimizing efficiency and maintaining furnace safety. However, it is difficult to establish a robust monitoring model with high precision through traditional data-driven methods, where prior knowledge of labeled data is required. This study proposes a novel approach for combustion stability monitoring through stacked sparse autoencoder based deep neural network. The proposed stacked sparse autoencoder is firstly utilized to extract flame representative features from the unlabeled images, and an improved loss function is used to enhance the training efficiency. The extracted features are then used to identify the classification label and stability index through clustering and statistical analysis. Classification and regression models incorporating the stacked sparse autoencoder are established for the qualitative and quantitative characterization of combustion stability. Experiments were carried out on a gas combustor to establish and evaluate the proposed models. It has been found that the classification model provides an F1-score of 0.99, whilst the R-squared of 0.98 is achieved through the regression model. Results obtained from the experiments demonstrated that the stacked sparse autoencoder model is capable of extracting flame representative features automatically without having manual interference. The results also show that the proposed model provides a higher prediction accuracy in comparison to the traditional data-driven methods and also demonstrates as a promising tool for monitoring the combustion stability accurately.

Estimation of spreading sequences in LC-DS-CDMA signals based on sparse auto-encoder

A method based on a sparse auto-encoder (SAE) network for the estimation of spreading sequences in long-code direct-sequence code-division multiple access (LC-DS-CDMA) signals is proposed. First, a network classification model based on SAE and softmax classifier is established. Next, the effectiveness of the proposed method is verified by estimating Walsh sequences and m sequences. To estimate the spreading sequences, the LC-DS-CDMA signal is divided into fragments. Then, each user’s spreading sequence is separated by the fast independent component analysis (Fast-ICA) algorithm, and the amplitude fuzziness is eliminated by the delay-and-multiply method. Finally, the spreading sequences are estimated by the SAE model. Experimental results showed that the proposed algorithm could effectively estimate the spreading sequences of LC-DS-CDMA signals. Compared to the existing matching algorithm and Fast-ICA algorithm, the estimation time required by the proposed algorithm was shorter, and its estimation performance at low signal-to-noise ratios was superior.

Unsupervised feature learning via prior information‐based similarity metric learning for face verification

Here, an efficient framework is developed to address the problem of unconstrained face verification. In particular, an unsupervised feature learning method for face image representation and a novel similarity metric model are discussed. First, the authors propose an unsupervised feature learning method with sparse auto-encoder (SAE) based on local descriptor (SAELD). A set of filter operators are learned based on SAE model from local patches, and face descriptors are extracted by applying the set of filter operators to convolve images. This can address the face discriminative representation issue of unconstrained face verification. Then pairwise SAELD descriptors are projected into the weighted subspace. Furthermore, a prior information-based similarity metric learning model is presented, in which the metric matrix is learned by enforcing a regularisation term based on the prior similar and discriminative information. This idea can improve the robustness to intra-personal variations and discrimination to inter-personal variations. Experimental results show that the proposed method has competitive performance compared with several state-of-the-art methods on challenging labelled faces in the wild data set.