Stacked Autoencoder Neural Network Research Articles

SANGRIA: Stacked Autoencoder Neural Networks with Gradient Boosting for Indoor Localization

Indoor localization is a critical task in many embedded applications, such as asset tracking, emergency response, and realtime navigation. In this article, we propose a novel fingerprintingbased framework for indoor localization called SANGRIA that uses stacked autoencoder neural networks with gradient boosted trees. Our approach is designed to overcome the device heterogeneity challenge that can create uncertainty in wireless signal measurements across embedded devices used for localization. We compare SANGRIA to several state-of-the-art frameworks and demonstrate 42.96% lower average localization error across diverse indoor locales and heterogeneous devices.

Trend Classification of InSAR Displacement Time Series Using SAE–CNN

Multi-temporal Interferometric Synthetic Aperture Radar technique (MTInSAR) has emerged as a valuable tool for measuring ground motion in a wide area. However, interpreting displacement time series and identifying dangerous signals from millions of InSAR coherent targets is challenging. In this study, we propose a method combining stacked autoencoder (SAE) and convolutional neural network (CNN) to classify InSAR time series and ease the interpretation of movements. The InSAR time series are classified into five categories, including stable, linear, accelerating, deceleration, and phase unwrapping error (PUE). The accuracy of labeled samples reaches 95.1%, reflecting the performance of the proposed method. This method was applied to the InSAR results for Kunming extracted from 171 ascending Sentinel-1 images from January 2017 to September 2022. The classification map of the InSAR time series shows that stable coherent points dominate around 79.28% of the area, with linear patterns at 10.70%, decelerating at 5.30%, accelerating at 4.72%, and PUE patterns at 3.60%. The results demonstrate that this method can distinguish different ground motion features and detect nonlinear deformation signals on a large scale without human intervention.

A comparative evaluation of Stacked Auto-Encoder neural network and Multi-Layer Extreme Learning Machine for detection and classification of faults in transmission lines using WAMS data

Smart grid is envisaged as a power grid that is extremely reliable and flexible. The electrical grid has wide-area measuring devices like Phasor measurement units (PMUs) deployed to provide real-time grid information and resolve issues effectively and speedily without compromising system availability. The development and application of machine learning approaches for power system protection and state estimation have been facilitated by the availability of measurement data. This research proposes a transmission line fault detection and classification (FD&C) system based on an auto-encoder neural network. A comparison between a Multi-Layer Extreme Learning Machine (ML-ELM) network model and a Stacked Auto-Encoder neural network (SAE) is made. Additionally, the performance of the models developed is compared to that of state-of-the-art classifier models employing feature datasets acquired by wavelet transform based feature extraction as well as other deep learning models. With substantially shorter testing time, the suggested auto-encoder models detect faults with 100% accuracy and classify faults with 99.92% and 99.79% accuracy. The computational efficiency of the ML-ELM model is demonstrated with high accuracy of classification with training time and testing time less than 50 ms. To emulate real system scenarios the models are developed with datasets with noise with signal-to-noise-ratio (SNR) ranging from 10 dB to 40 dB. The efficacy of the models is demonstrated with data from the IEEE 39 bus test system.

Classification of lemon quality using hybrid model based on Stacked AutoEncoder and convolutional neural network

Classification of lemon quality using hybrid model based on Stacked AutoEncoder and convolutional neural network

Screening Herbal Compound Candidates for Use as Anti-Inflammatory Drugs for COVID-19 Treatment Using Deep Semisupervised Learning

COVID-19 is a disease caused by the SARS-CoV-2 virus. Its symptoms include cough, fever, shortness of breath, and acute inflammation (hyperinflammation), and severe cases can lead to death. These symptoms tend to worsen if inflammation is not controlled. This research aims to build a stacked autoencoder deep neural network (SAE-DNN) model for identifying herbal compound candidates that can be used as anti-inflammatory drugs for COVID-19 treatment. The model’s performance is evaluated on the basis of different data representations. The research process involves data collection, data preprocessing, modeling, and testing the model on the herbal data to obtain herbal compound candidates. Results indicate that the developed SAE-DNN model with compound representation that combines fingerprints and dipeptide composition produces the best performance with an accuracy of 0.96722, a recall of 0.96419, area under the receiver operating characteristic of 0.99596, and an F1 score of 0.96567. A total of 33 herbal compounds are found as candidate anti-inflammatory drugs by using the SAE-DNN model.

Convolutional Neural Network based Retinal Vessel Segmentation

In human eye, the state of the blood vessel is a crucial diagnostic factor. The segmentation of blood vessel from the fundus image is difficult due to the spatial complexity, adjacency, overlapping and variability of blood vessel. The detection of ophthalmic pathologies like hypertensive disorders, diabetic retinopathy and cardiovascular diseases are remain challenging task due to the wide-ranging distribution of blood vessels. In this paper, Stacked Autoencoder and CNN (Convolutional Neural Network) technique is proposed to extract the blood vessel from the fundus image. Based on the experiments conducted using the Stacked Autoencoder and Convolutional Neural Network gives 90% & 95% accuracy for segmentation.

Classification of Alzheimer's Disease Based on Deep Learning of Brain Structural and Metabolic Data.

To improve the diagnosis and classification of Alzheimer’s disease (AD), a modeling method is proposed based on the combining magnetic resonance images (MRI) brain structural data with metabolite levels of the frontal and parietal regions. First, multi-atlas brain segmentation technology based on T1-weighted images and edited magnetic resonance spectroscopy (MRS) were used to extract data of 279 brain regions and levels of 12 metabolites from regions of interest (ROIs) in the frontal and parietal regions. The t-test combined with false discovery rate (FDR) correction was used to reduce the dimensionality in the data, and MRI structural data of 54 brain regions and levels of 4 metabolites that obviously correlated with AD were screened out. Lastly, the stacked auto-encoder neural network (SAE) was used to classify AD and healthy controls (HCs), which judged the effect of classification method by fivefold cross validation. The results indicated that the mean accuracy of the five experimental model increased from 96 to 100%, the AUC value increased from 0.97 to 1, specificity increased from 90 to 100%, and F1 value increased from 0.97 to 1. Comparing the effect of each metabolite on model performance revealed that the gamma-aminobutyric acid (GABA) + levels in the parietal region resulted in the most significant improvement in model performance, with the accuracy rate increasing from 96 to 98%, the AUC value increased from 0.97 to 0.99 and the specificity increasing from 90 to 95%. Moreover, the GABA + levels in the parietal region was significantly correlated with Mini Mental State Examination (MMSE) scores of patients with AD (r = 0.627), and the F statistics were largest (F = 25.538), which supports the hypothesis that dysfunctional GABAergic system play an important role in the pathogenesis of AD. Overall, our findings support that a comprehensive method that combines MRI structural and metabolic data of brain regions can improve model classification efficiency of AD.

Output-related fault detection in non-stationary processes using constructive correlative-SAE and demoting correlative-DNN

Fault detection in non-stationary processes is a timely research topic in industrial systems. The conventional approaches based on principal component regression (PCR) and partial least-squares (PLS) cannot be loosely used for non-stationary and non-linear processes when the statistical behaviour of the measurements does not follow a Gaussian distribution with constant mean and standard deviation values. This paper introduces a new application of deep learning (DL), specifically a combination of proposed correlative stacked auto-encoder (C-SAE) and correlative deep neural networks (C-DNN) for output-related anomaly detection without complete decomposition of process variables with respect to quality output(s). With this aim, two new constructive and demoting loss functions are proposed to relatively decompose the process measurements with respect to their relevance to quality output variable(s). The loss functions are modified with the incorporation of non-linear correlation analysis and hence integrated into SAE and DNN structures to suggest a correlative SAE and DNN. The proposed C-SAE and C-DNN are integrated into a scheme with an inverted pyramid structure that enables output-related fault detection without limiting stationarity assumptions. Moreover, the proposed framework can be freely applied to both linear and non-linear processes. The performance of the proposed DL strategy is tested and validated on a non-stationary numerical example and Tennessee Eastman Process. The comparison results with recent approaches indicate the outperformance of the proposed approach for process output-related fault detection purposes.

A Cost-Sensitive Deep Learning-Based Approach for Network Traffic Classification

Network traffic classification (NTC) plays an important role in cyber security and network performance, for example in intrusion detection and facilitating a higher quality of service. However, due to the unbalanced nature of traffic datasets, NTC can be extremely challenging and poor management can degrade classification performance. While existing NTC methods seek to re-balance data distribution through resampling strategies, such approaches are known to suffer from information loss, overfitting, and increased model complexity. To address these challenges, we propose a new cost-sensitive deep learning approach to increase the robustness of deep learning classifiers against the imbalanced class problem in NTC. First, the dataset is divided into different partitions, and a cost matrix is created for each partition by considering the data distribution. Then, the costs are applied to the cost function layer to penalize classification errors. In our approach, costs are diverse in each type of misclassification because the cost matrix is specifically generated for each partition. To determine its utility, we implement the proposed cost-sensitive learning method in two deep learning classifiers, namely: stacked autoencoder and convolution neural networks. Our experiments on the ISCX VPN-nonVPN dataset show that the proposed approach can obtain higher classification performance on low-frequency classes, in comparison to three other NTC methods.

Remaining useful life prediction of bearing based on stacked autoencoder and recurrent neural network

Remaining Useful Life (RUL) prediction play a crucial part in bearing maintenance, which directly affects the production efficiency and safety of equipment. Moreover, the accuracy of the prediction model is constrained by the feature extraction process and full life data of bearings. In this paper, the life prediction method of faulty rolling bearing with limited data is presented including degradation state model and RUL prediction model. In order to obtain health indication (HI) without human interference in the degradation state modeling stage, the bottleneck structure of Stacked Autoencoder (SAE) is utilized to fuse the four selected features into one HI using Intelligent Maintenance Systems (IMS) bearing dataset as training sample. In RUL prediction model, the Long Short-Term Memory (LSTM) neural network is carried out to establish the model with Standard deviation (Std) input and HI training label. In order to solve the problem of large training error caused by insufficient data in the failure stage of bearing acceleration test, the third-order spline curve interpolation is utilized to enhance the data points. Through parameter analysis, the RMSE and MAE of the test set on the prediction model are 0.032582 and 0.024038, respectively. Furthermore, the effectiveness of the proposed method is further validated by dataset from Case Western Reserve University (CWRU) with different bearing fault degrees. The analysis indicates that the RUL prediction of bearing fault data is consistent with the size of artificial added faults, that is,the more severe the fault the shorter the time of remaining life. The results validate that the proposed method can effectively extract the bearing health state by incorporating feature fusion and establish accurately prediction model for bearing remaining life.

DFTSA-Net: Deep Feature Transfer-Based Stacked Autoencoder Network for DME Diagnosis

Diabetic macular edema (DME) is the most common cause of irreversible vision loss in diabetes patients. Early diagnosis of DME is necessary for effective treatment of the disease. Visual detection of DME in retinal screening images by ophthalmologists is a time-consuming process. Recently, many computer-aided diagnosis systems have been developed to assist doctors by detecting DME automatically. In this paper, a new deep feature transfer-based stacked autoencoder neural network system is proposed for the automatic diagnosis of DME in fundus images. The proposed system integrates the power of pretrained convolutional neural networks as automatic feature extractors with the power of stacked autoencoders in feature selection and classification. Moreover, the system enables extracting a large set of features from a small input dataset using four standard pretrained deep networks: ResNet-50, SqueezeNet, Inception-v3, and GoogLeNet. The most informative features are then selected by a stacked autoencoder neural network. The stacked network is trained in a semi-supervised manner and is used for the classification of DME. It is found that the introduced system achieves a maximum classification accuracy of 96.8%, sensitivity of 97.5%, and specificity of 95.5%. The proposed system shows a superior performance over the original pretrained network classifiers and state-of-the-art findings.

A combination method of stacked autoencoder and 3D deep residual network for hyperspectral image classification

In comparison with conventional machine learning algorithms, deep learning can effectively express the deep features of remote sensing images. Considering the rich spectral and spatial information contained in hyperspectral images (HSIs), a combination method was proposed for HSI classification based on stacked autoencoder (SAE) and 3D deep residual network (3DDRN). Specifically, a SAE neural network was first built to reduce the dimensions of original HSIs. A 3D convolutional neural network (3DCNN) was then designed and the residual network module was introduced to build a 3DDRN. The dimension-reduced 3D HSI cubes were input into the 3DDRN to extract identifiable joint spectral-spatial features. Finally, the deep features continuously identified by the 3DDRN were input to Softmax classification layer to realize the classification. In addition, Batch Normalization (BN) and Dropout were used during the learning process to avoid overfitting on training data. The training and test sets of Indian Pines (IP), Pavia University (PU) and Salinas (SA) hyperspectral data sets were selected as the modeling and verification data sources. Six classical classification algorithms were adopted for comparing our proposed method, specifically including conventional machine learning algorithms of Radial Basis Function-Support Vector Machine (RBF-SVM), Kernel Simultaneous Orthogonal Matching Pursuit (KSOMP) and Local Binary Pattern-K-Nearest Neighbor (LBP-KNN), and mainstream deep learning algorithms of Variational Autoencoder (VAE), Convolutional Neural Network (CNN) and Spectral-Spatial Residual Network (SSRN). The results showed that the overall accuracy (OA) reached 98.97%, 99.69% and 99.24%, respectively, only based on 10%, 5% and 1% of training samples for IP, PU and SA. Consequently, the proposed method shows a better classification performance, even in the case of limited samples.

A deep neural network approach to QRS detection using autoencoders

Objective:In this paper, a stacked autoencoder deep neural network is proposed to extract the QRS complex from raw ECG signals without any conventional feature extraction phase. Methods:A simple architecture has been deeply trained on many datasets to ensure the generalization of the network at inference. Results:The proposed method achieved a QRS detection accuracy of 99.6% using more than 1042000 beats which is competitive with all state-of-the-art QRS detectors. Moreover, the proposed method produced only 0.82% of Detection Error Rate using six unseen datasets containing more than 1470000 beats. Thus confirms the high performance of our method to detect QRSs. Conclusion:Stacked autoencoder neural networks are very effective in QRS detection. At inference, our algorithm processes 1042309 beats in less than 25.32 s. Thus, it is favorably comparable with state-of-the-art deep learning methods. Significance:The stacked autoencoder is an efficient tool for QRS detection, which could replace conventional systems to help practitioners make fast and accurate decisions.

A novel approach to estimate blood pressure of blood loss continuously based on stacked auto-encoder neural networks

ObjectiveA novel blood pressure of blood loss (BPBL) estimation method with multi-parameter fusion based on stacked auto-encoder neural networks (SAE) is proposed in this work that aims to realize non-invasive continuous monitoring of BPBL. MethodsOur approach combined PTT, R-peak to R-peak interval (RRI), peak-foot values (PFV) and peak values (PV), extracted from electrocardiogram (ECG) and photoplethysmo- gram (PPG) for the estimation of BPBL. We used these parameters to establish a PPG-PTT and an RRI-PTT model, then employed the SAE method to get the calculation model between systolic blood pressure (SBP) diastolicblood pressure (DBP) and the characteristic parameters. ResultsThe animal experimental results based on five pigs demonstrated that the RRI-PTT model estimated the BPBL more accuratly and less error compared with the PPG-PTT model (the correlation between estimated SBP & DBP and actual SBP & DBP were 0.9954 and 0.9963, and the root mean square error for SBP & DBP were 2.56 and 2.57 mmHg). ConclusionThe PFV, PV, and RRI extracted in this work were correlated to BPBL, which can enhance the accuracy of BPBL estimation. In addition, the experimental results showed that the SAE method played a pivotal role in the non-invasive estimation of BPBL. SignificanceThe estimation method proposed in this study can innovate and expand the research work of non-invasive BP and BPBL, and provide a feasible practice for the non-invasive prediction of BPBL in the future.

Prediction of drug-induced cell viability by SAE-XGBoost algorithm based on LINCS-L1000 perturbation signal

Different cell lines have different perturbation signals in response to specific compounds, and it is important to predict cell viability based on these perturbation signals and to uncover the drug sensitivity hidden underneath the phenotype. We developed an SAE-XGBoost cell viability prediction algorithm based on the LINCS-L1000 perturbation signal. By matching and screening three major dataset, LINCS-L1000, CTRP and Achilles, a stacked autoencoder deep neural network was used to extract the gene information. These information were combined with the RW-XGBoost algorithm to predict the cell viability under drug induction, and then to complete drug sensitivity inference on the NCI60 and CCLE datasets. The model achieved good results compared to other methods with a Pearson correlation coefficient of 0.85. It was further validated on an independent dataset, corresponding to a Pearson correlation coefficient of 0.68. The results indicate that the proposed method can help discover novel and effective anti-cancer drugs for precision medicine.

Breath analysis based early gastric cancer classification from deep stacked sparse autoencoder neural network

Deep learning is an emerging tool, which is regularly used for disease diagnosis in the medical field. A new research direction has been developed for the detection of early-stage gastric cancer. The computer-aided diagnosis (CAD) systems reduce the mortality rate due to their effectiveness. In this study, we proposed a new method for feature extraction using a stacked sparse autoencoder to extract the discriminative features from the unlabeled data of breath samples. A Softmax classifier was then integrated to the proposed method of feature extraction, to classify gastric cancer from the breath samples. Precisely, we identified fifty peaks in each spectrum to distinguish the EGC, AGC, and healthy persons. This CAD system reduces the distance between the input and output by learning the features and preserve the structure of the input data set of breath samples. The features were extracted from the unlabeled data of the breath samples. After the completion of unsupervised training, autoencoders with Softmax classifier were cascaded to develop a deep stacked sparse autoencoder neural network. In last, fine-tuning of the developed neural network was carried out with labeled training data to make the model more reliable and repeatable. The proposed deep stacked sparse autoencoder neural network architecture exhibits excellent results, with an overall accuracy of 98.7% for advanced gastric cancer classification and 97.3% for early gastric cancer detection using breath analysis. Moreover, the developed model produces an excellent result for recall, precision, and f score value, making it suitable for clinical application.

Automatic Personality Traits Perception Using Asymmetric Auto-Encoder

On account of an increase in the human-computer interface applications, the study of automatic personality perception has become more and more prevalent than speech signal processing in recent years. These studies have shown that personality traits derived from psychology theories mainly affect acoustic features. However, some obstacles remain in the automatic personality perception classification, and the most important one is to extract the features related to each personality trait. Previous studies have shown that the personality effect differs from one acoustic feature to the others. Additionally, there are many features one can extract from speech signals. Curse of dimensionality in features also makes the classification difficult. This paper aimed to introduce and examine a novel and efficient automatic feature extraction method to classify the well-known big five personality traits. In this regard, three data augmentation methods for increasing data samples were examined. Afterwards, 6,373 statistical features were extracted from the nonverbal features of the SSPNet Speaker Personality Corpus. Finally, an innovative stacked asymmetric auto-encoder was utilized to extract useful features automatically to improve classification results. Compared with the conventional stacked auto-encoder and convolutional neural network, the proposed method exhibited an average improvement of 12.40%(10.14%) and 14.36%(1.42%) in terms of the unweighted average recall (accuracy), respectively. In comparison with other published works, classification results also revealed a notable average enhancement (11.78%) for unweighted average recall for all five traits and an average improvement of 5.1% for accuracy in two out of five personality traits.

Hybrid models based on genetic algorithm and deep learning algorithms for nutritional Anemia disease classification

Deep learning algorithms are an important part of disease prediction and diagnosis by analyzing health data. If not diagnosed and treated early, symptoms of nutritional anemia can be seen as a common laboratory finding of dyspnea, dizziness, lack of concentration, pale skin color, and life-threatening diseases. In the literature, several data mining techniques have been used for the prediction of nutritional anemia, especially, for the iron deficiency anemia. However, each algorithm does not perform well for every data, and therefore new techniques need to be developed. It is because the characteristics of each dataset are different and their dataset sizes, that is, the number of records and the number of parameters are different. In this study, we propose two hybrid models using genetic algorithm (GA) and deep learning algorithms of Stacked Autoencoder (SAE) and Convolutional Neural Network (CNN) for the prediction of HGB-anemia, nutritional anemia, (iron deficiency anemia, B12 deficiency anemia, and folate deficiency anemia), and patients without anemia. In the proposed GA-SAE and GA-CNN models, the hyperparameters of SAE and CNN algorithms are optimized using GA since it is not easy to determine suitable values of deep learning algorithms. Accuracy, F-score, precision, and sensitivity criteria were used to evaluate the prediction and classification performances of the proposed algorithms. As a result of the experimental evaluations using the dataset, the performance of the proposed GA-CNN algorithm whose layers trained separately and sequentially was found to be better than the performance of the studies proposed in the literature, by a 98.50% accuracy.

Detection of Citrus Huanglongbing Based on Multi-Input Neural Network Model of UAV Hyperspectral Remote Sensing

Citrus is an important cash crop in the world, and huanglongbing (HLB) is a destructive disease in the citrus industry. To efficiently detect the degree of HLB stress on large-scale orchard citrus trees, an UAV (Uncrewed Aerial Vehicle) hyperspectral remote sensing tool is used for HLB rapid detection. A Cubert S185 (Airborne Hyperspectral camera) was mounted on the UAV of DJI Matrice 600 Pro to capture the hyperspectral remote sensing images; and a ASD Handheld2 (spectrometer) was used to verify the effectiveness of the remote sensing data. Correlation-proven UAV hyperspectral remote sensing data were used, and canopy spectral samples based on single pixels were extracted for processing and analysis. The feature bands extracted by the genetic algorithm (GA) of the improved selection operator were 468 nm, 504 nm, 512 nm, 516 nm, 528 nm, 536 nm, 632 nm, 680 nm, 688 nm, and 852 nm for the HLB detection. The proposed HLB detection methods (based on the multi-feature fusion of vegetation index) and canopy spectral feature parameters constructed (based on the feature band in stacked autoencoder (SAE) neural network) have a classification accuracy of 99.33% and a loss of 0.0783 for the training set, and a classification accuracy of 99.72% and a loss of 0.0585 for the validation set. This performance is higher than that based on the full-band AutoEncoder neural network. The field-testing results show that the model could effectively detect the HLB plants and output the distribution of the disease in the canopy, thus judging the plant disease level in a large area efficiently.

Hyperspectral image classification using multi-task feature leverage with multi-variant deep learning

Deep learning framework aids the researchers in learning different application areas to a greater extent. Deep learning framework is preferred over machine learning since it helps to learn the input from end to end, whereas latter one require the inputs to be cut into pieces according to the need. This paper proposes a Multi-Variant Deep Learning framework for learning and classifying the hyperspectral images. Multi-task Feature Leverage is incorporated by doing two-ordered feature extraction. The first order feature extraction was done by using Two Dimensional Empirical Wavelet Transforms (2D-EWT) and the second-order feature extraction was done by using Stacked Autoencoder (SAE) and Convolutional Neural Network (CNN) for the approximation image of 2D-EWT. Because of the possibility of working on prominent feature, the proposed work uses approximation image than the raw image. The classification was carried out by using Random Forest (RF), Multi- Support Vector Machine (MSVM) and Extreme learning machine (ELM).