Stacked Autoencoder Network Research Articles

Precise single step and multistep short-term photovoltaic parameters forecasting based on reduced deep convolutional stack autoencoder and minimum variance multikernel random vector functional network

Accurate prediction of photovoltaic (PV) power can significantly alleviate energy crises. However, the inherent randomness and intermittency of PV power pose challenges to the stability and safety of PV-penetrated grid systems. To address this, we have developed a novel hybrid model: a reduced deep convolutional stack autoencoder with a minimum variance multikernel random vector functional link network (RDCSAE-MVMRVFLN). This model enhances grid efficacy and safety. We extract the most informative band-limited intrinsic mode functions (BLIMFs) of highly nonlinear and nonstationary solar energy parameters using an entropy, kurtosis, and correlation coefficient-based information-oriented variational mode decomposition (IOVMD). These efficient BLIMFs are concatenated and input into the RDCSAE for rich, abstract, and discriminative representation computation. A less computationally complex MVMKRVFLN regression method, incorporating these refined representations, is proposed for superior prediction accuracy and reduced computational complexity. Our method shows exceptional performance in predicting solar temperature, irradiation, and power for multi-horizon forecasts with minimal error metrics (correlation coefficients of 0.999±0.001, 0.992±0.001, 0.986±0.02 and 0.978±0.02, and RMSE of 0.016±0.001, 0.024±0.001, 0.034±0.001 and 0.045±0.001 for the interval of 10 minutes, 30 minutes, 1 hour and 3 hours respectively) in both single-step and multi-step forecasting compared to conventional methods. The RDCSAE-MVMKRVFLN model is implemented on a high-speed Xilinx Virtex-5 FPGA embedded processor to validate its simplicity, robustness, and practicability. Additionally, we examine the prediction performance using real-time data from a 1 MW solar farm in Odisha, India, demonstrating the model’s effectiveness and superiority.

Multi-sensor signal fusion for tool wear condition monitoring using denoising transformer auto-encoder Resnet

Multi-sensor signal fusion is commonly used in associate with the artificial intelligence model to monitor tool wears. However, AI models equipped with limited multi-sensor training samples still exist problems: 1) The limitation of training samples may cause the AI model failure due to the intrinsic wear features contaminated by heavy noises. 2) The selection of multi-sensor signals as training samples is a difficult task for the negative impact on the recognition accuracy using inappropriate sensor features. Therefore, this paper proposes a denoise transformer Auto-Encoder (DTAE) as pre-processor for the tool condition monitoring (TCM) classifiers. The reconstruction task of the DTAE allows the model to pay more attention to the intrinsic wear features and the appropriate selection of them from the multi-sensor signals during feature extraction. Moreover, the loss function for DTAE ResNet is formed by summing the reconstruction loss from DTAE with the classification loss from ResNet. Compared DTAE ResNet to stacked sparse auto-encoder network, deep stacked auto-encoder network, ResNet-18, VGG-16, and LSTM, experiments demonstrated that the present method would attain the highest classification accuracies for tool wear suitable for TCM. And a comparative experiment was conducted to verify the effectiveness of the DTAE preprocessor in improving the anti-noise performance of the classifier.

Feature Extraction With Stacked Autoencoders for EEG Channel Reduction in Emotion Recognition.

Emotion recognition by electroencephalogram (EEG) signals is one of the complex methods because the extraction and recognition of the features hidden in the signal are sophisticated and require a significant number of EEG channels. Presenting a method for feature analysis and an algorithm for reducing the number of EEG channels fulfills the need for research in this field. Accordingly, this study investigates the possibility of utilizing deep learning to reduce the number of channels while maintaining the quality of the EEG signal. A stacked autoencoder network extracts optimal features for emotion classification in valence and arousal dimensions. Autoencoder networks can extract complex features to provide linear and non- linear features which are a good representative of the signal. The accuracy of a conventional emotion recognition classifier (support vector machine) using features extracted from SAEs was obtained at 75.7% for valence and 74.4% for arousal dimensions, respectively. Further analysis also illustrates that valence dimension detection with reduced EEG channels has a different composition of EEG channels compared to the arousal dimension. In addition, the number of channels is reduced from 32 to 12, which is an excellent development for designing a small-size EEG device by applying these optimal features.

Stressed vegetation identification under natural gas micro-leakage from hyperspectral images using stacked autoencoder and multi-scale three-dimensional convolutional neural network

Stressed vegetation identification under natural gas micro-leakage from hyperspectral images using stacked autoencoder and multi-scale three-dimensional convolutional neural network

Trend Classification of InSAR Displacement Time Series Using SAE–CNN

Multi-temporal Interferometric Synthetic Aperture Radar technique (MTInSAR) has emerged as a valuable tool for measuring ground motion in a wide area. However, interpreting displacement time series and identifying dangerous signals from millions of InSAR coherent targets is challenging. In this study, we propose a method combining stacked autoencoder (SAE) and convolutional neural network (CNN) to classify InSAR time series and ease the interpretation of movements. The InSAR time series are classified into five categories, including stable, linear, accelerating, deceleration, and phase unwrapping error (PUE). The accuracy of labeled samples reaches 95.1%, reflecting the performance of the proposed method. This method was applied to the InSAR results for Kunming extracted from 171 ascending Sentinel-1 images from January 2017 to September 2022. The classification map of the InSAR time series shows that stable coherent points dominate around 79.28% of the area, with linear patterns at 10.70%, decelerating at 5.30%, accelerating at 4.72%, and PUE patterns at 3.60%. The results demonstrate that this method can distinguish different ground motion features and detect nonlinear deformation signals on a large scale without human intervention.

SCD: A Detection System for DDoS Attacks based on SAE-CNN Networks

The pervasive application of network technology has given rise to a numerous of network attacks, including Distributed Denial of Service (DDoS) attacks. DDoS attacks can lead to the collapse of network resources, making the target server unable to support legitimate users, which is a critical issue in cyberspace security. In complex real-world network environments, differentiating DDoS attack traffic from normal traffic is a challenging task, making it significant to effectively distinguish between attack types in order to resist DDoS attacks. However, traditional DDoS attack detection methods have certain limitations in terms of data preprocessing and detection efficiency. In this paper, we propose a lightweight framework based on deep learning called SAE-CNN-Detection (SCD), which combines stacked autoencoder network (SAE) and convolutional neural network (CNN) for DDoS attacks detection. The CIC-DDoS2019 dataset is used to simulate network traffic that has suffered from DDoS attacks, and this system employs adaptive preprocessing techniques for the dataset. The results demonstrate that multi-classification experiment achieves an accuracy of 97.2% for DDoS attack types, while the binary classification experiment achieves an accuracy of 99.1%.

Denoising magnetic resonance spectroscopy (MRS) data using stacked autoencoder for improving signal-to-noise ratio and speed of MRS.

While magnetic resonance imaging (MRI) provides high resolution anatomical images with sharp soft tissue contrast, magnetic resonance spectroscopy (MRS) enables non-invasive detection and measurement of biochemicals and metabolites. However, MRS has low signal-to-noise ratio (SNR) when concentrations of metabolites are in the range of millimolar. Standard approach of using a high number of signal averaging (NSA) to achieve sufficient SNR comes at the cost of a long acquisition time. We propose to use deep-learning approaches to denoise MRS data without increasing NSA. This method has potential to reduce the acquisition time as well as improve SNR and quality of spectra, which could enhance the diagnostic value and broaden the clinical applications of MRS. The study was conducted using data collected from the brain spectroscopy phantom and human subjects. We utilized a stack auto-encoder (SAE) network to train deep learning models for denoising low NSA data (NSA=1, 2, 4, 8, and 16) randomly truncated from high SNR data collected with high NSA (NSA=192), which were also used to obtain the ground truth. We applied both self-supervised and fully-supervised training approaches and compared their performance of denoising low NSA data based on improvement in SNR. To prevent overfitting, the SAE network was trained in a patch-based manner. We then tested the denoising methods on noise-containing data collected from the phantom and human subjects, including data from brain tumor patients. We evaluated their performance by comparing the SNR levels and mean squared errors (MSEs) calculated for the whole spectra against high SNR "ground truth", as well as the value of chemical shift of N-acetyl-aspartate (NAA) before and after denoising. With the SAE model, the SNR of low NSA data (NSA=1) obtained from the phantom increased by 28.5% and the MSE decreased by 42.9%. For low NSA data of the human parietal and temporal lobes, the SNR increased by 32.9% and the MSE decreased by 63.1%. In all cases, the chemical shift of NAA in the denoised spectra closely matched with the high SNR spectra without significant distortion to the spectra after denoising. Furthermore, the denoising performance of the SAE model was more effective in denoising spectra with higher noise levels. The reported SAE denoising method is a model-free approach to enhance the SNR of MRS data collected with low NSA. With the denoising capability, it is possible to acquire MRS data with a few NSA, shortening the scan time while maintaining adequate spectroscopic information for detecting and quantifying the metabolites of interest. This approach has the potential to improve the efficiency and effectiveness of clinical MRS data acquisition by reducing the scan time and increasing the quality of spectroscopic data.

Hierarchical graph augmented stacked autoencoders for multi-view representation learning

With recent success of deep neural networks, stacked autoencoder networks have received a lot of attention for robust unsupervised representation learning. However, recent autoencoder methods cannot make full use of multi-view information and thus fail to further improve many real-world applications by exploring the geometric structures of multi-view data. To address the above-mentioned issue, we introduce hierarchical graph augmented stacked autoencoders (HGSAE) for unsupervised multi-view representation learning. Specifically, a hierarchical graph structure is first adapted to stacked autoencoders to learn view-specific representations, aiming to preserve the geometric information of multi-view data through local and non-local graph regularizations. A general or common representation can then be learned by reconstructing each single view using fully connected neural networks. By doing this, the proposed method not only preserves the geometric information in multi-view data but also automatically balances the complementarity/consistency among different views. Extensive experiments on six popular unsupervised representation learning datasets demonstrate the effectiveness of our method when compared with recent state-of-the-art autoencoder methods.

Typical fault prediction method for wind turbines based on an improved stacked autoencoder network

Abstract Timely prediction of wind turbine states is valuable for reduction of potential significant losses resulting from deterioration of health condition. To enhance the accuracy of fault diagnosis and early warning, data collected from supervisory control and data acquisition (SCADA) system of wind turbines is graphically processed and used as input for a deep learning mode, which effectively reflects the correlation between the faults of different components of wind turbines and the multi-state information in SCADA data. An improved stacked autoencoder (ISAE) framework is proposed to address the issue of ineffective fault identification due to the scarcity of labeled samples for certain fault types. In the data augmentation module, synthetic samples are generated using SAE to enhance the training data. Another SAE model is trained using the augmented dataset in the data prediction module for future trend prediction. The attribute correlation information is embedded to compensate for the shortcomings of SAE in learning attribute relationships, and the optimal factor parameters are searched using the particle swarm optimization (PSO) algorithm. Finally, the state of wind turbines is predicted using a CNN-based fault diagnosis module. Experimental results demonstrate that the proposed method can effectively predict faults and identify fault types in advance, which is helpful for wind farms to take proactive measures and schedule maintenance plans to avoid significant losses.

Road traffic can be predicted by machine learning equally effectively as by complex microscopic model

Since high-quality real data acquired from selected road sections are not always available, a traffic control solution can use data from software traffic simulators working offline. The results show that in contrast to microscopic traffic simulation, the algorithms employing neural networks can work in real-time, so they can be used, among others, to determine the speed displayed on variable message road signs. This paper describes an experiment to develop and test machine learning models, i.e., long short-term memory, gated recurrent unit recurrent networks, and stacked autoencoder networks. It compares their effectiveness with traffic prediction results generated using a widely recognized traffic simulator that analyzes traffic at the level of individual vehicles.

Crop-Planting Area Prediction from Multi-Source Gaofen Satellite Images Using a Novel Deep Learning Model: A Case Study of Yangling District

Neural network models play an important role in crop extraction based on remote sensing data. However, when dealing with high-dimensional remote sensing data, these models are susceptible to performance degradation. In order to address the challenges associated with multi-source Gaofen satellite data, a novel method is proposed for dimension reduction and crop classification. This method combines the benefits of the stacked autoencoder network for data dimensionality reduction, and the convolutional neural network for classification. By leveraging the advantages of multi-dimensional remote sensing information, and mitigating the impact of dimensionality on the classification accuracy, this method aims to improve the effectiveness of crop classification. The proposed method was applied to the extraction of crop-planting areas in the Yangling Agricultural Demonstration Zone, using multi-temporal spectral data collected from the Gaofen satellites. The results demonstrate that the fusion network, which extracts low-dimensional characteristics, offers advantages in classification accuracy. At the same time, the proposed model is compared with methods such as the decision tree (DT), random forest (RF), support vector machine (SVM), hyperspectral image classification based on a convolutional neural network (HICCNN), and a characteristic selection classification method based on a convolutional neural network (CSCNN). The overall accuracy of the proposed method can reach 98.57%, which is 7.95%, 4.69%, 5.68%, 1.21%, and 1.10% higher than the above methods, respectively. The effectiveness of the proposed model was verified through experiments. Additionally, the model demonstrates a strong robustness when classifying based on new data. When extracting the crop area of the entire Yangling District, the errors for wheat and corn are only 9.6% and 6.3%, respectively, and the extraction results accurately reflect the actual planting situation of crops.

Spatial and temporal prediction of secondary crashes combining stacked sparse auto-encoder and long short-term memory

Secondary crashes occur within the spatial and temporal impact area of primary crashes, resulting in traffic delays and safety problems. While most existing studies focus on the likelihood of secondary crashes, predicting the spatio-temporal location of secondary crashes could offer valuable insights for implementing prevention strategies. This includes guiding the deployment of emergency response measures and determining appropriate speed limits. The main objective of this study is to develop a prediction method for the spatial and temporal locations of secondary crashes. A hybrid deep learning model SSAE-LSTM is proposed by combining stacked sparse auto-encoder (SSAE) and long short-term memory network (LSTM). Traffic and crash data on the California I-880 highway covering the period of 2017–2021 are collected. The identification of secondary crashes is performed by the speed contour map method. The time and distance gaps between primary and secondary crashes are modeled using multiple 5-minute interval traffic variables as inputs. Multiple models are developed for benchmarking purposes, including PCA-LSTM, which incorporates principal component analysis (PCA) and LSTM, SSAE-SVM, which incorporates SSAE and support vector machine (SVM), and back propagation neural network (BPNN). The performance comparison indicates that the hybrid SSAE-LSTM model outperforms the other models in terms of both spatial and temporal prediction. In particular, SSAE4-LSTM1 (with 4 SSAE layers and 1 LSTM layer) demonstrates superior spatial prediction performance, while SSAE4-LSTM2 (with 4 SSAE layers and 2 LSTM layers) excels in temporal prediction. A joint spatio-temporal evaluation is also conducted to measure the overall accuracy of the optimal models over different permitted spatio-temporal ranges. Finally, practical suggestions are provided for secondary crash prevention.

Deep Neural Networks in Power Systems: A Review

Identifying statistical trends for a wide range of practical power system applications, including sustainable energy forecasting, demand response, energy decomposition, and state estimation, is regarded as a significant task given the rapid expansion of power system measurements in terms of scale and complexity. In the last decade, deep learning has arisen as a new kind of artificial intelligence technique that expresses power grid datasets via an extensive hypothesis space, resulting in an outstanding performance in comparison with the majority of recent algorithms. This paper investigates the theoretical benefits of deep data representation in the study of power networks. We examine deep learning techniques described and deployed in a variety of supervised, unsupervised, and reinforcement learning scenarios. We explore different scenarios in which discriminative deep frameworks, such as Stacked Autoencoder networks and Convolution Networks, and generative deep architectures, including Deep Belief Networks and Variational Autoencoders, solve problems. This study’s empirical and theoretical evaluation of deep learning encourages long-term studies on improving this modern category of methods to accomplish substantial advancements in the future of electrical systems.

Deep Latent Space Clustering for Detection of Stealthy False Data Injection Attacks Against AC State Estimation in Power Systems

This paper proposes a self-supervised latent space clustering algorithm, called the Deep Latent Space Clustering, for the detection of stealthy false data injection attacks (FDIAs) in smart grids against state estimation algorithms. The stealthy FDI model considered is based on the accurate AC state estimation and is able to bypass conventional bad data detection (BDD) algorithms with ease. The key element of the detection model is a stacked autoencoder network that first undergoes a carefully designed two-step finetuning process, following which a trainable clustering head is stacked on top of the finetuned encoder and the final network is further trained to achieve a clean clustering of the data into benign and compromised samples without labelled supervision. To test the efficacy and scalability of the detection model, it is tested on the standard IEEE 14 bus, 118 bus and 300 bus test systems. The self-supervised clustering model is compared to several supervised, semi-supervised and unsupervised algorithms proposed in the literature for detection of FDI on the aforementioned test cases and has been found to perform at par with the state of the art among them.

Classification of lemon quality using hybrid model based on Stacked AutoEncoder and convolutional neural network

Classification of lemon quality using hybrid model based on Stacked AutoEncoder and convolutional neural network

A non‐linear non‐intrusive reduced order model of fluid flow by auto‐encoder and self‐attention deep learning methods

AbstractThis paper presents a new nonlinear non‐intrusive reduced‐order model (NL‐NIROM) that outperforms traditional proper orthogonal decomposition (POD)‐based reduced order model (ROM). This improvement is achieved through the use of auto‐encoder (AE) and self‐attention based deep learning methods. The novelty of this work is that it uses stacked auto‐encoder (SAE) network to project the original high‐dimensional dynamical systems onto a low dimensional nonlinear subspace and predict fluid dynamics using an self‐attention based deep learning method. This paper introduces a new model reduction neural network architecture for fluid flow problem, as well as, a linear non‐intrusive reduced order model (L‐NIROM) based on POD and self‐attention mechanism. In the NL‐NIROM, the SAE network compresses high‐dimensional physical information into several much smaller sized representations in a reduced latent space. These representations are expressed by a number of codes in the middle layer of SAE neural network. Then, those codes at different time levels are trained to construct a set of hyper‐surfaces using self‐attention based deep learning methods. The inputs of the self‐attention based network are previous time levels' codes and the outputs of the network are current time levels' codes. The codes at current time level are then projected back to the original full space by the decoder layers in the SAE network. The capability of the new model, NL‐NIROM, is demonstrated through two test cases: flow past a cylinder, and a lock exchange. The results show that the NL‐NIROM is more accurate than the popular model reduction method namely POD based L‐NIROM.

Screening Herbal Compound Candidates for Use as Anti-Inflammatory Drugs for COVID-19 Treatment Using Deep Semisupervised Learning

COVID-19 is a disease caused by the SARS-CoV-2 virus. Its symptoms include cough, fever, shortness of breath, and acute inflammation (hyperinflammation), and severe cases can lead to death. These symptoms tend to worsen if inflammation is not controlled. This research aims to build a stacked autoencoder deep neural network (SAE-DNN) model for identifying herbal compound candidates that can be used as anti-inflammatory drugs for COVID-19 treatment. The model’s performance is evaluated on the basis of different data representations. The research process involves data collection, data preprocessing, modeling, and testing the model on the herbal data to obtain herbal compound candidates. Results indicate that the developed SAE-DNN model with compound representation that combines fingerprints and dipeptide composition produces the best performance with an accuracy of 0.96722, a recall of 0.96419, area under the receiver operating characteristic of 0.99596, and an F1 score of 0.96567. A total of 33 herbal compounds are found as candidate anti-inflammatory drugs by using the SAE-DNN model.

Soft sensor method of multimode BOF steelmaking endpoint carbon content and temperature based on vMF-WSAE dynamic deep learning

Abstract The difficulty of endpoint determination in basic oxygen furnace (BOF) steelmaking lies in achieving accurate real-time measurements of carbon content and temperature. For the characteristics of serious nonlinearity between process data, deep learning can perform excellent nonlinear feature representation for complex structural data. However, there is a process drift phenomenon in BOF steelmaking, and the existing deep learning-based soft sensor models cannot adapt to changes in the characteristics of samples, which may lead to their performance degradation. To deal with this problem, considering the characteristics of multimode distribution of process data, an adaptive updating deep learning model based on von-Mises Fisher (vMF) mixture model and weighted stacked autoencoder is proposed. First, the stacked autoencoder (SAE) and vMF mixture model are constructed for complex structural data, which can initially establish nonlinear mapping relationships and division of different distributions. Second, for each query sample, the basic SAE network will perform online adaptive fine-tuning according to its data with the same distribution to achieve dynamic updating. Moreover, each sample is assigned a weight according to its similarity with the query sample. Through the designed weighted loss function, the updated deep network will better match the working conditions of the query sample. Experimental studies with numerical examples and actual BOF steelmaking process data are provided to demonstrate the effectiveness of the proposed method.

Fault diagnosis of a mixed-flow pump under cavitation condition based on deep learning techniques

Deep learning technique is an effective mean of processing complex data that has emerged in recent years, which has been applied to fault diagnosis of a wide range of equipment. In the present study, three types of deep learning techniques, namely, stacked autoencoder (SAE) network, long short term memory (LSTM) network, and convolutional neural network (CNN) are applied to fault diagnosis of a mixed-flow pump under cavitation conditions. Vibration signals of the mixed-flowed pump are collected from experiment measurements, and then employed as input datasets for deep learning networks. The operation status is clarified into normal, minor cavitation, and severe cavitation conditions according to visualized bubble density. The techniques of FFT and dropout algorithms are also applied to improve diagnosis accuracy. The results show that the diagnosis accuracy based on SAE and LSTM networks is lower than 50%, while is higher than 68% when using CNN. The maximum accuracy can reach 87.2% by mean of a combination of CNN, BN, MLP, and using frequency domain data by FFT as inputs, which validates the feasibility of applying CNN in mixed-flow pumps.

Convolutional Neural Network based Retinal Vessel Segmentation

In human eye, the state of the blood vessel is a crucial diagnostic factor. The segmentation of blood vessel from the fundus image is difficult due to the spatial complexity, adjacency, overlapping and variability of blood vessel. The detection of ophthalmic pathologies like hypertensive disorders, diabetic retinopathy and cardiovascular diseases are remain challenging task due to the wide-ranging distribution of blood vessels. In this paper, Stacked Autoencoder and CNN (Convolutional Neural Network) technique is proposed to extract the blood vessel from the fundus image. Based on the experiments conducted using the Stacked Autoencoder and Convolutional Neural Network gives 90% & 95% accuracy for segmentation.