Autoencoder-based Deep Neural Network Research Articles

Deep neural network for damage detection in Infante Dom Henrique bridge using multi-sensor data

This paper proposes a data-driven approach to detect damage using monitoring data from the Infante Dom Henrique bridge in Porto. The main contribution of this work lies in exploiting the combination of raw measurements from local (inclinations and stresses) and global (eigenfrequencies) variables in a full-scale structural health monitoring application. We exhaustively analyze and compare the advantages and drawbacks of employing each variable type and explore the potential of combining them. An autoencoder-based deep neural network is employed to properly reconstruct measurements under healthy conditions of the structure, which are influenced by environmental and operational variability. The damage-sensitive feature for outlier detection is the reconstruction error that measures the discrepancy between current and estimated measurements. Three autoencoder architectures are designed according to the input: local variables, global variables, and their combination. To test the performance of the methodology in detecting the presence of damage, we employ a finite element model to calculate the relative change in the structural response induced by damage at four locations. These relative variations between the healthy and damaged responses are employed to affect the experimental testing data, thus producing realistic time-domain damaged measurements. We analyze the receiver operating characteristic curves and investigate the latent feature representation of the data provided by the autoencoder in the presence of damage. Results reveal the existence of synergies between the different variable types, producing almost perfect classifiers throughout the performed tests when combining the two available data sources. When damage occurs far from the instrumented sections, the area under the curve in the combined approach increases [Formula: see text] compared to using local variables only. The classificatoin metrics also demonstrate the enhancement of combining both sources of data in the damage detection task, reaching close to [Formula: see text] precision values for the four considered test damage scenarios. Finally, we also investigate the capability of local variables to localize the damage, demonstrating the potential of including these variables in the damage detection task.

Intelligent Prediction of Network Security Situations based on Deep Reinforcement Learning Algorithm

The limitations of traditional network security assessment methods characterized by manual definitions and measurements, data overload, poor performance, and non-negligible drawbacks are addressed in this research. A novel network security system employing a deep learning algorithm is proposed to overcome these challenges. The research unfolds in three key phases. First, a deep self-encoding model is developed to distinguish various network attacks effectively. Subsequently, the creation of missing measurement weights enhances pattern detection, even when dealing with a limited number of training samples. Finally, the model assesses and computes attack issues, assigns impact scores to each attack, and determines the overall network security value. Experimental results demonstrate that the deep auto encoder-based deep neural network (DAEDNN), in conjunction with the proposed unique oversampling weighting (UOSW) algorithm, significantly outperforms traditional methods such as decision trees (DT), support vector machines (SVM), and long short-term memory (LSTM) models. The F1 score of UOSW surpasses these models by approximately 2.77, 10.5, and 5.2, respectively. The deep self-encoding model employed in the proposed system offers superior accuracy and recall rates, leading to more precise and efficient measurement results.

Patient specific higher order tensor based approach for the detection and localization of myocardial infarction using 12-lead ECG

Myocardial Infarction (MI) is an emergency condition that requires immediate medical treatment. The rapid and accurate diagnosis of MI using a 12-lead electrocardiogram (ECG) is extremely important in a clinical study to save the patient’s life. The manual interpretation of MI using a 12-lead ECG is tedious and time-consuming. Therefore, a patient-specific software-based computer-aided diagnosis framework is helpful to detect and localize MI disease accurately. This paper proposes a patient-specific higher-order tensor-based approach to detect and localize MI automatically using 12-lead ECG recordings. The 12-lead ECG recordings are segmented into 12-lead ECG beats using the multi-lead fusion-based QRS detection algorithm. The fast and adaptive multivariate empirical mode decomposition (FA-MVEMD) based multiscale analysis method decomposes 12-lead ECG beat into a third-order tensor containing the information from the samples, beat, and intrinsic mode functions (IMFs). Furthermore, a fourth-order tensor is formulated by considering beats, samples, lead, and IMFs information of 12-lead ECG recording. The multilinear singular value decomposition (MLSVD) extracts features from the fourth-order tensors and third-order tensors of 12-lead ECG. The K-nearest neighbor (KNN), support vector machine (SVM), and stacked autoencoder-based deep neural network (SAE-DNN) models are used for the detection and localization of MI using fourth-order and third-order tensor domain features. The proposed approach is evaluated using 73 healthy control (HC) and 100 different types of MI-based 12-lead ECG recordings from a public database. The proposed approach has obtained the classification accuracy values of (98.84%, 98.27%, 98.27%) and (86.64%, 83.17%, and 81.98%) using (KNN, SVM, and SAE-DNN) models for MI detection, and localization, respectively using 30-min duration of 12-lead ECG recordings. For MI detection and localization, the suggested approach has obtained accuracy values of 96.53% and 93.32%, respectively, using the 4-s duration of 12-lead ECG recordings. Our approach outperformed existing MI detection and localization methods using 12-lead ECG recordings regarding classification performance.

Three-stage data generation algorithm for multiclass network intrusion detection with highly imbalanced dataset

The Internet plays a crucial role in our daily routines. Ensuring cybersecurity to Internet users will provide a safe online environment. Automatic network intrusion detection (NID) using machine learning algorithms has recently received increased attention recently. The NID model is prone to bias towards the classes with more training samples due to highly imbalanced datasets across different types of attacks. The challenge in generating additional training data for minority classes is the generation of insufficient data. The study's purpose is to address this challenge, which extends the data generation ability by proposing a three-stage data generation algorithm using the synthetic minority over-sampling technique, a generative adversarial network (GAN), and a variational autoencoder. A convolutional neural network is employed to extract the representative features from the data, which were fed into a support vector machine with a customised kernel function. An ablation study evaluated the effectiveness of the three-stage data generation, feature extraction, and customised kernel. This was followed by a performance comparison between our study and existing studies. The findings revealed that the proposed NID model achieved an accuracy of 91.9%–96.2% in the four benchmark datasets. In addition, it outperformed existing methods such as GAN-based deep neural networks, conditional Wasserstein GAN-based stacked autoencoder, synthesised minority oversampling technique-based random forest, and variational autoencoder-based deep neural network, by 1.51%–28.4%.

Channel Estimation for UAV Communication Systems Using Deep Neural Networks

Channel modeling of unmanned aerial vehicles (UAVs) from wireless communications has gained great interest for rapid deployment in wireless communication. The UAV channel has its own distinctive characteristics compared to satellite and cellular networks. Many proposed techniques consider and formulate the channel modeling of UAVs as a classification problem, where the key is to extract the discriminative features of the UAV wireless signal. For this issue, we propose a framework of multiple Gaussian–Bernoulli restricted Boltzmann machines (GBRBM) for dimension reduction and pre-training utilization incorporated with an autoencoder-based deep neural network. The developed system used UAV measurements of a town’s already existing commercial cellular network for training and validation. To evaluate the proposed approach, we run ray-tracing simulations in the program Remcom Wireless InSite at a distinct frequency of 28 GHz and used them for training and validation. The results demonstrate that the proposed method is accurate in channel acquisition for various UAV flying scenarios and outperforms the conventional DNNs.

A new framework for damage detection of steel frames using burg autoregressive and stacked autoencoder-based deep neural network

A new framework for damage detection of steel frames using burg autoregressive and stacked autoencoder-based deep neural network

Forest Fire Prediction with Imbalanced Data Using a Deep Neural Network Method

Forests suffer from heavy losses due to the occurrence of fires. A prediction model based on environmental condition, such as meteorological and vegetation indexes, is considered a promising tool to control forest fires. The construction of prediction models can be challenging due to (i) the requirement of selection of features most relevant to the prediction task, and (ii) heavily imbalanced data distribution where the number of large-scale forest fires is much less than that of small-scale ones. In this paper, we propose a forest fire prediction method that employs a sparse autoencoder-based deep neural network and a novel data balancing procedure. The method was tested on a forest fire dataset collected from the Montesinho Natural Park of Portugal. Compared to the best prediction results of other state-of-the-art methods, the proposed method could predict large-scale forest fires more accurately, and reduces the mean absolute error by 3–19.3 and root mean squared error by 0.95–19.3. The proposed method can better benefit the management of wildland fires in advance and the prevention of serious fire accidents. It is expected that the prediction performance could be further improved if additional information and more data are available.

A method for fault detection in multi-component systems based on sparse autoencoder-based deep neural networks

In multi-component systems, degradation, maintenance, renewal and operational mode change continuously the operating conditions. The identification of the onset of abnormal conditions from signal measurements taken in such evolving environments can be quite challenging, due to the difficulty of distinguishing the real cause of the signal variations. In this work, we present a method for fault detection in evolving environments that uses a Sparse Autoencoder-based Deep Neural Network (SAE-DNN) and a novel procedure that remarkably reduces the computational burden for setting the values of the hyperparameters. The method is applied to a synthetic case study and to a bearing vibration dataset. The results show that it is able to accurately detect faults in multi-component systems, outperforming other state-of-the-art methods.

Deep Learning Aided Data-Driven Fault Diagnosis of Rotatory Machine: A Comprehensive Review

This paper presents a comprehensive review of the developments made in rotating bearing fault diagnosis, a crucial component of a rotatory machine, during the past decade. A data-driven fault diagnosis framework consists of data acquisition, feature extraction/feature learning, and decision making based on shallow/deep learning algorithms. In this review paper, various signal processing techniques, classical machine learning approaches, and deep learning algorithms used for bearing fault diagnosis have been discussed. Moreover, highlights of the available public datasets that have been widely used in bearing fault diagnosis experiments, such as Case Western Reserve University (CWRU), Paderborn University Bearing, PRONOSTIA, and Intelligent Maintenance Systems (IMS), are discussed in this paper. A comparison of machine learning techniques, such as support vector machines, k-nearest neighbors, artificial neural networks, etc., deep learning algorithms such as a deep convolutional network (CNN), auto-encoder-based deep neural network (AE-DNN), deep belief network (DBN), deep recurrent neural network (RNN), and other deep learning methods that have been utilized for the diagnosis of rotary machines bearing fault, is presented.

High-speed computer-generated holography using an autoencoder-based deep neural network.

Learning-based computer-generated holography (CGH) provides a rapid hologram generation approach for holographic displays. Supervised training requires a large-scale dataset with target images and corresponding holograms. We propose an autoencoder-based neural network (holoencoder) for phase-only hologram generation. Physical diffraction propagation was incorporated into the autoencoder's decoding part. The holoencoder can automatically learn the latent encodings of phase-only holograms in an unsupervised manner. The proposed holoencoder was able to generate high-fidelity 4K resolution holograms in 0.15 s. The reconstruction results validate the good generalizability of the holoencoder, and the experiments show fewer speckles in the reconstructed image compared with the existing CGH algorithms.

An Adaptive Human Activity-Aided Hand-Held Smartphone-Based Pedestrian Dead Reckoning Positioning System

Pedestrian dead reckoning (PDR), enabled by smartphones’ embedded inertial sensors, is widely applied as a type of indoor positioning system (IPS). However, traditional PDR faces two challenges to improve its accuracy: lack of robustness for different PDR-related human activities and positioning error accumulation over elapsed time. To cope with these issues, we propose a novel adaptive human activity-aided PDR (HAA-PDR) IPS that consists of two main parts, human activity recognition (HAR) and PDR optimization. (1) For HAR, eight different locomotion-related activities are divided into two classes: steady-heading activities (ascending/descending stairs, stationary, normal walking, stationary stepping, and lateral walking) and non-steady-heading activities (door opening and turning). A hierarchical combination of a support vector machine (SVM) and decision tree (DT) is used to recognize steady-heading activities. An autoencoder-based deep neural network (DNN) and a heading range-based method to recognize door opening and turning, respectively. The overall HAR accuracy is over 98.44%. (2) For optimization methods, a process automatically sets the parameters of the PDR differently for different activities to enhance step counting and step length estimation. Furthermore, a method of trajectory optimization mitigates PDR error accumulation utilizing the non-steady-heading activities. We divided the trajectory into small segments and reconstructed it after targeted optimization of each segment. Our method does not use any a priori knowledge of the building layout, plan, or map. Finally, the mean positioning error of our HAA-PDR in a multilevel building is 1.79 m, which is a significant improvement in accuracy compared with a baseline state-of-the-art PDR system.

An end-to-end denoising autoencoder-based deep neural network approach for fault diagnosis of analog circuit

Fault diagnosis of analog circuit is critical to improve safety and reliability in electrical systems and reduce losses. Traditional fault diagnosis methods of analog circuit usually rely on the hand design feature extractor and can not generalize well to other diagnosis domains. To address these issues, an end-to-end denoising autoencoder (EEDAE)-based fault diagnosis approach is proposed. The proposed approach includes denoising autoencoder (DAE) and a softmax classifier. The DAE is designed to automatically extract fault features from the raw time series signals without any signal processing techniques and diagnostic expertise, and then the softmax classifier is used to classify the fault mode of analog circuits. Specifically, we design a novel loss function by jointly minimizing reconstruction loss and classification loss to improve training efficiency. The proposed approach just has one training stage, in which the encoder, decoder, and classifier are trained simultaneously. The experimental results demonstrate that compared with traditional methods, the proposed method has higher accuracy and lower requirements on data.

A Stack Autoencoders Based Deep Neural Network Approach for Cervical Cell Classification in Pap-Smear Images

Background: Early detection of cervical cancer may give life to women all over the world. Pap-smear test and Human papillomavirus test are techniques used for the detection and prevention of cervical cancer. Objective: In this paper, pap-smear images are analysed and cells are classified using stacked autoencoder based deep neural network. Pap-smear cells are classified into 2 classes and 4 classes. Twoclass classification includes classification of cells in normal and abnormal cells while four-class classification includes classification of cells in normal cells , mild dysplastic cells, moderate dysplastic cells and severe dysplastic cells. Methods: The features are extracted by deep neural networks based on their architecture. Proposed deep neural networks consist of three stacked auto encoders with hidden sizes 512, 256 and 128, respectively. Softmax used as the outer layer for the classification of pap smear cells. Results: Average accuracy achieved for 2-class classification among normal and abnormal cells is 98.2 % while for 4-class classification among normal, mild, moderate and severe dysplastic cells is 93.8 % respectively. Conclusion: The proposed approach avoids image segmentation and feature extraction applied by previous works. This study highlights deep learning as an important tool for cells classification of pap-smear images. The accuracy of the proposed method may vary with the different combination of hidden size and number of autoencoders.

Network traffic classification using deep convolutional recurrent autoencoder neural networks for spatial–temporal features extraction

The right choice of features to be extracted from individual or aggregated observations is an extremely critical factor for the success of modern network traffic classification approaches based on machine learning. Such activity, usually in charge of the designers of the classification scheme is strongly related to their experience and skills, and definitely characterizes the whole approach, implementation strategy as well as its performance. The main aim of this work is supporting this process by mining new and more expressive, meaningful and discriminating features from the basic ones without human intervention. For this purpose, a novel autoencoder-based deep neural network architecture is proposed where multiple autoencoders are embedded with convolutional and recurrent neural networks to elicit relevant knowledge about the relations existing among the basic features (spatial-features) and their evolution over time (temporal-features). Such knowledge, consisting in new properties that are not immediately evident and better represent the most hidden and representative traffic dynamics can be successfully exploited by machine learning-based classifiers. Different network combinations are analyzed both from a theoretical perspective, and through specific performance evaluation experiments on a real network traffic dataset. We show that the traffic classifier obtained by stacking the autoencoder with a fully-connected neural network, achieves up to a 28% improvement in average accuracy over state-of-the-art machine learning-based approaches, up to a 10% over pure convolutional and recurrent stacked neural networks, and 18% over pure feed-forward networks. It is also able to maintain high accuracy even in the presence of unbalanced training datasets.

Novelty detection meets collider physics

Novelty detection is the machine learning task to recognize data, which belong to an unknown pattern. Complementary to supervised learning, it allows to analyze data model-independently. We demonstrate the potential role of novelty detection in collider physics, using autoencoder-based deep neural network. Explicitly, we develop a set of density-based novelty evaluators, which are sensitive to the clustering of unknown-pattern testing data or new-physics signal events, for the design of detection algorithms. We also explore the influence of the known-pattern data fluctuations, arising from non-signal regions, on detection sensitivity. Strategies to address it are proposed. The algorithms are applied to detecting fermionic di-top partner and resonant di-top productions at LHC, and exotic Higgs decays of two specific modes at a $e^+e^-$ future collider. With parton-level analysis, we conclude that potentially the new-physics benchmarks can be recognized with high efficiency.

Multi-DoF continuous estimation for wrist torques using stacked autoencoder

Human machine interface (HMI) based on surface electromyography (sEMG) promises to provide an intuitive and noninvasive way to interact with peripheral equipments, such as prostheses, exoskeletons, and robots. Most recently, advances in machine learning, especially in deep learning algorithms, present the capabilities in constructing complicated mapping functions. In this study, we construct a stacked autoencoder-based deep neural network (SAE-DNN) to continuously estimate multiple degrees-of-freedom (DoFs) kinetics of wrist from sEMG signals. During the experiments, high-density sEMG signals and multi-DoF wrist torques were simultaneously acquired under the guidance of a visual feedback system, with eight healthy subjects and an amputee recruited. Moreover, the estimation performance of SAE-DNN was compared with two of commonly used conventional regressors, linear regression (LR) and support vector regression (SVR). As a consequence, the results demonstrate the feasibility of this scheme and significant superiority of SAE-DNN over LR and SVR with higher R2 values across all DoFs (SAE-DNN: 0.829±0.050, LR: 0.757±0.075, SVR: 0.751±0.079). The outcomes of this study provide us with a perspective and a feasible scheme for simultaneous and proportional control.

ReLTanh: An activation function with vanishing gradient resistance for SAE-based DNNs and its application to rotating machinery fault diagnosis

Tanh is a sigmoidal activation function that suffers from vanishing gradient problem, so researchers have proposed some alternative functions including rectified linear unit (ReLU), however those vanishing-proof functions bring some other problem such as bias shift problem and noise-sensitiveness as well. Mainly for overcoming vanishing gradient problem as well as avoiding to introduce other problems, we propose a new activation function named Rectified Linear Tanh (ReLTanh) by improving traditional Tanh. ReLTanh is constructed by replacing Tanh’s saturated waveforms in positive and negative inactive regions with two straight lines, and the slopes of the lines are calculated by the Tanh’s derivatives at two learnable thresholds. The middle Tanh waveform provides ReLTanh with the ability of nonlinear fitting, and the linear parts contribute to the relief of vanishing gradient problem. Besides, thresholds of ReLTanh that determines the slopes of line parts are learnable, so it can tolerate the variation of inputs and help to minimize the cost function and maximize the data fitting performance. Theoretical proofs by mathematical derivations demonstrate that ReLTanh is available to diminish vanishing gradient problem and feasible to train thresholds. For verifying the practical feasibility and effectiveness of ReLTanh, fault diagnosis experiments for planetary gearboxes and rolling bearings are conducted by stacked autoencoder-based deep neural network (SAE-based DNNs). ReLTanh alleviates successfully vanishing gradient problem and the it learns faster, more steadily and precisely than Tanh, which is consistent with the theoretical analysis. Additionally, ReLTanh surpasses other popular activation functions such as ReLU family, Hexpo and Swish, which shows that ReLTanh has certain applying potential and researching value.

Deep Discriminative Representation Learning for Nonlinear Process Fault Detection

Nonlinear process fault detection remains a challenge, with representation learning being a key step. In this article, a deep neural network (DNN)-based discriminative representation learning approach is proposed to achieve efficient fault detection for nonlinear plant-wide processes. An early-stage fault rarely affects several independent variables concurrently; hence, mutual information-based block division and randomized fault construction are performed to generate faulty validation data. By using the training data from the normal operation training data and the constructed validation data, a DNN with stacked autoencoders and a softmax classifier is trained to generate discriminative representations that maximize the capability of discriminating normal and abnormal statuses. Finally, on the basis of the learned deep discriminative representations, support vector data description is employed to discriminate the normal and abnormal process statuses. The proposed monitoring approach is tested on a numerical example and an industrial tail-gas treatment process, through which the efficiency is verified. Note to Practitioners —A modern process is generally characterized by a large scale and complex nonlinear correlation, and monitoring of such nonlinear plant-wide processes is imperative. Nowadays, a large amount of process data is generally available, and deep neural network-based monitoring is promising in dealing with such data on nonlinear processes. This article proposes a deep discriminative representation learning method for efficient nonlinear plant-wide process monitoring. The key idea is to first decompose a large-scale process into multiple units according to a variable relationship, and then generate faulty validation data based on randomized fault construction. Then, deep discriminative representations are learned by optimizing a stacked autoencoder-based deep neural network (DNN). This article provides guidelines for designing an efficient monitoring algorithm for plant-wide nonlinear processes in the industrial big data environment.

Open-circuit fault diagnosis of power rectifier using sparse autoencoder based deep neural network

Abstract This paper is concerned with the open-circuit fault diagnosis of phase-controlled three-phase full-bridge rectifier by using a sparse autoencoder-based deep neural network (SAE-based DNN). Firstly, some preliminaries on SAE-based DNN are briefly introduced to automatically learn the representative fault features from the raw fault signals. Then, a novel strategy is developed to design the structure of the SAE-based DNN, by which the depth and hidden neurons of the SAE-based DNN could be regularly determined to extract the features of input signals. Furthermore, the fault model and system framework are presented to diagnose the open-circuit fault of the three-phase full-bridge rectifier. Finally, the effectiveness of the developed novel strategy is verified by the results of simulation experiments, and the superiority of the novel SAE-based DNN is evaluated by comparing with other frequently used approaches.

A sparse autoencoder-based deep neural network for protein solvent accessibility and contact number prediction

BackgroundDirect prediction of the three-dimensional (3D) structures of proteins from one-dimensional (1D) sequences is a challenging problem. Significant structural characteristics such as solvent accessibility and contact number are essential for deriving restrains in modeling protein folding and protein 3D structure. Thus, accurately predicting these features is a critical step for 3D protein structure building.ResultsIn this study, we present DeepSacon, a computational method that can effectively predict protein solvent accessibility and contact number by using a deep neural network, which is built based on stacked autoencoder and a dropout method. The results demonstrate that our proposed DeepSacon achieves a significant improvement in the prediction quality compared with the state-of-the-art methods. We obtain 0.70 three-state accuracy for solvent accessibility, 0.33 15-state accuracy and 0.74 Pearson Correlation Coefficient (PCC) for the contact number on the 5729 monomeric soluble globular protein dataset. We also evaluate the performance on the CASP11 benchmark dataset, DeepSacon achieves 0.68 three-state accuracy and 0.69 PCC for solvent accessibility and contact number, respectively.ConclusionsWe have shown that DeepSacon can reliably predict solvent accessibility and contact number with stacked sparse autoencoder and a dropout approach.