Short-term Memory Autoencoder Research Articles

Deep learning based detection and localization of road accidents from traffic surveillance videos

Real-world traffic surveillance videos need continuous supervision to monitor and take appropriate actions in case of fatal accidents. However, continuously monitoring them with human supervision is error prone and tedious. Therefore, a deep learning approach for automatic detection and localization of road accidents has been proposed by formulating the problem as anomaly detection. The method follows one-class classification approach and applies spatio-temporal autoencoder and sequence-to-sequence long short-term memory autoencoder for modeling spatial and temporal representations in the video. The model is executed on a real-world video traffic surveillance datasets and significant results have been achieved both qualitatively and quantitatively.

Multi-output Spatio-temporal air pollution forecasting using neural network approach

Multi-step ahead pollution forecasting has an essential role in mitigating health risks. Multi-Input and Multi-Output (MIMO) multi-step forecasting is valuable, especially for long-term pollution forecasting. Multi-output pollution forecasting results help comprehend the overall idea of a particular location’s air quality level for a better environmental decision-making process. The traditional single output forecasting techniques include each step’s errors and ignore its prediction values for multi-step ahead forecasting. The proposed Multi-Output Long Short-Term Memory Auto-Encoder (M-LSTMA) with the MIMO approach accumulates each step prediction value to conduct multi-step forecasting in light of this challenge. In addition to that, the proposed model utilized a multiple imputation approach to handle the missing values of the dataset and employed both the climate and spatial attributes to improve forecasting results. Overall, the results demonstrate that the M-LSTMA model improves forecasting accuracy more significantly than the baseline models, proving its efficiency for long-term Spatio-temporal forecasting.

Hybrid machine learning-based model for solubilities prediction of various gases in deep eutectic solvent for rigorous process design of hydrogen purification

Deep eutectic solvents (DES) are used as a green sustainable alternative to room temperature ionic liquids (RTILs), given their low cost and environmentally friendly nature. In this work, solubilities of CO2, CO, CH4, H2 and N2 gases in choline chloride/urea (ChCl/Urea) based DES is investigated. Experimental solubility data from literature is used to train machine learning models to predict the solubilities of different gases in ChCl/Urea at temperatures ranging from 298.15 K to 372.15 K and pressure ranging from 0.01 to 5 MPa. In this context, a Support Vector Machine and Long Short-term Memory Auto Encoder–based hybrid machine learning model is proposed. The hybrid model exhibited excellent prediction accuracy with low root mean square error values of 0.000985, 0.00055, 0.00037, 0.000583 and 0.000164 for CO2, CO, CH4, H2 and N2, respectively. The predicted solubility data is regressed in commercial software Aspen Plus V11 for process design of H2 separation from gaseous feed mixture. Complex feed mixture consisting of CO2, CH4, H2, CO, and N2 is absorbed in ChCl/Urea. As a result, hydrogen is recovered and purified from complex feed mixture at specific energy consumption of 6.03 kWh/kgH2. Furthermore, carbon removal is observed as > 99% from feed gas at the expense of 3.02 MJ/kgCO2.

A semi-supervised leakage detection method driven by multivariate time series for natural gas gathering pipeline

Real-time and accurate leakage detection of natural gas gathering pipelines is critical to the safe and reliable operation of the gas and oil industry. Modern data-driven fault detection and diagnosis techniques have recently gained increasing attention because model-based techniques are practically prohibitive. However, most existing data-driven leakage detection approaches are obtained through supervised learning, which requires a substantial set of labelled data. Especially in real-world scenarios, leakage samples are rare. To reduce the dependence of the leakage detection method on leakage data and make full use of numerous normal datasets generated under normal working conditions, we propose a semi-supervised leakage detection method which consists of two components: an improved long short term memory autoencoder (LSTM-AE) network and one-class support vector machine (OCSVM). The LSTM-AE is first trained to learn the intrinsic features of a normal dataset of pipeline parameters which are multivariate time series. The OCSVM is then applied to calculate the score which is used to infer leak existence. The performance of the proposed method is evaluated on the real natural gas gathering pipelines, and the results confirm that the proposed method achieved 98% accuracy and 99% AUC (Area Under Curve) in a real-life dataset.

A machine-learning approach for long-term prediction of experimental cardiac action potential time series using an autoencoder and echo state networks.

Computational modeling and experimental/clinical prediction of the complex signals during cardiac arrhythmias have the potential to lead to new approaches for prevention and treatment. Machine-learning (ML) and deep-learning approaches can be used for time-series forecasting and have recently been applied to cardiac electrophysiology. While the high spatiotemporal nonlinearity of cardiac electrical dynamics has hindered application of these approaches, the fact that cardiac voltage time series are not random suggests that reliable and efficient ML methods have the potential to predict future action potentials. This work introduces and evaluates an integrated architecture in which a long short-term memory autoencoder (AE) is integrated into the echo state network (ESN) framework. In this approach, the AE learns a compressed representation of the input nonlinear time series. Then, the trained encoder serves as a feature-extraction component, feeding the learned features into the recurrent ESN reservoir. The proposed AE-ESN approach is evaluated using synthetic and experimental voltage time series from cardiac cells, which exhibit nonlinear and chaotic behavior. Compared to the baseline and physics-informed ESN approaches, the AE-ESN yields mean absolute errors in predicted voltage 6-14 times smaller when forecasting approximately 20 future action potentials for the datasets considered. The AE-ESN also demonstrates less sensitivity to algorithmic parameter settings. Furthermore, the representation provided by the feature-extraction component removes the requirement in previous work for explicitly introducing external stimulus currents, which may not be easily extracted from real-world datasets, as additional time series, thereby making the AE-ESN easier to apply to clinical data.

Evaluation of deep learning approaches for oil & gas pipeline leak detection using wireless sensor networks

Pipelines are one of the most common systems for storing and transporting petroleum products, both liquid and gaseous. Despite the durable structures, leakages can occur for many reasons, causing environmental disasters, energy waste, and, in some cases, human losses. The object of the ESTHISIS project is the development of a low-cost and low-energy wireless sensor system for the immediate detection of leaks in metallic piping systems for the transport of liquid and gaseous petroleum products in a noisy industrial environment. In this study, two distinct leakage detection methodologies are presented. First, a 2D-Convolutional Neural Network (CNN) model undertakes supervised classification in spectrograms extracted by the signals acquired by the accelerometers mounted on the pipeline wall. This approach allows us to supplant large-signal datasets with a more memory-efficient alternative to storing static images. The second methodology entails a Long Short-Term Memory Autoencoder (LSTM AE), which directly receives the signals from the accelerometers, providing an unsupervised leakage detection solution. Field tests for the validation of our methods were performed using an experimental pipeline network, while evaluation of their efficiency in a real environment was conducted in the premises of an oil refinery in Greece. Results evince the potency of the LSTM AE to recognize in real-time the emergence of deficiencies and the efficacy of the CNN models to classify accurately spectrograms reflecting the operational condition of the monitored pipelines.

Behavioral Change Prediction from Physiological Signals Using Deep Learned Features.

Predicting change from multivariate time series has relevant applications ranging from the medical to engineering fields. Multisensory stimulation therapy in patients with dementia aims to change the patient’s behavioral state. For example, patients who exhibit a baseline of agitation may be paced to change their behavioral state to relaxed. This study aimed to predict changes in one’s behavioral state from the analysis of the physiological and neurovegetative parameters to support the therapist during the stimulation session. In order to extract valuable indicators for predicting changes, both handcrafted and learned features were evaluated and compared. The handcrafted features were defined starting from the CATCH22 feature collection, while the learned ones were extracted using a temporal convolutional network, and the behavioral state was predicted through bidirectional long short-term memory auto-encoder, operating jointly. From the comparison with the state of the art, the learned features-based approach exhibits superior performance with accuracy rates of up to 99.42% with a time window of 70 seconds and up to 98.44% with a time window of 10 seconds.

Pattern Formation of the COVID-19 Pandemic Based on Lunar Calendar by Employing LSTM Auto Encoder

The number of detected COVID-19 cases for the states of Massachusetts, Colorado, and Nevada are taken from the Center for Disease Control and Prevention dashboard from the period of March 2019 to December 2019. The above-mentioned raw dataset is used for the present work as a nonlinear time series signal. Furthermore, the raw dataset is pre-processed by classifying the same into eight categories depending on the lunar calendar, which in turn is based on the octet phases of the moon. The susceptible-exposed-infectious-removed model is developed from this pre-processed data by employing the Long Short Term Memory auto encoder. The ensuing pattern formation from the auto encoder is investigated.

Deep learning model based on cascaded autoencoders and one‐class learning for detection and localization of anomalies from surveillance videos

Due to the need for increased security measures for monitoring and safeguarding the activities, video anomaly detection is considered as one of the significant research aspects in the domain of computer vision. Assigning human personnel to continuously check the surveillance videos for finding suspicious activities such as violence, robbery, wrong U-turns, to mention a few, is a laborious and error-prone task. It gives rise to the need for devising automated video surveillance systems ensuring security. Motivated by the same, this paper addresses the problem of detection and localization of anomalies from surveillance videos using pipelined deep autoencoders and one-class learning. Specifically, we used a convolutional autoencoder and a sequence-to-sequence long short-term memory autoencoder in a pipelined fashion for spatial and temporal learning of the videos, respectively. The authors followed the principle of one-class classification for training the model on normal data and testing it on anomalous testing data. The authors achieved a reasonably significant performance in terms of an equal error rate and the time required for anomaly detection and localization comparable to standard benchmarked approaches, thus, qualifies to work in a near-real-time manner for anomaly detection and localization.

LSTM-Autoencoder Deep Learning Technique for PAPR Reduction in Visible Light Communication

Visible light communication (VLC) is a relatively new wireless communication technology that allows for high data rate transfer. Because of its capability to enable high-speed transmission and eliminate inter-symbol interference, orthogonal frequency division multiplexing (OFDM) is widely employed in VLC. Peak to average power ratio (PAPR) is an issue that impacts the effectiveness of OFDM systems, particularly in VLC systems, because the signal is distorted by the nonlinearity of light-emitting diodes (LEDs). The proposed method Long Short Term Memory-Autoencoder (LSTM-AE) uses an autoencoder as well as an LSTM to learn a compact representation of an input, allowing the model to handle variable length input sequences as well as predict or produce variable length output sequences. This study compares the suggested model with various PAPR reduction strategies to demonstrate that it offers a superior improvement in PAPR reduction of the transmitted signal while maintaining BER. Also, this model provides a flexible compromisation between PAPR and BER.

Multi-task deep autoencoder to predict Alzheimer’s disease progression using temporal DNA methylation data in peripheral blood

Traditional approaches for diagnosing Alzheimer’s disease (AD) such as brain imaging and cerebrospinal fluid are invasive and expensive. It is desirable to develop a useful diagnostic tool by exploiting biomarkers obtained from peripheral tissues due to their noninvasive and easily accessible characteristics. However, the capacity of using DNA methylation data in peripheral blood for predicting AD progression is rarely known. It is also challenging to develop an efficient prediction model considering the complex and high-dimensional DNA methylation data in a longitudinal study. Here, we develop two multi-task deep autoencoders, which are based on the convolutional autoencoder and long short-term memory autoencoder to learn the compressed feature representation by jointly minimizing the reconstruction error and maximizing the prediction accuracy. By benchmarking on longitudinal DNA methylation data collected from the peripheral blood in Alzheimer’s Disease Neuroimaging Initiative, we demonstrate that the proposed multi-task deep autoencoders outperform state-of-the-art machine learning approaches for both predicting AD progression and reconstructing the temporal DNA methylation profiles. In addition, the proposed multi-task deep autoencoders can predict AD progression accurately using only the historical DNA methylation data and the performance is further improved by including all temporal DNA methylation data.Availability::https://github.com/lichen-lab/MTAE.

A machine learning pipeline for fuel-economical driving model

PurposeA cost-effective way to achieve fuel economy is to reinforce positive driving behaviour. Driving behaviour can be controlled if drivers can be alerted for behaviour that results in poor fuel economy. Fuel consumption must be tracked and monitored instantaneously rather than tracking average fuel economy for the entire trip duration. A single-step application of machine learning (ML) is not sufficient to model prediction of instantaneous fuel consumption and detection of anomalous fuel economy. The study designs an ML pipeline to track and monitor instantaneous fuel economy and detect anomalies.Design/methodology/approachThis research iteratively applies different variations of a two-step ML pipeline to the driving dataset for hatchback cars. The first step addresses the problem of accurate measurement and prediction of fuel economy using time series driving data, and the second step detects abnormal fuel economy in relation to contextual information. Long short-term memory autoencoder method learns and uses the most salient features of time series data to build a regression model. The contextual anomaly is detected by following two approaches, kernel quantile estimator and one-class support vector machine. The kernel quantile estimator sets dynamic threshold for detecting anomalous behaviour. Any error beyond a threshold is classified as an anomaly. The one-class support vector machine learns training error pattern and applies the model to test data for anomaly detection. The two-step ML pipeline is further modified by replacing long short term memory autoencoder with gated recurrent network autoencoder, and the performance of both models is compared. The speed recommendations and feedback are issued to the driver based on detected anomalies for controlling aggressive behaviour.FindingsA composite long short-term memory autoencoder was compared with gated recurrent unit autoencoder. Both models achieve prediction accuracy within a range of 98%–100% for prediction as a first step. Recall and accuracy metrics for anomaly detection using kernel quantile estimator remains within 98%–100%, whereas the one-class support vector machine approach performs within the range of 99.3%–100%.Research limitations/implicationsThe proposed approach does not consider socio-demographics or physiological information of drivers due to privacy concerns. However, it can be extended to correlate driver's physiological state such as fatigue, sleep and stress to correlate with driving behaviour and fuel economy. The anomaly detection approach here is limited to providing feedback to driver, it can be extended to give contextual feedback to the steering controller or throttle controller. In the future, a controller-based system can be associated with an anomaly detection approach to control the acceleration and braking action of the driver.Practical implicationsThe suggested approach is helpful in monitoring and reinforcing fuel-economical driving behaviour among fleet drivers as per different environmental contexts. It can also be used as a training tool for improving driving efficiency for new drivers. It keeps drivers engaged positively by issuing a relevant warning for significant contextual anomalies and avoids issuing a warning for minor operational errors.Originality/valueThis paper contributes to the existing literature by providing an ML pipeline approach to track and monitor instantaneous fuel economy rather than relying on average fuel economy values. The approach is further extended to detect contextual driving behaviour anomalies and optimises fuel economy. The main contributions for this approach are as follows: (1) a prediction model is applied to fine-grained time series driving data to predict instantaneous fuel consumption. (2) Anomalous fuel economy is detected by comparing prediction error against a threshold and analysing error patterns based on contextual information.

Data-Driven Anomaly Detection in High-Voltage Transformer Bushings with LSTM Auto-Encoder.

The reliability and health of bushings in high-voltage (HV) power transformers is essential in the power supply industry, as any unexpected failure can cause power outage leading to heavy financial losses. The challenge is to identify the point at which insulation deterioration puts the bushing at an unacceptable risk of failure. By monitoring relevant measurements we can trace any change that occurs and may indicate an anomaly in the equipment’s condition. In this work we propose a machine-learning-based method for real-time anomaly detection in current magnitude and phase angle from three bushing taps. The proposed method is fast, self-supervised and flexible. It consists of a Long Short-Term Memory Auto-Encoder (LSTMAE) network which learns the normal current and phase measurements of the bushing and detects any point when these measurements change based on the Mean Absolute Error (MAE) metric evaluation. This approach was successfully evaluated using real-world data measured from HV transformer bushings where anomalous events have been identified.

A maintenance hemodialysis mortality prediction model based on anomaly detection using longitudinal hemodialysis data

BackgroundMost end-stage renal disease patients rely on hemodialysis (HD) to maintain their life, and they face a serious financial burden and high risk of mortality. Due to the current situation of the health care system in China, a large number of patients on HD are lost to follow-up, making the identification of patients with high mortality risks an intractable problem. ObjectiveThis paper aims to propose a maintenance HD mortality prediction approach using longitudinal HD data under the situation of data imbalance caused by follow-up losses. MethodsA long short-term memory autoencoder (LSTM AE) based model is proposed to capture the physical condition changes of HD patients and distinguish between surviving and nonsurviving patients. The approach adopts anomaly detection theory, using only the surviving samples in the model training and identifying dead samples based on autoencoder reconstruction errors. The data are from a Chinese hospital electronic health record system between July 30, 2007, and August 25, 2016, and 36/72/108 continuous HD sessions were used to predict mortality within prediction windows of 90/180/365 days. Furthermore, the model performance is compared to that of logistic regression, support vector machine, random forest, LSTM classifier, isolation forest, and stacked autoencoder models. ResultsData for 1200 patients (survival: 1055, death: 145) were used to predict mortality during the next 90 days using 36 continuous HD sessions. The area under the PR curve for the LSTM AE was 0.57, the Recallmacro was 0.86, and the F1-scoremacro was 0.87, outperforming the other models. Upon varying the observation window or prediction window length, LSTM AE continued to outperform the other models. According to the variable importance analysis, the dialysis session length was the feature that contributed the most to the prediction model. ConclusionsThe proposed approach was able to detect patients on maintenance HD with high mortality risk from an imbalanced dataset using anomaly detection theory and leveraging longitudinal HD data.

Estimation of Frequency-Dependent Impedances in Power Grids by Deep LSTM Autoencoder and Random Forest

This paper proposes a deep-learning-based method for frequency-dependent grid impedance estimation. Through measurement of voltages and currents at a specific system bus, the estimate of the grid impedance was obtained by first extracting the sequences of the time-dependent features for the measured data using a long short-term memory autoencoder (LSTM-AE) followed by a random forest (RF) regression method to find the nonlinear map function between extracted features and the corresponding grid impedance for a wide range of frequencies. The method was trained via simulation by using time-series measurements (i.e., voltage and current) for different system parameters and verified through several case studies. The obtained results show that: (1) extracting the time-dependent features of the voltage/current data improves the performance of the RF regression method; (2) the RF regression method is robust and allows grid impedance estimation within 1.5 grid cycles; (3) the proposed method can effectively estimate the grid impedance both in steady state and in case of large transients like electrical faults.

Memory-Efficient Deep Learning for Botnet Attack Detection in IoT Networks

Cyber attackers exploit a network of compromised computing devices, known as a botnet, to attack Internet-of-Things (IoT) networks. Recent research works have recommended the use of Deep Recurrent Neural Network (DRNN) for botnet attack detection in IoT networks. However, for high feature dimensionality in the training data, high network bandwidth and a large memory space will be needed to transmit and store the data, respectively in IoT back-end server or cloud platform for Deep Learning (DL). Furthermore, given highly imbalanced network traffic data, the DRNN model produces low classification performance in minority classes. In this paper, we exploit the joint advantages of Long Short-Term Memory Autoencoder (LAE), Synthetic Minority Oversampling Technique (SMOTE), and DRNN to develop a memory-efficient DL method, named LS-DRNN. The effectiveness of this method is evaluated with the Bot-IoT dataset. Results show that the LAE method reduced the dimensionality of network traffic features in the training set from 37 to 10, and this consequently reduced the memory space required for data storage by 86.49%. SMOTE method helped the LS-DRNN model to achieve high classification performance in minority classes, and the overall detection rate increased by 10.94%. Furthermore, the LS-DRNN model outperformed state-of-the-art models.

Reducing balancing cost of a wind power plant by deep learning in market data: A case study for Turkey

By the liberalization of energy markets, renewable energy producers are increasingly selling their electricity in the day-ahead market. However, day-ahead forecasts of wind generators are not sufficiently accurate and therefore they are exposed to an imbalance cost due to the incorrect offerings. Although extensive and detailed market data are constantly publicized by the market operator, historical market data are not utilized effectively to reduce this cost. The present study initially casts the imbalance cost reducing problem as a binary classification problem and constructs a framework that consists of a long short term memory autoencoder and a blend of advanced classifiers. Then, the method extracts information from the market data if the day-ahead or imbalance price will be higher at a given hour of the next day. Using this information, auxiliary algorithms alter existing production forecasts and prevents abrupt rises in the imbalance cost. Extensive tests throughout a year show that the strategy performs reliably well and it has provided between 6.258% and 11.195% decrease in the balancing cost for four tested wind power plants.

Using Deep Autoencoders for In-vehicle Audio Anomaly Detection

Current developments on self-driving cars have increased the interest on autonomous shared taxicabs. While most self-driving technologies focus on the outside environment, there is also a need to provide in-vehicle intelligence (e.g., detect health and safety issues related with the car occupants). Set within an R&D project focused on in-vehicle cockpit intelligence, the research presented in this paper addresses an unsupervised Acoustic Anomaly Detection (AAD) task. Since data is nonexistent in this domain, we first design an in-vehicle sound event data simulator that can realistically mix background audios (recorded from car driving trips) with normal (e.g., people talking, radio on) and abnormal (e.g., people arguing, cough) event sounds, allowing the generation of three synthetic in-vehicle sound datasets. Then, we explore two main sound feature extraction methods (based on a combination of three audio features and mel frequency energy coefficients) and propose a novel Long Short-Term Memory Autoencoder (LSTM-AE) deep learning architecture for in-vehicle sound anomaly detection. Competitive results were achieved by the proposed LSTM-AE when compared with two state-of-the-art methods, namely a dense Autoencoder (AE) and a two-stage clustering.

Deep Learning Approach for Detecting Malicious Activities Over Encrypted Secure Channels

Nowadays, most cyber attackers exploit secure communication channels to hide malicious activities and imitate the behaviors of a legitimate user. These attacks over a secure channel make networked systems more vulnerable to new threats and increase the possibility of significant damage to other end users. Traditional TCP/IP-level traffic inspections do not suffice in investigating a secure sockets layer (SSL) conversation because the SSL conversation data is encrypted by a public key system and the SSL uses its own data unit of an SSL record. In this paper, we propose a novel malicious SSL traffic detection method, which reassembles SSL records from captured IP packets and inspects the characteristics of SSL records using a deep learning method. After an SSL record is reassembled from a single or multiple IP packets, the proposed method extracts unencrypted contents of the reassembled record and generates a sequence of unencrypted data from successive SSL records for deep learning-based classification. The sequences of SSL records are encoded using a long short-term memory autoencoder, and then an encoded feature map is generated for each SSL flow. These feature maps are forwarded to the convolutional neural network-based classifier to determine whether the SSL flow is malicious or not. The experiment shows that our proposed approach has a great separability between benign and malicious traffic flows on an encrypted SSL channel.

Convolutional Long Short-Term Memory Autoencoder-Based Feature Learning for Fault Detection in Industrial Processes

Process signals show the characteristics of large scale, high dimension, and strong correlation in modern industrial processes, which brings a big challenge for process fault detection and diagnosis. Due to the powerful feature learning ability, deep learning has been widely used in image and visual processing. This article proposes a new deep neural network (DNN), convolutional long short-term memory autoencoder (CLSTM-AE) for feature learning from process signals. The convolutional LSTM (ConvLSTM) is proposed to describe the distribution of the process data and learn effective features on time series data for fault detection. A selective residual block is embedded in the deep network to improve the training accuracy and perform feature selection from process signals. Two statistics, the $T^{2}$ and the squared prediction error (SPE), are generated in the feature space and residual space of CLSTM-AE, respectively. Finally, the feasibility and advantages of CLSTM-AE are shown on a simulated process, the Tennessee-Eastman process (TEP), and the continuous stirred tank reactor (CSTR). CLSTM-AE has good fault detection performance in these cases, which shows that it is capable of learning effective features from complex process signals. The hybrid learning technique with convolutional LSTM and autoencoder provides a new way for feature learning and fault detection for complex industrial processes.