LSTM Autoencoder Research Articles

Sentence embedding approach using LSTM auto-encoder for discussion threads summarization

Online discussion forums are repositories of valuable information where users interact and articulate their ideas, opinions, and share experiences about nu merous topics. They are internet-based online communities where users can ask for help and find the solution to a problem. On online discussion forums, a new user becomes exhausted from reading the significant number of replies in a discussion. An automated discussion thread summarizing system (DTS) is necessary to create a candid view of the entire discussion of a query. Most of the previous approaches for automated DTS use the continuous bag of words (CBOW) model as a sentence embedding tool, which is poor at capturing the overall meaning of the sentence and is unable to grasp word dependency. To overcome this limitation, we introduce the LSTM Auto-encoder as a sentence embedding technique to improve the per formance of DTS. The empirical result in the context of average precision, recall, and F-measure of the proposed approach with respect to ROGUE-1 and ROUGE-2 of two standard experimental datasets proves the effectiveness and efficiency of the proposed approach and outperforms the state-of-the-art CBOW model in sentence embedding tasks by boosting the performance of the automated DTS model.

AN UNSUPERVISED MALWARE DETECTION SYSTEM FOR WINDOWS BASED SYSTEM CALL SEQUENCES

Malware attacks have grown in prominence in recent years, posing severe security risks and resulting in significant financial losses. The ability to rapidly and reliably classify malware is vital to cybersecurity due to the exponential growth of malware variants. The role of artificial intelligence plays a significant role in cybersecurity industry. Recently, in the field of malware detection deep learning technique seeks more attention than the machine learning techniques due to the complexity of its behavior. Because the deep learning technique performs well than the machine learning techniques in terms of accuracy and it is well suited for large amount of data. The input attribute for the proposed model is windows-based system call sequence which is collected from NT mal detect project. In this work, the unsupervised deep learning technique used for text classification namely LSTM autoencoder and the performance of proposed model compares with existing DL methods such as CNN, RNN and LSTM with the performance parameters of accuracy, precision, recall and F1-measure.

Characterizing residential load patterns on multi-time scales utilizing LSTM autoencoder and electricity consumption data

• We propose a novel framework to characterize residential load patterns utilizing LSTM-AE and electricity consumption data. • LSTM-AE model is designed for dimensionality reduction and feature extraction. • Two-level clustering method is proposed to discover TLPs and MLPs on multi-time scales. • TLPs profile resident's load patterns from a global view, MLPs characterize load patterns of individual and groups. • Consumer groups and residential energy lifestyles are revealed to customize personal demand response strategies. Load patterns represent a clear picture of electricity usage, reflecting the consumer's habits. Previous works mainly focused on load patterns discovery on a fixed scale, but limited to characterize load patterns on multi-time scales utilizing electricity consumption data (ECD). Therefore, we propose a novel framework to characterize residential load patterns on multi-time scales. The long-short-term memory autoencoder (LSTM-AE) model is designed for dimensionality reduction and feature extraction. Furthermore, a two-level clustering method is proposed to discover and characterize typical load patterns (TLPs) and multifaceted load patterns (MLPs) on multi-time scales. The proposed framework is comprehensively evaluated via extensive experiments on three real ECD. Results show that: (1) Reconstruction errors of LSTM-AE are lower than 6 benchmark models across different time scales, which validates the superiority of LSTM-AE. (2) TLPs and MLPs on daily, weekly, monthly and yearly scale are discovered by the two-level clustering method. TLPs profile the resident's electricity usages from a global view. (3) MLPs present the consumer segmentation and characterize residential load patterns of individual and groups. Especially, customer groups and electricity usage habits or lifestyles are revealed thoroughly to customize personal demand response strategies. This study can provide new valuable insights for smart grid applications.

ADRIoT: An Edge-Assisted Anomaly Detection Framework Against IoT-Based Network Attacks

Internet of Things (IoT) has entered a stage of rapid development and increasing deployment. Meanwhile, these low-power devices typically cannot support complex security mechanisms and, thus, are highly susceptible to malware. This article proposes ADRIoT, an anomaly detection framework for IoT networks, which leverages edge computing to uncover potential threats. An edge is empowered with an anomaly detection module, which consists of a traffic capturer, a traffic preprocessor, and a collection of anomaly detectors dedicated to each type of device. Each detector is constructed by an LSTM autoencoder in an unsupervised manner that requires no labeled attack data and is able to handle emerging zero-day attacks. When a device connects to the edge, the edge will fetch the corresponding detector from the cloud and execute it locally. Another problem is the resource constraint of a single edge device like a home router hinders the deployment of such a detection module. To mitigate this problem, we design a multiedge collaborative mechanism that integrates the resource of multiple edges in a local network to increase the overall load capacity. The evaluation demonstrates that ADRIoT can detect various IoT-based attacks effectively and efficiently, showing that ADRIoT can feasibly help build a more secure IoT environment.

Anomaly detection in rotating machinery using autoencoders based on bidirectional LSTM and GRU neural networks

A time series anomaly is a form of anomalous subsequence that indicates future faults will occur. The development of novel techniques for detecting this type of anomaly is significant for real-time system monitoring. Several algorithms have been used to classify anomalies successfully. However, the time series anomaly detection algorithm was not studied well. We use a new bidirectional LSTM and GRU neural networks-based hybrid autoencoder to detect if a machine is operating normally in this research. An autoencoder is trained on a set of 12 features taken from healthy operating data gathered promptly after a planned maintenance period using vibration sensors. The features taken from new data are then reconstructed using the trained model. If the model accurately reconstructs the features, the machine is in good working order. If the reconstruction exceeds a certain error threshold, the machine is functioning strangely and needs to be serviced.

Pattern Formation of the COVID-19 Pandemic Based on Lunar Calendar by Employing LSTM Auto Encoder

The number of detected COVID-19 cases for the states of Massachusetts, Colorado, and Nevada are taken from the Center for Disease Control and Prevention dashboard from the period of March 2019 to December 2019. The above-mentioned raw dataset is used for the present work as a nonlinear time series signal. Furthermore, the raw dataset is pre-processed by classifying the same into eight categories depending on the lunar calendar, which in turn is based on the octet phases of the moon. The susceptible-exposed-infectious-removed model is developed from this pre-processed data by employing the Long Short Term Memory auto encoder. The ensuing pattern formation from the auto encoder is investigated.

Improved Data-Driven Root Cause Analysis in a Fog Computing Environment

Internet of things (IoT) and cloud computing are used in many real-time smart applications such as smart healthcare, smart traffic, smart city, and smart industries. Fog computing has been introduced as an intermediate layer to reduce communication delay between cloud and IoT devices. To improve the performance of these smart applications, a predictive maintenance system needs to adopt an anomaly detection and root cause analysis model that helps to resolve anomalies and avoid such anomalies in the future. The state-of-the-art work on data-driven root cause analysis suffers from scalability, accuracy, and interpretability. In this paper, a multi-agent-based improved data-driven root cause analysis technique is introduced to identify anomalies and their root causes. The deep learning model LSTM autoencoder is used to find the anomalies, and a game theory approach called SHAP algorithm is used to find the root cause of the anomaly. The evaluation result shows the improvement in accuracy and interpretability as compared to state-of-the-art works.

Pollutant specific optimal deep learning and statistical model building for air quality forecasting

Poor air quality is becoming a critical environmental concern in different countries over the last several years. Most of the air pollutants have serious consequences on human health and wellbeing. In this context, efficient forecasting of air pollutants is currently crucial to predict future events with a view to taking corrective actions and framing effective environmental policies. Although deep learning (DL) as well as statistical forecasting models are investigated in the literature, they have rarely used in air pollutant-specific optimal model building for long-term forecasting. In this paper, our aim is to develop the pollutant-specific optimal forecasting models for the phases spanning from preprocessing to model building by investigating a set of predictive techniques. In this regard, this paper presents a methodology for long-term forecasting of some important air pollutants. More specifically, a total of eight best performing models such as stacked LSTM, LSTM auto-encoder, Bi-LSTM, convLSTM, Holt-Winters, auto-regressive (AR), SARIMA, and Prophet are investigated for developing pollutant-specific optimal forecasting models. The study is carried out based on the real-world data obtained from government-run air quality monitoring units in Kolkata over a period of 4 years. The models such as Holt-Winters, Bi-LSTM, and ConvLSTM achieve high forecasting accuracy with respect to MAE and RMSE values for majority of the pollutants.

Estimate remaining useful life for predictive railways maintenance based on LSTM autoencoder

Estimate remaining useful life for predictive railways maintenance based on LSTM autoencoder

Real-Time Radio Technology and Modulation Classification via an LSTM Auto-Encoder

Identification of the type of communication technology and/or modulation scheme based on detected radio signal are challenging problems encountered in a variety of applications including spectrum allocation and radio interference mitigation. They are rendered difficult due to a growing number of emitter types and varied effects of real-world channels upon the radio signal. Existing spectrum monitoring techniques are capable of acquiring massive amounts of radio and real-time spectrum data using compact sensors deployed in a variety of settings. However, state-of-the-art methods that use such data to classify emitter types and detect communication schemes struggle to achieve required levels of accuracy at a computational efficiency that would allow their implementation on low-cost computational platforms. In this paper, we present a learning framework based on an LSTM denoising auto-encoder designed to automatically extract stable and robust features from noisy radio signals, and infer modulation or technology type using the learned features. The algorithm utilizes a compact neural network architecture readily implemented on a low-cost computational platform while exceeding state-of-the-art accuracy. Results on realistic synthetic as well as over-the-air radio data demonstrate that the proposed framework reliably and efficiently classifies received radio signals, often demonstrating superior performance compared to state-of-the-art methods. Source codes are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/WuLoli/LSTMDAE</uri> .

A Knowledge Distillation Ensemble Framework for Predicting Short- and Long-Term Hospitalization Outcomes From Electronic Health Records Data.

The ability to perform accurate prognosis is crucial for proactive clinical decision making, informed resource management and personalised care. Existing outcome prediction models suffer from a low recall of infrequent positive outcomes. We present a highly-scalable and robust machine learning framework to automatically predict adversity represented by mortality and ICU admission and readmission from time-series of vital signs and laboratory results obtained within the first 24 hours of hospital admission. The stacked ensemble platform comprises two components: a) an unsupervised LSTM Autoencoder that learns an optimal representation of the time-series, using it to differentiate the less frequent patterns which conclude with an adverse event from the majority patterns that do not, and b) a gradient boosting model, which relies on the constructed representation to refine prediction by incorporating static features. The model is used to assess a patient's risk of adversity and provides visual justifications of its prediction. Results of three case studies show that the model outperforms existing platforms in ICU and general ward settings, achieving average Precision-Recall Areas Under the Curve (PR-AUCs) of 0.891 (95% CI: 0.878-0.939) for mortality and 0.908 (95% CI: 0.870-0.935) in predicting ICU admission and readmission.

RLF-LPI: An ensemble learning framework using sequence information for predicting lncRNA-protein interaction based on AE-ResLSTM and fuzzy decision.

Long non-coding RNAs (lncRNAs) play a regulatory role in many biological cells, and the recognition of lncRNA-protein interactions is helpful to reveal the functional mechanism of lncRNAs. Identification of lncRNA-protein interaction by biological techniques is costly and time-consuming. Here, an ensemble learning framework, RLF-LPI is proposed, to predict lncRNA-protein interactions. The RLF-LPI of the residual LSTM autoencoder module with fusion attention mechanism can extract the potential representation of features and capture the dependencies between sequences and structures by k-mer method. Finally, the relationship between lncRNA and protein is learned through the method of fuzzy decision. The experimental results show that the ACC of RLF-LPI is 0.912 on ATH948 dataset and 0.921 on ZEA22133 dataset. Thus, it is demonstrated that our proposed method performed better in predicting lncRNA-protein interaction than other methods.

Video Risk Detection and Localization using Bidirectional LSTM Autoencoder and Faster R-CNN

Video Risk Detection and Localization using Bidirectional LSTM Autoencoder and Faster R-CNN

Data-Driven Anomaly Detection in High-Voltage Transformer Bushings with LSTM Auto-Encoder.

The reliability and health of bushings in high-voltage (HV) power transformers is essential in the power supply industry, as any unexpected failure can cause power outage leading to heavy financial losses. The challenge is to identify the point at which insulation deterioration puts the bushing at an unacceptable risk of failure. By monitoring relevant measurements we can trace any change that occurs and may indicate an anomaly in the equipment’s condition. In this work we propose a machine-learning-based method for real-time anomaly detection in current magnitude and phase angle from three bushing taps. The proposed method is fast, self-supervised and flexible. It consists of a Long Short-Term Memory Auto-Encoder (LSTMAE) network which learns the normal current and phase measurements of the bushing and detects any point when these measurements change based on the Mean Absolute Error (MAE) metric evaluation. This approach was successfully evaluated using real-world data measured from HV transformer bushings where anomalous events have been identified.

Arrhythmia classification of LSTM autoencoder based on time series anomaly detection

Electrocardiogram (ECG) is widely used in the diagnosis of heart disease because of its noninvasiveness and simplicity. The time series signals contained in the signal are usually obtained by the professional medical staff and used for the classification of heartbeat diagnosis. Professional physicians can use the electrocardiogram to know whether the patient has serious congenital heart disease and whether there is an abnormal heart structure. A lot of work has been done to achieve automatic classification of arrhythmia types. For example, Autoencoder can obtain the time series characteristics of ECG signals and be used for ECG signal classification. However, some traditional methods are abstruse and difficult to understand in principle. In the classification of arrhythmias carried out in recent years, some researchers only use Autoencoder to provide structural characteristics, without giving too much explanation to the design reasons. Therefore, we optimized a new network layer design based on LSTM to obtain the autoencoder structure. This structure can cooperate with the ECG preprocessing process designed by us to obtain better arrhythmia classification effect. This method enables direct input of ECG signals into the model without complicated preprocessing such as manual parameter input. Also, it eliminates the gradient vanishing problem existing in traditional convolutional neural network. We used five different types of ECG data in MIT-BIH arrhythmia database and MIT-BIH supraventricular arrhythmia database: atrial premature beats (APB), left bundle branch block (LBBB), normal heartbeat (NSR), right bundle branch block (RBBB) and ventricular premature beats (PVC). High accuracy, precision and recall were obtained. Compared with traditional methods, this method has better performance in arrhythmia classification.

Estimation of Frequency-Dependent Impedances in Power Grids by Deep LSTM Autoencoder and Random Forest

This paper proposes a deep-learning-based method for frequency-dependent grid impedance estimation. Through measurement of voltages and currents at a specific system bus, the estimate of the grid impedance was obtained by first extracting the sequences of the time-dependent features for the measured data using a long short-term memory autoencoder (LSTM-AE) followed by a random forest (RF) regression method to find the nonlinear map function between extracted features and the corresponding grid impedance for a wide range of frequencies. The method was trained via simulation by using time-series measurements (i.e., voltage and current) for different system parameters and verified through several case studies. The obtained results show that: (1) extracting the time-dependent features of the voltage/current data improves the performance of the RF regression method; (2) the RF regression method is robust and allows grid impedance estimation within 1.5 grid cycles; (3) the proposed method can effectively estimate the grid impedance both in steady state and in case of large transients like electrical faults.

Multi-label LSTM autoencoder for non-intrusive appliance load monitoring

This work follows the multi-label classification based paradigm for non-intrusive load monitoring (NILM). Power consumption signals used for NILM are inherently time varying. However prior multi-label classification techniques could not model this dynamical behaviour. They used off-the-shelf algorithms for classifying static signals on NILM problems. This is the first work that shows how to account for the temporal variability of input signals in a multi-label classification framework. Results on benchmark datasets like REDD and Pecan Street show considerable improvement over several state-of-the-art shallow and deep techniques.

기상정보 예측 성능 개선을 위한 빅데이터 병합 및 딥러닝 모델 최적화

급속한 산업화의 영향으로 많은 형태의 대기오염이 증가하고 있으며, 특히 미세먼지는 심장 및 폐 관련 질환을 발생시키거나 악화시키는 등 인체에 악영향을 끼친다. 본 연구에서는 미세먼지 피해를 줄이기 위한 해결방안으로 사전에 대응할 수 있도록 8시간 후의 미세먼지 수치를 예측한다. 우리는 정확한 예측을 위해 다양한 dataset을 구성하였고 시계열 데이터 예측에서 좋은 성능을 보여주는 LSTM 계열의 모델인 Stacked LSTM, LSTM AutoEncoder, LSTM Variational AutoEncoder를 이용한다. 연구 결과 데이터 수의 증가는 모델의 성능을 향상시키지 못했으나 feature 수의 증가는 모델의 성능을 향상시켰다. 또한, feature 수가 많은 dataset일수록 LSTM AutoEncoder가 가장 좋은 성능을 보였다.

Anomaly detection in Solar radiation forecasting using LSTM autoencoder architecture

Anomaly detection in Solar radiation forecasting using LSTM autoencoder architecture

Bilingual Cyber-aggression detection on social media using LSTM autoencoder

Cyber-aggression is an offensive behaviour attacking people based on race, ethnicity, religion, gender, sexual orientation and other traits. It has become a major issue plaguing the online social media. In this research, we have developed a deep learning-based model to identify different levels of aggression (direct, indirect and no aggression) in a social media post in a bilingual scenario. The model is an autoencoder built using the LSTM network and trained with non-aggressive comments only. Any aggressive comment (direct or indirect) will be regarded as an anomaly to the system and will be marked as Overtly (direct) or Covertly (indirect) aggressive comment depending on the reconstruction loss by the autoencoder. The validation results on the dataset from two popular social media sites: Facebook and Twitter with bilingual (English and Hindi) data outperformed the current state-of-the-art models with improvements of more than 11% on the test sets of the English dataset and more than 6% on the test sets of the Hindi dataset.