Long Short Time Memory Research Articles

A Predictive Model Using Long Short-Time Memory (LSTM) Technique for Power System Voltage Stability

The stability of the operation of the power system is essential to ensure a continuous supply of electricity to meet the load of the system. In the operational process, voltage stability (VS) should be recognized and predicted as a basic requirement. In electrical systems, deep learning and machine learning algorithms have found widespread applications. These algorithms can learn from previous data to detect and predict future scenarios of potential instability. This study introduces long short-term memory (LSTM) technology to predict the stability of the nominal voltage of the power system. Based on the results, the recommended LSTM technology achieved the highest accuracy target of 99.5%. In addition, the LSTM model outperforms other machine learning (ML) and deep learning techniques, i.e., support vector machines (SVMs), Naive Bayes (NB), and convolutional neural networks (CNNs), when comparing the accuracy of the VS forecast. The results show that the LSTM method is useful to predict the voltage of an electrical system. The IEEE 33-bus system indicates that the recommended approach can rapidly and precisely verify the system stability category. Furthermore, the proposed method outperforms conventional assessment methods that rely on shallow learning.

A wave forecasting method based on probabilistic diffusion LSTM network for model predictive control of wave energy converters

Accurate long-term prediction of wave excitation forces is critical for optimizing wave energy converters, enabling enhanced control, and improving energy absorption efficiency. Traditional prediction models often employ deterministic conservative approaches, overlooking uncertainties. This paper introduces a novel prediction method based on the probabilistic diffusion model, offering a more precise prediction while accounting for uncertainty. Notably, our approach goes beyond previous works by simultaneously achieving control objectives for maximizing energy absorption, contrasting with methodologies solely focused on prediction and control tasks. The proposed method utilizes long short-time memory to extract temporal information from historical wave excitation observation. The hidden features are then processed through a diffused probabilistic-based unit. Subsequently, a time-scale neural network is developed for wave excitation moment prediction, followed by the application of nonlinear model predictive control to maximize the energy absorption. The methodology is validated on the 1:20 Wavestar prototype devices through numerical simulations. Results indicate that the predicted excitation moment closely align with measured values. In cases 4, 5, 6, the proposed method yields a substantial improvement in maximum control energy absorption–22%, 16%, and 31%, respectively–compared to traditional prediction methods. The proposed approach not only achieves a more accurate wave excitation moment prediction with uncertainty consideration but also introduces advanced control strategies in a full-scale plant application. This work contributes significantly to the field by bridging the gap between precise prediction and effective control in wave energy conversion systems.

A Novel Deep Learning Approach for Forecasting Myocardial Infarction Occurrences with Time Series Patient Data.

Myocardial Infarction (MI) commonly referred to as a heart attack, results from the abrupt obstruction of blood supply to a section of the heart muscle, leading to the deterioration or death of the affected tissue due to a lack of oxygen. MI, poses a significant public health concern worldwide, particularly affecting the citizens of the Chittagong Metropolitan Area. The challenges lie in both prevention and treatment, as the emergence of MI has inflicted considerable suffering among residents. Early warning systems are crucial for managing epidemics promptly, especially given the escalating disease burden in older populations and the complexities of assessing present and future demands. The primary objective of this study is to forecast MI incidence early using a deep learning model, predicting the prevalence of heart attacks in patients. Our approach involves a novel dataset collected from daily heart attack incidence Time Series Patient Data spanning January 1, 2020, to December 31, 2021, in the Chittagong Metropolitan Area. Initially, we applied various advanced models, including Autoregressive Integrated Moving Average (ARIMA), Error-Trend-Seasonal (ETS), Trigonometric seasonality, Box-Cox transformation, ARMA errors, Trend and Seasonal (TBATS), and Long Short Time Memory (LSTM). To enhance prediction accuracy, we propose a novel Myocardial Sequence Classification (MSC)-LSTM method tailored to forecast heart attack occurrences in patients using the newly collected data from the Chittagong Metropolitan Area. Comprehensive results comparisons reveal that the novel MSC-LSTM model outperforms other applied models in terms of performance, achieving a minimum Mean Percentage Error (MPE) score of 1.6477. This research aids in predicting the likely future course of heart attack occurrences, facilitating the development of thorough plans for future preventive measures. The forecasting of MI occurrences contributes to effective resource allocation, capacity planning, policy creation, budgeting, public awareness, research identification, quality improvement, and disaster preparedness.

Applying LSTM and GRU Methods to Recognize and Interpret Hand Gestures, Poses, and Face-Based Sign Language in Real Time

This research uses a real-time, human-computer interaction application to examine sign language recognition. This work develops a rule-based hand gesture approach for Indonesian sign language in order to interpret some words using a combination of hand movements, mimics, and poses. The main objective in this study is the recognition of sign language that is based on hand movements made in front of the body with one or two hands, movements which may involve switching between the left and right hand or may be combined with mimics and poses. To overcome this problem, a research framework is developed by coordinating hand gestures with poses and mimics to create features by using holistic MediaPipe. To train and test data in real time, the long short time memory (LSTM) and gated recurrent unit (GRU) approaches are used. The research findings presented in this paper show that hand gestures in real-time interactions are reliably recognized, and some words are interpreted with the high accuracy rates of 94% and 96% for the LSTM and GRU methods, respectively.

Abnormal data detection and recovery of sensors network based on spatiotemporal deep learning methodology

This paper proposes a novel deep-learning-enabled framework for abnormal data detection and recovery considering spatial-temporal correlation among sensors. The wavelet scattering transform is adopted to replace the first layer of one-dimension convolutional neural network classification model. The multiple typical sensor failure modes of sensor are identified including bias, drifting, gain, precision degradation, and three complete faults (constant; constant + noise; noise). The bidirectional long-short time memory (BLSTM) network is developed to recover abnormal data through positive and negative propagation. The BLSTM cell involving forget, input, and output gate is utilized to adaptively transmit valuable information of long-term data. Different sensor configurations are conducted to select the optimal input sensors of BLSTM model. The proposed models are then validated by three engineering cases. The results show that the failure mode diagnosis and recovery accuracy are over 90 %. Sensor selection has a large impact on the data reconstruction, and selecting sensors with higher correlation can achieve a higher accuracy. In addition, the data recovery accuracy of the proposed BLSTM model surpasses that of traditional long-short time memory models.

An automatic method using MFCC features for sleep stage classification

Sleep stage classification is a necessary step for diagnosing sleep disorders. Generally, experts use traditional methods based on every 30 seconds (s) of the biological signals, such as electrooculograms (EOGs), electrocardiograms (ECGs), electromyograms (EMGs), and electroencephalograms (EEGs), to classify sleep stages. Recently, various state-of-the-art approaches based on a deep learning model have been demonstrated to have efficient and accurate outcomes in sleep stage classification. In this paper, a novel deep convolutional neural network (CNN) combined with a long short-time memory (LSTM) model is proposed for sleep scoring tasks. A key frequency domain feature named Mel-frequency Cepstral Coefficient (MFCC) is extracted from EEG and EMG signals. The proposed method can learn features from frequency domains on different bio-signal channels. It firstly extracts the MFCC features from multi-channel signals, and then inputs them to several convolutional layers and an LSTM layer. Secondly, the learned representations are fed to a fully connected layer and a softmax classifier for sleep stage classification. The experiments are conducted on two widely used sleep datasets, Sleep Heart Health Study (SHHS) and Vincent’s University Hospital/University College Dublin Sleep Apnoea (UCDDB) to test the effectiveness of the method. The results of this study indicate that the model can perform well in the classification of sleep stages using the features of the 2-dimensional (2D) MFCC feature. The advantage of using the feature is that it can be used to input a two-dimensional data stream, which can be used to retain information about each sleep stage. Using 2D data streams can reduce the time it takes to retrieve the data from the one-dimensional stream. Another advantage of this method is that it eliminates the need for deep layers, which can help improve the performance of the model. For instance, by reducing the number of layers, our seven layers of the model structure takes around 400 s to train and test 100 subjects in the SHHS1 dataset. Its best accuracy and Cohen’s kappa are 82.35% and 0.75 for the SHHS dataset, and 73.07% and 0.63 for the UCDDB dataset, respectively.

Lifetime Improvement of 28 nm Resistive Random Access Memory Chip by Machine Learning‐Assisted Prediction Model Collaborated with Resurrection Algorithm

AbstractIn this work, a machine learning‐assisted prediction model is proposed to analyze the reliability issues in the 28 nm resistive random access memory (RRAM) chip with raw data measured from RRAM test chip. The neural network of long‐short time memory (LSTM) is trained by the voltages and resistance during the endurance test (input vectors) and generates the output of the dichotomy states with a satisfied testing result >83.08%. According to the prediction results, the “real fail” state or “fake fail” state of the devices can get in the future. By collaborating with a well‐designed resurrection algorithm (RA), the percentage of real and fake failed cells dropped by 35% and 29%, respectively. Besides, the tail bits in retention significantly reduced from 33% to 14.6% due to the reduction of oxygen vacancies in the gaps of conducting filaments by applying ten consecutive cycles of RA. This intelligent prediction and repair module can prolong the lifetime of RRAM chips effectively in practical applications.

Deep attributes: innovative LSTM-based seismic attributes

SUMMARY Seismic attributes are derived measures from seismic data that help characterize subsurface geological features and enhance the interpretation of subsurface structures: we propose to exploit the hidden layers of Long–Short Time Memory neural network predictions as possible new reflection seismic attributes. The idea is based on the inference process of a neural network, which in its hidden layers stores information related to different features embedded in the input data and which usually are not considered. Neural network applications typically ignore such intermediate steps because the main interest lies in the final output, which is considered as the exclusive exploitable feature of the process. On the contrary, here we analyse the possibility to exploit the intermediate prediction steps, hereafter referred as ‘deep attributes’ because they are produced by a deep learning algorithm, to highlight features and emphasize characteristics embedded in the data but neither recognizable by traditional interpretation, nor evident with classical attributes or multi-attribute approaches. Nowadays, classical signal attributes are numerous and used for different purposes; we here propose an original strategy to calculate attributes previously never exploited, which are potentially complementary or a good alternative to the classical ones. We tested the proposed procedure on synthetic and field 2-D and 3-D reflection seismic data sets to test and demonstrate the stability, affordability and versatility of the entire approach. Furthermore, we evaluated the performance of deep attributes on a 4-D seismic data set to assess the applicability and effectiveness for time-monitoring purposes and comparing them with the sweetness attribute.

Nonferrous metal price forecasting based on signal decomposition and ensemble learning

Nonferrous metals are indispensable raw materials for modern industry. The price forecasting of nonferrous metals is vital for business operators and investors. Based on the decomposition-integration framework, we propose a signal decomposition model combining variational mode decomposition (VMD) and an improved long-short time memory (LSTM) network. Using the MAE metric as a benchmark, the improved LSTM model (Mogrifier LSTM) obtained an average accuracy improvement of 5.99%. VMD is an efficient decomposition algorithm. However, it needs to set hyperparameters in advance. Unreasonable parameters will lead to poor decomposition results. Therefore, a method based on subseries complexity and reconstruction error (CAE) is proposed to reasonably decompose signals, improving 21.13% accuracy and reducing 37.56% computational overhead than other strategies. The structural model is introduced as a complement to the signal decomposition model, which learns different features by incorporating theoretical analyses into the choice of explanatory variables. The combining of two models achieves effective complementarity, obtaining an average accuracy improvement of 7.43%. Comparative tests on three datasets demonstrate the superiority of the proposed prediction framework. On the one hand, a reasonable decomposition strategy can play an essential role in the signal decomposition model. On the other hand, improving the prediction model and integrating different models is also an effective strategy to enhance accuracy.

Implementation of offline iterative hybrid simulation based on neural networks

AbstractReal‐time hybrid simulation (RTHS) is a testing method that combines numerical simulation and physical testing, enabling large‐scale or even full‐scale tests of large and complex engineering structures using the existing experimental facilities. At present, the accuracy and stability of RTHS are limited by the loading device and numerical solution efficiency. The experimental method was improved based on the neural network, and an off‐line iterative hybrid simulation method based on neural networks was implemented. Taking the tuned damping structure as examples, the dynamic metamodels of the tuned mass damper and tuned liquid damper were established based on the long‐short time memory (LSTM) neural network, and the model was iteratively simulated globally with the 9‐story benchmark model to calculate the response of the damping structure. The error of damper reaction predicted by LSTM neural network model is within 5.16%. The global iterative method can converge after a limited number of iterations, and the peak error of the structural response is within 7.85%.

Self-adaptive early warning of undesirable gas-liquid flow pattern in offshore oil and gas pipeline-riser system

It is an important function of submarine pipeline flow assurance system to monitor undesirable gas-liquid flow patterns in pipeline in real time, and release early warning ahead of the formation of these flow patterns. In order to tackle the problem of existing method that requires a large training set, and A method of self-adaptive early warning of transition to unstable regime is developed based on transient experiment of gas-liquid two-phase flow performed on a 150-m-long laboratorial pipeline-riser system. The responses of different differential pressure signals were investigated first, so as to pick out the optimum ones used for early warning. Then, the long-short time memory (LSTM) neural network combined with empirical mode decomposition (EMD) was adopted to judge transient condition. In the next step, a series of EMD-LSTM models were trained in order to predict whether undesirable flow pattern would occur, and the warning would be released if the result was ‘yes’. Within the range of experiments, all transient cases were correctly judged. The principles of proposed method were then tested and verified by data from a longer laboratorial pipeline-riser system and an offshore oil field, indicating good adaptability of the method.

The Short Time Prediction of the Dst Index Based on the Long-Short Time Memory and Empirical Mode Decomposition–Long-Short Time Memory Models

The Dst index is the geomagnetic storm index used to measure the energy level of geomagnetic storms, and the prediction of this index is of great significance for geomagnetic storm studies and solar activities. In contrast to traditional numerical modeling techniques, machine learning, which emerged decades ago based on rapidly developing computer hardware and software and artificial intelligence methods, has been unprecedentedly developed in geophysics, especially solar–terrestrial space physics. This study uses two machine learning models, the LSTM (Long-Short Time Memory, LSTM) and EMD-LSTM models (Empirical Mode Decomposition, EMD), to model and predict the Dst index. By building the Dst index data series from 2018 to 2023, two models were built to fit and predict the data. Firstly, we evaluated the influences of the learning rate and the amount of training data on the prediction accuracy of the LSTM model, and finally, 10−3 was thought to be the optimal learning rate. Secondly, the two models were used to predict the Dst index in the solar active and quiet periods, respectively, and the RMSE (Root Mean Square Error) of the LSTM model in the active period was 7.34 nT and the CC (correlation coefficient) was 0.96, and those of the quiet period were 2.64 nT and 0.97. The RMSE and CC of the EMD-LSTM model were 8.87 nT and 0.93 in the active period and 3.29 nT and 0.95 in the quiet period. Finally, the prediction accuracy of the LSTM model in the short time period was slightly better than the EMD-LSTM model. However, there will be a problem of prediction lag, which the EMD-LSTM model can solve and better predict the geomagnetic storm.

Combined wind speed forecasting model based on secondary decomposition and quantile regression closed-form continuous-time neural network

ABSTRACT Reliable interval forecasts can quantify the potential risk of wind speeds, which can help people make better use of wind energy. In this study, a new probabilistic prediction model, called Quantile Regression Closed-Form Continuous-time Neural Network (QRCfC), is proposed by combining quantile regression and closed-form continuous-time neural network. A new combined model combining QRCfC, secondary decomposition, multi-objective optimization, and dynamic weight combination strategy is proposed, which makes full use of the advantages of each single model to obtain accurate deterministic prediction and reliable probabilistic interval prediction of wind speed. First, the original wind speed series is divided into several subseries using a secondary decomposition technique built on variational mode decomposition and singular spectrum analysis. Then, four base models are used to predict these subseries separately. After that, the predicted values of the four base models are input into QRCfC for training, where the hyperparameters of QRCfC are dynamically adjusted by a multi-objective ant lion optimization algorithm. Finally, to verify the effectiveness of the proposed models, experiments are conducted using data sets from three wind farms in Gansu, China. Simulation results show that the proposed model achieves excellent prediction performance in both deterministic and probabilistic prediction. For example, compared with the unoptimized quantile regression time convolution network, quantile regression long-short time memory, quantile regression gated recurrent unit, and quantile regression neural network, the proposed model reduces the MAPE by 21.89% and the CRPS by 11.14% for 1-step prediction at site 1.

Deep Learning Approach for Detecting Cardiovascular Arrhythmias in Seven Lead ECG Signal from Holter

Cardiac arrhythmias are abnormalities caused by irregularities in the heart’s electrical conduction system. Cardiovascular diseases (CVD) have been identified as the leading cause of death worldwide. Premature ventricular contraction (PVC) is one of these diseases. It is an arrhythmia that can be linked to a several heart diseases that affect between 40% and 75% of the population. Ventricular bigeminy occurs when one or two premature beats are detected on an electrocardiogram when there is ventricular contraction between two normal heartbeats or trigeminy. The appearance of ventricular bigeminy or trigeminy rhythms is related to angina. Myocardial infarction, hypertension, and congestive heart failure are also possible conditions. Based on deep learning, this paper proposes creating a robust approach for automatically detecting and classifying cardiovascular arrhythmias in long-term electrocardiogram (ECG) recordings from halters based on deep learning (DL). We present a convolutional neural network (CNN) and long-short-time memory (LSTM) model that identifies cardiovascular arrhythmias. We have designed and implemented the proposed model using Python. The model was trained and validated on a database that includes a total of 17 long-recorded ECG signals (24 h) from 17 subjects, which were obtained from Yfa Hospital. The signals were recorded with seven leads holter. The CNN classifier achieved an accuracy of 91.14% as a final result, validated through a 10-fold cross-validation. Moreover, the proposed model was found to be capable of analyzing ECG recordings to classify multiple cardiovascular arrhythmias in the ECG record signals efficiently.

Auditing of hadoop log file for dynamic detection of threats using H-ISSM-MIM and convolutional neural network

Hadoop is a big data processing system that enables the distributed processing of massive data sets across multiple computers using straightforward programming techniques. Hadoop has been extensively investigated in many attacks as a result of its growing significance in industry. A company may learn about the actions of invaders as well as the weaknesses of the Hadoop cluster by examining a significant quantity of data from the log file. In a Big Data setting, the goal of the paper is to generate an analytical classification for intrusion detection. In this study, Hadoop log files were examined based on assaults that were recorded in the log files. Prior to analysis, the log data is cleaned and improved using a Hadoop preprocessing tool. For feature extraction, the hybrid Improved Sparrow Search Algorithm with Mutual Information Maximization (H-ISSA-MIM). Then the CNN (Convolutional Neural Network) classifier will detect the intrusions. The implementation is performed using the MATLAB 2020a software. The performance metrics like accuracy, precision, F-score, recall, specificity, FPR, FNR are calculated for the proposed methodology and it is compared with the existing techniques like Decision Tree (DT), Principal Components Analysis (PCA)- K means, Long Short Time Memory (LSTM). The maximum value of accuracy finds out in the proposed method 98% .

A Deep Reinforcement Learning-Based Path-Following Control Scheme for an Uncertain Under-Actuated Autonomous Marine Vehicle

In this article, a deep reinforcement learning-based path-following control scheme is established for an under-actuated autonomous marine vehicle (AMV) in the presence of model uncertainties and unknown marine environment disturbances is presented. By virtue of light-of-sight guidance, a surge-heading joint guidance method is developed within the kinematic level, thereby enabling the AMV to follow the desired path accurately. Within the dynamic level, model uncertainties and time-varying environment disturbances are taken into account, and the reinforcement learning control method using the twin-delay deep deterministic policy gradient (TD3) is developed for the under-actuated vehicle, where path-following actions are generated via the state space and hybrid rewards. Additionally, actor-critic networks are developed using the long-short time memory (LSTM) network, and the vehicle can successfully make a decision by the aid of historical states, thus enhancing the convergence rate of dynamic controllers. Simulation results and comprehensive comparisons on a prototype AMV demonstrate the remarkable effectiveness and superiority of the proposed LSTM-TD3-based path-following control scheme.

Deep Learning With Processing Algorithms for Forecasting Tourist Arrivals

The DL (Deep Learning) method is the standard for forecasting tourist arrivals. This method provides very good forecasting results but needs improvement if the data is small. Statistical data from the BPS (Central Bureau of Statistics) needs to be corrected, resulting in forecasts that tend to be invalid. This study uses statistical data and GT (Google Trends) as a solution so that the data is sufficient. GT data has a lot of noise because there is a shift between web searches and departures. This difference will produce noise that needs to be cleaned. We use monthly data from January 2008 to December 2021 from BPS sources combined with GT. Hilbert-Huang Transform (HHT) is proposed to clean data from various disturbances. The DL used in this study is long short-time memory (LSTM) and was evaluated using the root mean squared error RMSE and mean absolute percentage error (MAPE). The evaluation results show that the HHT-LSTM results are better than without data cleaning.

Comparison of Prediction of the Number of People Exposed to Covid 19 Using the Lagrange Interpolation Method with the Newton Gregory Maju Polynomial Interpolation Method

In March 2020 the World Health Organization stated that the Corona Virus pandemic (Covid-19) was due to its massive spread and hit all countries in the world. Academics and practitioners are called upon to carry out research activities in order to obtain a mathematical model that can be used to predict the number of people exposed to Covid-19 or other diseases. The researchers previously tried research to predict the number of people exposed to Covid-19 from early 2021 using the Monte Karlo method, the Hybrid Nonlinear Regression Logistic– Double Exponential Smoothing method, the Arima method, the BackPropagation and Fuzzy Tsukamoto methods, the K-Nearest method. Neighbors, Time Series Analysis method, Winter Method and Long Short Time Memory (LSTM) Artificial Neural Network method.

Modeling Methodology Based on Fast and Refined Neural Networks for Non-Isolated DC–DC Converters With Configurable Parameter Settings

Compared with conventional physics-based methods, e.g., analytical modeling and numerical modeling, data-driven methods can extract input-to-output relationships from the data without much prior knowledge of the physical system, thus showing great potential in modeling power electronics (PE) converters with complex switching behaviors and configurable parameter settings. Previous data-driven PE circuit modeling approaches are mostly based on sequential neural networks, and their execution speed suffers from large sequential lengths due to a high sampling rate for high modeling accuracy. Moreover, modeling of refined singular ripples is missing and configurable parameter settings are not available in these data-driven modeling approaches. To address the above-mentioned issues, this paper proposes a hybrid physics-informed machine learning (ML) method to model the non-isolated DC-DC converters. The approach empirically decomposes the output signals into transient large signals and periodic small signals. For transient large signals, a fully-connected neural network (NN) is used to map circuit parameters with system characteristics, such that configurable circuit parameter settings are allowed. For periodic signals, a long short-time memory (LSTM) network together with convolutional neural network (CNN) is used to accelerate the simulation by predicting signal features in the compressed latent space. A buck converter with configurable parameter settings is modeled by the proposed hybrid physics-informed ML method. Periodic ripples are successfully generated, while execution speed is about 10 times faster than that of conventional numerical methods.

Technical Paper on Stock Prediction using Machine Learning Algorithms

Abstract: In today's financial world, a stock market is quite important. The stock market's worth is an important part of today's global economic system. The stock market attracts a large number of people from all areas of life, whether corporate or academic. Because of the non-linear structure of the stock market, research is one of the world's hottest and most important topics. People invest in the stock market based on previous theory outcomes or assumptions. When it comes to forecasting, people typically look for ways or procedures to help them reduce risk and improve performance. As a result, in today's competitive stock market, stock value forecast is crucial. Critical and technical studies' approach cannot guarantee the consistency and accuracy of estimations. As a result, machine learning technology has risen in popularity in stock forecasting over the last 5-6 years, with estimates based on current stock values that have been educated in terms of prior values. This research focuses on using LSTM (Long Short Time Memory) technology to forecast future market trends.