Long Short-term Memory Method Research Articles

Prediction accuracy improvement of pressure pulsation signals of reversible pump‐turbine: A LSTM and VMD‐based optimization approach

AbstractThe reversible pump‐turbine plays an important role in hydropower stations, but pressure pulsation during their operation affects their performance and lifespan. Accurate prediction of pressure pulsation signals can provide an important basis for energy planning and stable operation of pumped storage units, thereby promoting sustainable development of the environment. This study introduces an optimization method that combines long short‐term memory (LSTM) and variable mode decomposition (VMD) to enhance the prediction accuracy of pressure pulsation signals. First, by decomposing the pressure pulsation signal into multiple relatively stable subsequence components using VMD, the characteristics of the original signal become more distinct. Subsequently, individual LSTM‐based time series prediction models were constructed for each modal function, and the hyperparameters related to subsequence were optimized using the sparrow search algorithm. To validate the efficacy of the proposed approach, this paper conducted experiments using pressure pulsation signals of a pump‐turbine obtained through numerical simulation. The experimental data was divided into training and testing sets, with the former used to train the LSTM model and the latter used for validation. The experimental results show that the optimized VMD with an optimized LSTM method can effectively improve the prediction accuracy of pressure pulsation signals in reversible pump‐turbine.

Ultrasonic lamination defects detection of carbon fiber composite plates based on multilevel LSTM

During the production of carbon fiber composites (CFC), various forming methods and complex processes can introduce different types of defects, with lamination defects being a major concern. In this paper, we propose a multilevel Long Short-Term Memory (LSTM) neural network combined with ultrasonic detection to identify the lamination defects in carbon fiber composites. Unlike conventional ultrasonic detection that requires multiple sets of probes, this method only employs a one-to-one transmission and reception mode. The COMSOL-Multiphysics finite element software is utilized to simulate the ultrasonic transmission and generate the necessary ultrasonic data. By incorporating multiple levels of learning, the accuracy and convergence of the traditional LSTM can be obviously enhanced. This approach uses ultrasound waveform data collected from a single set of probes to predict the locations and sizes of lamination defects. Based on fewer than 10,000 datasets where each dataset represents a waveform, the numerical results demonstrate a prediction accuracy of over 90% for defect position and size. Moreover, the multilevel LSTM method exhibits convergence, and incorporating more data can further promote the prediction accuracy. This method offers a time-saving, labor-saving, and cost-effective solution for detecting and analyzing defects in CFC.

Hybrid Time-Series Prediction Method Based on Entropy Fusion Feature

High-precision time sequence forecasting is a complicated cyber-physical system (CPS) task. Due to the diversity of data scales and types, the classic time-series prediction model meets the challenge to deliver accurate prediction results for many forms of time-series data. This work proposes a hybrid model with long short-term memory (LSTM) and embedded empirical mode decomposition (EEMD) based on the entropy fusion feature. First, we apply EEMD in entropy fusion feature long short-term memory (ELSTM) to lessen pattern confusion and edge effects in traditional empirical mode decomposition (EMD). The sequence is then divided into intrinsic mode functions (IMF) by using EEMD. Then, feature vectors are constructed between IMFs and their respective information entropy for feature merging. LSTM is used to build a full connection network for each entropy fusion feature IMF subsequence for prediction and each type of IMF subsequence as the feature dimension to obtain its prediction results. Finally, the output results of all IMF subsequences are reconstructed to obtain the final prediction result. Compared with the LSTM method, the performance of the proposed method has been improved 64.33% on the evaluation metric MAPE. The proposed model has also delivered the best prediction outcomes across four different time-series datasets. The experimental results conclusively show that the proposed method outperforms other models compared.

Improvement Interpolation Method for Vessel Trajectory Prediction based on AIS Data

An acceptable shipping monitoring system should be able to provide early detection of vessel accidents. One of the keys to early detection is trajectory prediction. Even though the prediction results depend heavily on the historical data of the vessel trajectory, there are often missing values in the trajectory data due to disturbances in the Automatic Identification System (AIS) data transmission process. Therefore, this study proposes a preprocessing method that includes data cleaning, trajectory extraction, and a combination of Linear interpolation methods for straight-shaped trajectories and Cubic Spline interpolation for curved-shaped trajectories. Test results involving three different trajectories showed that the Gated Recurrent Units (GRU) method produces a smaller Root Mean Square Error (RMSE) value on linear interpolation. Visually, however, the GRU method with linear interpolation has a deficiency in the curved-shaped trajectories. Our studies involving the Bidirectional GRU (BiGRU), Long Short Term Memory (LSTM), and Bidirectional LSTM (BiLSTM) methods showed that the model built using the proposed interpolation method has a lower RMSE value. This study emphasizes that good predictions of vessel trajectories based on AIS data requires an additional process, namely the interpolation of vessel trajectories according to their shape.

Application of the combination on data-driven and discrete element model: Particle segregation prediction and classification in rotary drums

With the complexity and variability of particle systems in industrial scenarios and the huge cost of repetitive testing, predicting and classifying particle systems in advance have a certain necessity. Combined with discrete element method (DEM) and data-driven model, this work develops a prediction and classification model for particle flow characteristics based on the long-short term memory (LSTM) method. On the basis of the generated macroscopic and microscopic particle flow characteristic information, the LSTM model was constructed and then described particle flow information based on optimal model parameters determined by hyperparameter optimization. Compared of predicted results with the DEM simulation validated the interpolation and extrapolation ability of the established model. And it has better prediction accuracy and performance comparing with traditional prediction methods. In addition, for the classification of multiple working condition categories, the LSTM model trained from fused dataset obtained the best results with the accuracy of more than 95 % in all cases. With the combination of DEM model and data-driven approach, the rapid prediction of particle flow characteristics and parameter optimization in various industrial applications will be achieved.

DETECTION MODEL FOR FAKE NEWS ON COVID-19 IN INDONESIA

Today, fake information has become a significant problem, exacerbated by the acceleration of access to information. The spread of fake information has a dangerous impact, especially regarding global health issues, for example COVID-19. People can access various resources to obtain information, including online sites and social media. One of the methods to control the spread of false information is detecting hoaxes. Many methods have been developed to identify hoaxes; most previous studies have focused on developing hoax detection methods using data from a single source in English. The present study is carried out to detect fake news in Indonesian language using multiple data sources, including traditional and social media in the context of COVID-19. The study uses Long Short-Term Memory (LSTM) and the Robustly Optimised Bidirectional Encoder Representations from Transformers Pre-Training Approach (RoBERTa). The LSTM approach is used to develop four different architectures that varied based on: (1) the use of text-only versus the use of both title and text; (2) the number of LSTM and dense layers; and (3) the activation function. The LSTM model with text-only data, a single LSTM layer and two dense layers, outperformed other LSTM architectures, achieving the highest accuracy of 92.17%. The LSTM models require a considerably short training time of 23–27 minutes for 3,847 articles and has a detection time of 3.8–4.1 ms per article. The RoBERTa classifiers outperformed all LSTM models with an accuracy of over 97% and a significantly better training time, with a margin of more than 50% compared to LSTM classifiers, although it had a slightly longer test time. Both LSTM and RoBERTa models outperformed the Naïve Bayes and SVM benchmark methods in terms of accuracy, precision, and recall. Therefore, this study shows that both LSTM and RoBERTa methods are reliable and can be reasonably implemented for real-time fake news detection.

Stock price prediction based on the long short-term memory network

Stock analysis is a challenging task that involves modelling complex and nonlinear dynamics of stock prices and volumes. Long Short-Term Memory (LSTM) is a type of recurrent neural network that can capture long-term dependencies and temporal patterns in time series data. In this paper, a stock analysis method based on LSTM is proposed that can predict future stock prices and transactions using historical data. Yfinance is used to obtain stock data of four technology companies (i.e. Apple, Google, Microsoft, and Amazon) and apply LSTM to extract features and forecast trends. Various techniques are also used such as moving average, correlation analysis, and risk assessment to evaluate the performance and risk of different stocks. When compare the method in this paper with other neural network models such as RNN and GRU, the result show that LSTM achieves better accuracy and stability in stock prediction. This paper demonstrates the effectiveness and applicability of LSTM method through experiments on real-world data sets.

Prediction of Wind Power with Machine Learning Models

Wind power is a vital power grid component, and wind power forecasting represents a challenging task. In this study, a series of multiobjective predictive models were created utilising a range of cutting-edge machine learning (ML) methodologies, namely, artificial neural networks (ANNs), recurrent neural networks (RNNs), convolutional neural networks, and long short-term memory (LSTM) networks. In this study, two independent data sets were combined and used to predict wind power. The first data set contained internal values such as wind speed (m/s), wind direction (°), theoretical power (kW), and active power (kW). The second data set was external values that contained the meteorological data set, which can affect the wind power forecast. The k-nearest neighbours (kNN) algorithm completed the missing data in the data set. The results showed that the LSTM, RNN, CNN, and ANN algorithms were powerful in forecasting wind power. Furthermore, the performance of these models was evaluated by incorporating statistical indicators of performance deviation to demonstrate the efficacy of the employed methodology effectively. Moreover, the performance of these models was evaluated by incorporating statistical indicators of performance deviation, including the coefficient of determination (R2), root mean square error (RMSE), mean absolute error (MAE), and mean square error (MSE) metrics to effectively demonstrate the efficacy of the employed methodology. When the metrics are examined, it can be said that ANN, RNN, CNN, and LSTM methods effectively forecast wind power. However, it can be said that the LSTM model is more successful in estimating the wind power with an R2 value of 0.9574, MAE of 0.0209, MSE of 0.0038, and RMSE of 0.0614.

A causal-temporal graphic convolutional network (CT-GCN) approach for TBM load prediction in tunnel excavation

This research proposes a novel deep learning approach named causal-temporal graphic convolutional network (CT-GCN) which aims to provide accurate predictions on tunnel boring machine’s (TBM) load parameters. The causal associations among the key TBM operational parameters are detected and quantified with causal effects through the Peter and Clark momentary conditional independence plus (PCMCI+) method. The discovered causality can be further integrated with a deep learning model that consists of graph convolutional network (GCN) and long short-term memory (LSTM) layers for accurate prediction of TBM’s torque and thrust. Data collected from a realistic tunnel project in Singapore is utilized to demonstrate the effectiveness of the proposed approach. The load parameters are predicted with their historical values and another 7 key operational parameters. The results indicate that (1) The proposed CT-GCN approach can achieve a low mean absolute error (MAE) and root mean squared error (RMSE) of 40.90kN∙m and 64.53kN∙m for thrust force (y1) and that of 635.76kN and 1168.59kN for cutterhead torque (y2), respectively. (2) The proposed CT-GCN method achieves 12.86 % and 24.38 % average improvements in the coefficient of determination (R2) for y1 and y2, respectively when compared with the long-short term memory (LSTM) method and that of 5.62 % and 8.60 % improvements when compared with the GCN-LSTM method. (3) Compared with correlation-based GCN models, the proposed approach exerts an average improvement of 43.10 %, 41.97 %, and 22.72 % in terms of MAE, RMSE, and R2 for torque and much better performance for thrust estimation. This study contributes to improving the understanding of TBM operation by quantifying the causality among the key operational parameters. It also contributes to developing a novel deep-learning method that estimates TBM’s load parameters with high accuracy.

Anomaly detection using K-Means and long-short term memory for predictive maintenance of large-scale solar (LSS) photovoltaic plant

Detecting and monitoring faults in solar photovoltaic (PV) systems is crucial to ensure optimal efficiency and prevent safety and fire hazards. However, the conventional operation and maintenance (O&M) of solar PV systems do not utilize machine learning for fault detection and classification. This poses challenges for plant operators, especially those managing large-scale solar (LSS) PV plants, who typically rely on manual approaches to screen large amounts of electrical data and inspect numerous string panels. Consequently, the cost of O&M is high. This study aims to use advanced machine-learning techniques to detect anomalies in a Large-Scale Photovoltaic (LSSPV) plant. The study collects data from the plant which located in the central of Peninsular Malaysia and employs K-Means for clustering and Long-Short Term Memory (LSTM) for anomaly detection in the predicted electrical current of string modules. The model is developed using Jupyter Notebook from the Python Package Index. To validate the accuracy of the proposed model, the study compares LSTM with Artificial Neural Network (ANN) using relative error as the evaluation metric. The results indicate that the LSTM method identifies anomalies in the predicted output current of the string modules more accurately and with lower relative error than the conventional ANN technique. The proposed model could help plant operators perform predictive maintenance of LSSPV plants at a minimal cost and time.

LSTM for Modeling of Cylinder Pressure in HCCI Engines at Different Intake Temperatures via Time-Series Prediction

Modeling engines using physics-based approaches is a traditional and widely-accepted method for predicting in-cylinder pressure and the start of combustion (SOC). However, developing such intricate models typically demands significant effort, time, and knowledge about the underlying physical processes. In contrast, machine learning techniques have demonstrated their potential for building models that are not only rapidly developed but also efficient. In this study, we employ a machine learning approach to predict the cylinder pressure of a homogeneous charge compression ignition (HCCI) engine. We utilize a long short-term memory (LSTM) based machine learning model and compare its performance against a fully connected neural network model, which has been employed in previous research. The LSTM model’s results are evaluated against experimental data, yielding a mean absolute error of 0.37 and a mean squared error of 0.20. The cylinder pressure prediction is presented as a time series, expanding upon prior work that focused on predicting pressure at discrete points in time. Our findings indicate that the LSTM method can accurately predict the cylinder pressure of HCCI engines up to 256 time steps ahead.

Predicting Multiple Numerical Solutions to the Duffing Equation Using Machine Learning

This study addresses the problem of predicting convergence outcomes in the Duffing equation, a nonlinear second-order differential equation. The Duffing equation exhibits intriguing behavior in both undamped free vibration and forced vibration with damping, making it a subject of significant interest. In undamped free vibration, the convergence result oscillates randomly between 1 and −1, contingent upon initial conditions. For forced vibration with damping, multiple variables, including initial conditions and external forces, influence the vibration patterns, leading to diverse outcomes. To tackle this complex problem, we employ the fourth-order Runge–Kutta method to gather convergence results for both scenarios. Our approach leverages machine learning techniques, specifically the Long Short-Term Memory (LSTM) model and the LSTM-Neural Network (LSTM-NN) hybrid model. The LSTM-NN model, featuring additional hidden layers of neurons, offers enhanced predictive capabilities, achieving an impressive 98% accuracy on binary datasets. However, when predicting multiple solutions, the traditional LSTM method excels. The research encompasses three critical stages: data preprocessing, model training, and verification. Our findings demonstrate that while the LSTM-NN model performs exceptionally well in predicting binary outcomes, the LSTM model surpasses it in predicting multiple solutions.

The Menganalisis Pergerakan Harga Saham PT Indofood CBP Sukses Makmur Tbk: Sebuah Tinjauan Sebelum dan Sesudah Masa Pandemi

This study aims to determine the share price of Indofood CBP Sukses Makmur TBK (ICBP) using three different methods, namely Long Short-Term Memory (LSTM), Logistic Regression and Support Vector Machine (SVM). Daily stock price data from October 2010 to January 2023 are used for input in this study. The LSTM method is used to predict ICBP's stock price by taking information from several previous periods, while Logistic Regression and SVM are used to predict stock price trends.

The Application of the Long-Short Term Memory (LSTM) Forecasting Method on the Impact of Tropical Cyclones in Indonesia

Effective disaster mitigation strategies are paramount in the realm of risk management concerning natural calamities, with the primary objective of mitigating potential devastation. A pragmatic and impactful method involves predicting the contributory aspects of such disasters, encompassing variables such as torrential rainfall and formidable wind velocities that tropical cyclones bring. In this study, a comparative analysis of forecasting methodologies is undertaken, precisely the Long Short-Term Memory (LSTM) technique and the Holt Winter approach, both directed toward gauging the impact of tropical cyclones. This investigation focuses on two critical factors: the forecast of precipitation intensity and the estimation of maximum wind speed. The outcomes underscore the superior predictive capabilities of the LSTM method, unequivocally revealing its aptitude for predicting rainfall and wind speed. Impressively, the LSTM method yields remarkable precision levels of 97.433% for rainfall and an even higher accuracy of 99.018% for maximum wind speed forecasting. In essence, this study highlights LSTM's efficacy in disaster prediction with substantial accuracy.

Lithium-ion battery state-of-charge estimation based on long-short-term memory neural network and square-root cubature Kalman filter

Due to the widespread use of Li-ion batteries in electric vehicles, battery management systems for monitoring the status and ensuring the safe operation of Li-ion batteries have been extensively studied. Online monitoring of the state of charge (SOC) is crucial for lithium-ion batteries, but achieving precise SOC estimation is a difficult task due to battery dynamics and the influence of factors such as current, temperature, and operating conditions on SOC variability. This paper introduces a novel approach that combines a Long Short-Term Memory (LSTM) network with a square-root cubature Kalman filter (SRCKF) to address this challenge. To tackle the issue at hand, the proposed methodology employs a two-step approach. Initially, the LSTM network is utilized to capture the complex relationship between the state of charge (SOC) and the measured variables, including current, voltage, and temperature. Subsequently, the output of the LSTM network is subjected to smoothing using the SRCKF technique, leading to precise and consistent SOC estimation. A notable advantage of this method is its ability to simplify the arduous task of parameter tuning for the LSTM network, eliminating the need for constructing a battery model. The experimental results demonstrate that the maximum error in estimating the state of charge (SOC) using this particular method is constrained within the 5% threshold. Compared with using a separate LSTM method at different temperatures, by using the combination method, the root mean square error and maximum error of SOC estimation are greatly reduced.

Univariate and Multivariate Long Short Term Memory (LSTM) Model to Predict Covid-19 Cases in Malaysia Using Integrated Meteorological Data

The rate of transmission of coronavirus disease (COVID-19) has been very fast since the first reported case in December 2019 in Wuhan, China. The disease has infected more than 3 million people worldwide and resulted in more than 224 thousand deaths as of May 1, 2020, reported by The World Health Organization (WHO). In the past, meteorological parameters such as temperature and humidity were essential and effective factors against serious infectious diseases such as influenza and Severe Acute Respiratory Syndrome (SARS). Therefore, exploring the relationship between meteorological factors and active COVID-19 cases is essential. This study employs the long-short term memory (LSTM) method to predict Covid-19 Cases in Malaysia. We propose a univariate and multivariate model using Covid-19 cases and meteorology data. The univariate LSTM model uses Covid-19 active cases data in a year as a control attribute for model development. The multivariate LSTM model uses the integrated Covid-19 cases, and meteorology data consists of attributes: minimum, maximum, and average values of Humidity, Temperature, Windspeed, and Pressure from 13 states of Malaysia. The model's performance is evaluated using errors such as MAE, RMSE, MAPE, and the R2 Score. The low errors and higher R2 score indicate the model's excellent performance. We observed that the univariate LSTM model gives the least error in five states, indicating that those states' daily active cases are the main contributing factors. In the multivariate LSTM model, the daily cases and humidity, temperature, and windspeed are the main factors in several different states. The result of the study is to help the government to prevent and manage the spread of the COVID-19 and other upcoming pandemic better.

Deep learning approach to forecast air pollution based on novel hourly index

Air pollution is a pressing concern that the entire world is striving to combat. Among air pollutants, particulate matter poses a significant threat to human health. The Sustainable Development Goals (SGD3, SGD7 and SGD11) include initiatives to address air pollution. Two innovative methods are proposed in this research to predict the PM2.5 concentration in advance. While multivariate time series prediction models typically employ multiple features as inputs, this research reduces the number of inputs, which makes the proposed combination of approaches simple and effective. The approaches involve the development of two new indexing methods, namely, the Hourly Relative Mean Index and the Hourly Weighted Index. This research offers innovative hybrid deep learning models that incorporate the newly developed indices, Long Short Term Memory (LSTM) models, and robust preprocessing techniques. Multivariate Isolation Forest Relative Index Bidirectional LSTM and Multivariate Isolation Forest Weighted Index LSTM methods are used to forecast PM2.5 concentration for an hourly time frame. Further, Multivariate Isolation Forest Relative Index LSTM and Multivariate Isolation Forest Weighted Index LSTM methods are used to forecast PM2.5 concentration 48 h ahead. The study establishes that the proposed combination of approaches outperform traditional ways to achieve superior performance with reduced complexities requiring fewer inputs for predictions.

Comparison of Gold Price Prediction Techniques Using Long Short Term Memory (LSTM) And Fuzzy Time Series (FTS) Method

Gold is a precious metal that has economic value and is often used as an investment tool. The demand for gold from day to day is increasing, because many know and think that gold can be used as ownership in the form of investment assets that have low risk. Therefore, it is necessary to predict the gold price to avoid losses. This study aims to predict the gold price using a machine learning architecture including deep learning, namely Long Short Term Memory (LSTM) and Fuzzy Time Series (FTS). Several trial processes were carried out in the training process and predict the LSTM and FTS models to get the best results. The data used in this experiment is real data from the period 15 September 2016 – 15 September 2021. The final results obtained from the LSTM method have an RMSE value of training data of 391.95 RMSE, and a value of test data 412.36 RMSE, and the FTS method has an RMSE value of 10449.115791541652

Application, interpretability and prediction of machine learning method combined with LSTM and LightGBM-a case study for runoff simulation in an arid area

The runoff prediction can provide scientific basis for flood control, disaster reduction and water resources planning. Due to a large number of uncertainties in runoff prediction, it is difficult to make precise predictions. To improve the accuracy of runoff prediction, this study combines techniques of Long Short-Term Memory (LSTM) and Light Gradient Boosting Machine (LightGBM) in machine learning with reciprocal error method to develop an integrated data-driven model (i.e., LSTM-LightGBM) for runoff prediction. To demonstrate its applicability, the model is applied to the annual runoff prediction of the Caiqi hydrological monitoring station in the Shiyang River in an arid area. Indicators include Error of Peak (EP), Nash-Sutcliffe Efficiency (NSE), Root Mean Squared Error (RMSE), and Mean Absolute Error (MAE) are adopted to evaluate the prediction performance of the LSTM, LightGBM, and LSTM-LightGBM methods under the same hyperparameter combinations. Then, the interpretability of LSTM and LightGBM models is also explored based on the permutation importance method and Shapley Additive exPlanations (SHAP) values, respectively. Finally, future annual runoff at the Caiqi for the next 50 years (2025–2075) is predicted based on LSTM-LightGBM model under 12 climate scenarios. Therefore, results show that: 1. the integrated model (LSTM-LightGBM) has good performance than two single models in NSE (0.92), RMSE (0.075 million m3) and MAE (0.046 million m3) and EP value (i.e., for bridging the peak-valley runoff). 2. In this case, it is found that four feature variables have the greatest influence on the target variables through the interpretable analysis. 3. The 12 combined climate scenarios used in this investigation produced generally steady predictions. The scenarios with the highest and lowest mean values are GFDL RCP 6.0 (3.12 × 108m3) and IPSL RCP 2.6 (3.04 × 108m3), respectively, with a decrease of 24.09 % and 26.03 % compared to the mean annual runoff of 4.11 × 108m3 in the baseline period (1955–2017). These findings can provide scientific bases for future water resources planning in the downstream of the Shiyang River Basin.

Rancang Bangun Aplikasi Peramalan Jumlah Penumpang Menggunakan Long Short-Term Memory (LSTM)

Public services such as public transportation are closely related to user satisfaction. Busway DKI Jakarta is one of the first public transportation services in Southeast and South Asia. In order to maintain passenger satisfaction, the management of Busway continues to improve services such as adding buses and opening a new line. Opening a new lane or adding buses must necessarily be adjusted also with the increasing number of passengers. So to know the number of passengers in the future, it is necessary to forecast the number of passengers through existing historical data. The historical data used is time series data from January 2015 to January 2016. The method used in forecasting is Long Short-Term Memory (LSTM), one of the machine learning methods. The method is measured in accuracy using Root Mean Squared Error (RMSE) and Mean Absolute Percentage Error (MAPE). As a comparison of LSTM accuracy in forecasting, we also use the Exponential Smoothing method. Based on the results of forecasting, the most and least dominant method of producing RMSE and MAPE is the LSTM method. Only in corridor 3 LSTM can not provide RMSE and MAPE values below baseline values. While corridor 5 LSTM can give better results after the data transformation process by using exponential smoothing. However, overall the LSTM method provides the best accuracy based on the lowest average RMSE and MAPE values, namely RMSE of 2640.53 and MAPE of 9.14%.