LSTM Model Research Articles

Neural network analysis in time series forecasting

Objectives. To build neural network models of time series (LSTM, GRU, RNN) and compare the results of forecasting with their mutual help and the results of standard models (ARIMA, ETS), in order to ascertain in which cases a certain group of models should be used.Methods. The paper provides a review of neural network models and considers the structure of RNN, LSTM, and GRU models. They are used for modeling time series in Russian macroeconomic statistics. The quality of model adjustment to the data and the quality of forecasts are compared experimentally. Neural network and standard models can be used both for the entire series and for its parts (trend and seasonality). When building a forecast for several time intervals in the future, two approaches are considered: building a forecast for the entire interval at once, and step-by-step forecasting. In this way there are several combinations of models that can be used for forecasting. These approaches are analyzed in the computational experiment.Results. Several experiments have been conducted in which standard (ARIMA, ETS, LOESS) and neural network models (LSTM, GRU, RNN) are built and compared in terms of proximity of the forecast to the series data in the test period.Conclusions. In the case of seasonal time series, models based on neural networks surpassed the standard ARIMA and ETS models in terms of forecast accuracy for the test period. The single-step forecast is computationally less efficient than the integral forecast for the entire target period. However, it is not possible to accurately indicate which approach is the best in terms of quality for a given series. Combined models (neural networks for trend, ARIMA for seasonality) almost always give good results. When forecasting a non-seasonal heteroskedastic series of share price, the standard approaches (LOESS method and ETS model) showed the best results.

Deep learning for Bitcoin price direction prediction: models and trading strategies empirically compared

This paper applies deep learning models to predict Bitcoin price directions and the subsequent profitability of trading strategies based on these predictions. The study compares the performance of the convolutional neural network–long short-term memory (CNN–LSTM), long- and short-term time-series network, temporal convolutional network, and ARIMA (benchmark) models for predicting Bitcoin prices using on-chain data. Feature-selection methods—i.e., Boruta, genetic algorithm, and light gradient boosting machine—are applied to address the curse of dimensionality that could result from a large feature set. Results indicate that combining Boruta feature selection with the CNN–LSTM model consistently outperforms other combinations, achieving an accuracy of 82.44%. Three trading strategies and three investment positions are examined through backtesting. The long-and-short buy-and-sell investment approach generated an extraordinary annual return of 6654% when informed by higher-accuracy price-direction predictions. This study provides evidence of the potential profitability of predictive models in Bitcoin trading.

Integration of Price Data in Wind Power Sales Systems Using Association Rule Mining Combined with Deep Learning Algorithms

With the growing attention of the global population towards renewable energy sources such as wind power derived from natural resources such as Solar, Geothermal, and Wind energy, there is a growing need for genuine estimation of wind energy. Wind energy is important in the supply of electricity in the global energy markets because of its enormous importance in the delivery of renewable energy. As such, it is crucial for accurate appreciation of the power of the wind to ably respond to issues that relate to the trading of power while at the same time addressing issues to do with planning, scheduling and strategic positioning of wind power generation. Therefore, the present work aims to develop a new model known as the Association Rule with DL-based Wind Power Generation and Price Prediction (ARDL-WPGPP). It utilizes two datasets: An Energy dataset includes various columns like tie, wind onshore forecast, and price actual, whereas the weather features dataset includes only wind speed. These datasets are combined to create a dataset where each transaction represents 2 hours of wind generation. A data mining approach is employed to uncover hidden patterns, rules, concepts, and correlations within these datasets, operating on various types of data including quantitative, textual, and multimedia formats. To efficiently extract rules from the dataset, an improved Apriori algorithm is introduced. Subsequently, the generated rules incorporating wind speed are passed into an improved LSTM model, which learns by comparing the label data. The price actual value serves as the label data, assigning labels to data points based on their actual price value. High price actual values indicate high wind power prices, while low values indicate low wind power prices.

Management of Exchange Rate Forecasting Through Vanilla Long Short-Term Memory Networks (LSTM) and Auto-Regressive Integrated Moving Average (ARIMA)

Predicting foreign exchange rates presents a formidable challenge within financial forecasting, given its pivotal role in influencing a country's economic trajectory. To address this challenge, numerous forecasting models are employed with the aim of anticipating future exchange rate movements. This study aims to determine the efficacy of two prominent machine learning models, namely Vanilla Long Short-Term Memory Networks (LSTM) and Auto-Regressive Integrated Moving Average (ARIMA), in forecasting the exchange rate between the Malaysian Ringgit (MYR) and the United States Dollar (USD). Employing Python's robust statistical packages for time series forecasting, both Vanilla LSTM and ARIMA models undergo rigorous training on the dataset. Leveraging Python's programming capabilities enables in-depth analysis, essential for model refinement and accuracy assessment. Upon comparing the error measures of both models, it becomes evident that the Vanilla LSTM model outperforms ARIMA, exhibiting lower Mean Squared Error (MSE) and Root Mean Squared Error (RMSE) values. Specifically, the MSE and RMSE for Vanilla LSTM stand at 0.0102 and 0.1011, respectively, surpassing ARIMA's 0.0113 and 0.1062. Thus, affirming Vanilla LSTM's superiority in exchange rate forecasting. Consequently, the study concludes that Vanilla LSTM emerges as the most accurate model for exchange rate prediction, with a projected exchange rate of RM4.22 for July 2022.

Prediction of novel malware using hybrid convolution neural network and long short-term memory approach

The rapid evolution of network communication technologies has led to the emergence of new forms of malware and cybercrimes, posing significant threats to user safety, network infrastructure integrity, and data privacy. Despite efforts to develop advanced algorithms for detecting malicious activity, constructing models that are both accurate and reliable remains a challenge, especially in handling vast and dynamically shifting data patterns. The prevalent bag-of-words (BOW) method, while widely used, falls short in capturing crucial spatial and sequence information vital for detecting malware patterns. To address this challenge, the work presented in this paper proposes hybrid convolution neural network-long short-term memory network (CNN-LSTM) combination models, leveraging CNN's spatial information extraction and LSTM's temporal modeling capabilities. Focused on predicting the infiltration of malicious software into personal computers, the proposed hybrid CNN-LSTM model considers factors such as location, firmware version, operating system, and anti-virus software. The proposed models undergo training and evaluation using Microsoft's malware dataset, demonstrating superior performance compared to traditional CNN and LSTM models. The CNN-LSTM model achieves an impressive accuracy of 95% on the Microsoft malware dataset, highlighting its effectiveness in malware detection.

Research on precise lithium battery state of charge estimation method based on CALSE-LSTM model and pelican algorithm

This paper presents an innovative fusion model called “CALSE-LSTM,” which integrates Convolutional Neural Networks (CNNs), Long Short-Term Memory Networks (LSTMs), self-attention mechanisms, and squeeze-and-excitation attention mechanisms to optimize the estimation accuracy of the State of Charge (SoC). The model incorporates battery historical data as input and employs a dual-attention mechanism based on CNN-LSTM to extract diverse features from the input data, thereby enhancing the model's ability to learn hidden information. To further improve model performance, we fine-tune the model parameters using the Pelican algorithm. Experiments conducted under Urban Dynamometer Driving Schedule (UDDS) conditions show that the CALSE-LSTM model achieves a Root Mean Squared Error (RMSE) of only 1.73 % in lithium battery SoC estimation, significantly better than GRU, LSTM, and CNN-LSTM models, reducing errors by 31.9 %, 31.3 %, and 15 %, respectively. Ablation experiments further confirm the effectiveness of the dual-attention mechanism and its potential to improve SoC estimation performance. Additionally, we validate the learning efficiency of CALSE-LSTM by comparing model training time with the number of iterations. Finally, in the comparative experiment with the Kalman filtering method, the model in this paper significantly improved its performance by incorporating power consumption as an additional feature input. This further verifies the accuracy of CALSE-LSTM in estimating the State of Charge (SoC) of lithium batteries.

RL-Chord: CLSTM-Based Melody Harmonization Using Deep Reinforcement Learning.

Automatic music generation is the combination of artificial intelligence and art, in which melody harmonization is a significant and challenging task. However, previous recurrent neural network (RNN)-based work fails to maintain long-term dependency and neglects the guidance of music theory. In this article, we first devise a universal chord representation with a fixed small dimension, which can cover most existing chords and is easy to expand. Then a novel melody harmonization system based on reinforcement learning (RL), RL-Chord, is proposed to generate high-quality chord progressions. Specifically, a melody conditional LSTM (CLSTM) model is put forward that learns the transition and duration of chords well, based on which RL algorithms with three well-designed reward modules are combined to construct RL-Chord. We compare three widely used RL algorithms (i.e., policy gradient, Q -learning, and actor-critic algorithms) on the melody harmonization task for the first time and prove the superiority of deep Q -network (DQN). Furthermore, a style classifier is devised to fine-tune the pretrained DQN-Chord for zero-shot Chinese folk (CF) melody harmonization. Experimental results demonstrate that the proposed model can generate harmonious and fluent chord progressions for diverse melodies. Quantitatively, DQN-Chord achieves better performance than the compared methods on multiple evaluation metrics, such as chord histogram similarity (CHS), chord tonal distance (CTD), and melody-chord tonal distance (MCTD).

Application of machine learning in on-line calibration of flow measurement errors

With the vigorous construction of an intelligent pipe network, an automatic flowmeter was the key to realize online flow measurement of oil and gas. Ensuring the accuracy of flow measurement was very important to achieve real-time feedback on oil–gas production status for adjusting the oil recovery technology. However, due to the changing flow state of the oil–gas in the pipe and the error of the flow-meter itself, there were some errors during flow measurement. Therefore, based on the machine learning method, this paper corrected the errors caused by the pulsating flow and the flowmeter itself in data drift. The results showed that the toy–LSTM model can measure the pulsating flow accurately. When the number of toy–LSTM model neurons was ranged between 10 and 50, both RMSE and R2 showed good performance. The average of multiple predicted results presented higher accuracy than that of single predicted results, and furthermore, the drop rate can significantly increase the robustness of the toy–LSTM model. It should be noted that the optimal case proposed in this paper was based on a specific condition in the field. In the actual application, it was necessary to comprehensively consider the number of branch pipes, the change in oil well production, the characteristics of the oil–gas–water mixture, and other factors to determine the specific values of the number of neurons, the drop rate, and other parameters. Aiming at the distortion caused by the data drift of the flowmeter itself, the program was designed and data were collected from the software platform, information collector, and base station. The data acquisition of temperature–pressure integrated sensor and flowmeter was carried out in the field. In 20 groups of experiments, the online corrected program based on the Mexican Hat wavelet transform can realize accurate identification and automatic corrected responses, and the corrected time was within 5 min.

Data-Driven LSTM Model and Predictive Control for Vehicle Lateral Motion

Data-Driven LSTM Model and Predictive Control for Vehicle Lateral Motion

Stock prediction using Bi-LSTM and RoBERTa sentiment analysis

Stock price forecasting is a widespread research issue in the field of finance. Stock price prediction can give investors more accurate advice and yield higher returns. With the popularity of social media, more and more people choose to express their opinions, and more and more media outlets published news on social media. This paper mainly conducts stock analysis and prediction by presenting the RoBERTa-BiLSTM model with the Bahdanau attention mechanism. It consists of two existing models: the RoBERTa and bidirectional LSTM models. The experimental results show that this model is more accurate in predicting stock prices than the Multi-Layer Perception (MLP) and LSTM models. Inputs to the model include historical AAPL stock price data and sentiment data analysed by news and Twitter tweet sentiment. In this paper, the high-precision RoBERTa-BiLSTM model for stock prediction was proposed, which provides a solution for stock prediction. The practical applications and the typical problem of most studies about stock prediction called the time-shifted problem, are provided in this paper and also have research value.

Predicting financial enterprise stocks and economic data trends using machine learning time series analysis

This paper explores the application of machine learning in financial time series analysis, focusing on predicting trends in financial enterprise stocks and economic data. It begins by distinguishing stocks from stocks and elucidates risk management strategies in the stock market. Traditional statistical methods such as ARIMA and exponential smoothing are discussed in terms of their advantages and limitations in economic forecasting. Subsequently, the effectiveness of machine learning techniques, particularly LSTM and CNN-BiLSTM hybrid models, in financial market prediction is detailed, highlighting their capability to capture nonlinear patterns in dynamic markets. Finally, the paper outlines prospects for machine learning in financial forecasting, laying a theoretical foundation and methodological framework for achieving more precise and reliable economic predictions.

Novel optimized coupled rainfall model simulation based on stepwise decomposition technique.

Precipitation forecasting plays a pivotal role in guiding the effective management of regional water resources and providing crucial warnings for regional droughts and floods. Finding a monthly precipitation simulation model with robust fitting performance is a significant research endeavor in practical precipitation prediction. This paper introduces two modified African vulture optimization algorithms (MAVOA1 and MAVOA2). It provides hyperparameter optimization techniques for the least squares support vector machine (LSSVM), long short-term memory neural network (LSTM), and random forest (RF) models. These techniques are used to construct a monthly precipitation simulation model based on algorithmic optimization coupled with variational mode decomposition for full decomposition. The test results at five typical stations in the North China Plain reveal the following: (1) the LSSVM model demonstrates significantly better performance than the LSTM and RF models. (2) the MAVOA2-LSSVM model has the best-integrated effect: the average test fitting error is RMSE = 17.50 mm/month, MRE = 117.25%, NSE = 0.90, which shows its superiority in practical application and can significantly improve the accuracy of precipitation prediction; MAVOA2 is more suitable for machine learning models with more hyperparameters of its own, which provides a reference for hyperparameter optimization algorithms in the other fields.

Monthly climate prediction using deep convolutional neural network and long short-term memory

Climate change affects plant growth, food production, ecosystems, sustainable socio-economic development, and human health. The different artificial intelligence models are proposed to simulate climate parameters of Jinan city in China, include artificial neural network (ANN), recurrent NN (RNN), long short-term memory neural network (LSTM), deep convolutional NN (CNN), and CNN-LSTM. These models are used to forecast six climatic factors on a monthly ahead. The climate data for 72 years (1 January 1951–31 December 2022) used in this study include monthly average atmospheric temperature, extreme minimum atmospheric temperature, extreme maximum atmospheric temperature, precipitation, average relative humidity, and sunlight hours. The time series of 12 month delayed data are used as input signals to the models. The efficiency of the proposed models are examined utilizing diverse evaluation criteria namely mean absolute error, root mean square error (RMSE), and correlation coefficient (R). The modeling result inherits that the proposed hybrid CNN-LSTM model achieves a greater accuracy than other compared models. The hybrid CNN-LSTM model significantly reduces the forecasting error compared to the models for the one month time step ahead. For instance, the RMSE values of the ANN, RNN, LSTM, CNN, and CNN-LSTM models for monthly average atmospheric temperature in the forecasting stage are 2.0669, 1.4416, 1.3482, 0.8015 and 0.6292 °C, respectively. The findings of climate simulations shows the potential of CNN-LSTM models to improve climate forecasting. Climate prediction will contribute to meteorological disaster prevention and reduction, as well as flood control and drought resistance.

Forecasting of COVID-19 Cases Under Different Precaution Strategies in Turkey

The coronavirus disease started at the end of 2019 and affected all the countries in the world. In Turkey, the vaccination process started at the beginning of 2021 but performed in slow progress. Thus, the Turkish Government tried to implement precautions to control this virus's spread. In this study, we evaluated and compared five different forecasting models, ARIMA, Prophet, NARNN, Stacked LSTM, and Bidirectional LSTM, in order to show the effect of these precaution strategies on virus spread using a real-world data set. According to the test results, ARIMA and Prophet were found to be the most accurate models for small data sets that are split regarding precautions. Moreover, test results showed that when data size grows, LSTM model performance increases. However, these models' performance decreased when we fed these models by using the entire data set without splitting.

Real-time mechanical specific energy prediction for optimizing the drilling parameters using CNN-LSTM model

MSE is an important parameter in drilling process as it determines drilling efficiency. Therefore, accurately predicting the MSE during drilling and real-time adjusting the drilling parameters to enhance drilling efficiency have been persistent research topics. In this paper, based on Log Data, including WOB, ROP, RPMX and other parameters, and compares the ability of six machine learning methods to predict MSE. The findings indicate that the CNN-LSTM-Attention model combines the spatial feature extraction capability of CNN and the temporal sequence dependency learning ability of LSTM, while utilizing attention mechanisms to focus on important information. It predicts MSE more accurately than other models, exhibiting a better fit between predicted values and actual values compared to the traditional LSTM model, with a correlation coefficient (R2) of 0.93. This demonstrates the effectiveness of combining temporal and spatial features in predicting drilling parameters. Using the trained model, real-time optimization of drilling parameters is performed to search for the optimal MSE within a certain range, while outputting the corresponding RPM and WOB. Ultimately, the average MSE of the target well decreased by 35.6%. The research findings hold significant importance in reducing MSE and improving drilling efficiency.

Comparative Analysis of LSTM and ARIMA Models in Stock Price Prediction: A Technology Company Example

Stock price forecasting has been an important area of interest for economists and computer scientists. In addition to traditional statistical methods, advanced artificial intelligence techniques such as machine learning can stand out with their ability to process complex data sets and adapt to historical data. In recent years, hybrid models combining deep learning and time series methods have demonstrated superior performance in stock selection and portfolio optimisation. This study comparatively analyses the performance of LSTM and ARIMA models in time series forecasting. In the study, the stock prices of Oracle company are predicted using two different models, LSTM and ARIMA. Model performance is evaluated using metrics like MSE, MAE, RMSE, and MAPE. Both models have been found to be successful in different metrics. The LSTM model has lower error values; meanwhile, the ARIMA model produced proportionally more accurate forecasts. The study concludes that given the potential offered by deep learning, models such as LSTM are essential for time series forecasting. The flexibility of deep learning allows the development of customized models for different data types and time series problems.

Prediction and analysis of sea surface temperature based on LSTM-transformer model

This paper introduces a novel method for predicting sea surface temperature (SST) using a hybrid LSTM-Transformer model. The study utilizes the ERA5 dataset for SST prediction at six specific locations near China. The LSTM-Transformer model combines the temporal processing capability of LSTM with the efficient data processing power of Transformer, showing superior performance in reducing Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE), as well as improving the R-squared (R²) value, compared to standalone LSTM, Transformer, and traditional Logistic Regression (LR) models. This is particularly evident in spring and autumn, indicating its robustness to seasonal changes. The model's performance varies across different geographic locations, with lower prediction errors in low-latitude and open sea areas, attributed to the less complex environmental dynamics compared to continental shelf areas. Overall, the LSTM-Transformer hybrid model presents a significant advancement in SST prediction, providing important implications for fisheries, meteorology, and climate change research.

Long-term prediction of Iranian blood product supply using LSTM: a 5-year forecast

BackgroundThis study aims to predict the trend of procurement and storage of various blood products, as well as planning and monitoring the consumption of blood products in different centers across Iran based on artificial intelligence until the year 2027.MethodsThis research constitutes a time-series investigation within the realm of longitudinal studies. In this study, information on the number of packed red blood cells (RBC), leukoreduced red blood cells (LR-RBC), and platelets (PLT), PLT-Apheresis, and fresh frozen plasma (FFP) was requested from all blood transfusion centers in the country and extracted using a unified protocol. After the initial examination of the information and addressing data issues and inconsistencies, the corrected data were analyzed. Both conventional and artificial intelligence approaches were used to predict each product in this study. The best model was selected based on goodness-of-fit indicators RMSE and MAPE.ResultsBased on the obtained results, the FFP product will follow a relatively consistent process similar to previous years in the next five years. The PLT product is predicted to have a growing trend over the next 5 years, which applies to both the demand and supply of the product. The PLT-Apheresis product also shows a similar upward trend, albeit with a lower growth rate. The RBC product will have a constant trend over a 5-year period (long-term) according to both models, taking into account short-term changes. Similarly, there is a similar trend in LR-RBC, with the expectation that short-term pattern repetition will continue over a 5-year period (long-term). Comparing the goodness-of-fit results, the LSTM model proved to be better for predicting the dominant blood products.ConclusionsThe growth of the elderly population and diseases related to old age, and on the other hand, the trend of increasing the consumption of the product with a short lifespan (PLT) requires the activation of the management of the patient’s blood, especially in relation to this product in medical centers. The trend for other products in the next five years is similar to previous years, and no growth in demand is observed. The LSTM method, considering periodic and cyclical events, has performed the prediction.

Strain Prediction for High-Speed Rail Canopies in Cold Regions Based on LSTM Models

Strain Prediction for High-Speed Rail Canopies in Cold Regions Based on LSTM Models

Optimized Deep Learning Model Architecture for the Feature Extraction to Predict Trend in Stock Market

Predicting stock trends is a complex task influenced by various factors such as market sentiment, economic indicators, and company performance. Analysts often employ technical analysis, studying historical price patterns and trading volumes, as well as fundamental analysis, assessing financial statements and industry trends. Deep Learning models have also gained popularity for predicting stock trends, using algorithms to identify patterns and relationships in large datasets. Deep learning algorithms, particularly neural networks, excel at recognizing intricate patterns and relationships within complex datasets, making them well-suited for predicting stock prices, identifying trends, and managing risk. Hence, this paper proposed a Bird Swarm Optimization ARIMA LSTM (BSO-ARIMA-DL) model for stock trend prediction. The proposed BSO-ARIMA-DL model performance is applied in the company datasets Apple, Amazon, and Infosys for stock trend prediction. With the proposed BSO-ARIMA-DL model features are optimized for the identification of features in the dataset for the evaluation of optimal features. Upon the estimation of features, the ARIMA model with the LSTM architecture is implemented for the stock trend analysis. The proposed BSO-ARIMA-DL model deep learning model is implemented for the stock trend prediction in the companies. The results demonstrated that the proposed BSO-ARIMA-DL model exhibits a minimal error of ~10% minimal to the conventional ARIMA model.