LSTM Recurrent Neural Network Research Articles

A hybrid model based on LSTM neural networks with attention mechanism for short-term wind power forecasting

Wind power plants have gained prominence in recent decades owing to their positive environmental and economic impact. However, the unpredictability of wind resources poses significant challenges to the secure and stable operation of the power grid. To address this challenge, numerous computational and statistical methods have been proposed in the literature to forecast short-term wind power generation. However, the demand for more accurate and reliable methodologies to tackle this problem remains. In this context, this paper proposes a new hybrid framework that combines a statistical pre-processing stage with an attention-based deep learning approach to overcome the shortcomings of existing forecasting strategies in accurately predicting multi-seasonal wind power time series. The proposed ensemble model involves a data transformation stage that normalizes the data distribution, along with modeling and removing multiple seasonal patterns from the historical time-series. Considering these results, the proposed model further incorporates an LSTM Recurrent Neural Network (RNN) model with an attention mechanism, for each month of the year, to better capture the relevant temporal dependencies in the input residuals sequence. The model was trained and evaluated on hourly wind power data obtained from the Spanish electricity market, spanning the period from 2008 to 2019. Experimental results show that the proposed model outperforms well-established DL-based models, achieving lower error metrics. These findings have potential applications in energy trading, grid planning, and renewable energy management.

Improving the understanding of web user behaviors through machine learning analysis of eye-tracking data

Eye-tracking techniques are widely used to analyze user behavior. While eye-trackers collect valuable quantitative data, the results are often described in a qualitative manner due to the lack of a model that interprets the gaze trajectories generated by routine tasks, such as reading or comparing two products. The aim of this work is to propose a new quantitative way to analyze gaze trajectories (scanpaths) using machine learning. We conducted a within-subjects study (N = 30) testing six different tasks that simulated specific user behaviors in web sites (attentional, comparing two images, reading in different contexts, and free surfing). We evaluated the scanpath results with three different classifiers (long short-term memory recurrent neural network—LSTM, random forest, and multilayer perceptron neural network—MLP) to discriminate between tasks. The results revealed that it is possible to classify and distinguish between the 6 different web behaviors proposed in this study based on the user’s scanpath. The classifier that achieved the best results was the LSTM, with a 95.7% accuracy. To the best of our knowledge, this is the first study to provide insight about MLP and LSTM classifiers to discriminate between tasks. In the discussion, we propose practical implications of the study results.

Recurrent neural network modeling of multivariate time series and its application in temperature forecasting.

Temperature forecasting plays an important role in human production and operational activities. Traditional temperature forecasting mainly relies on numerical forecasting models to operate, which takes a long time and has higher requirements for the computing power and storage capacity of computers. In order to reduce computation time and improve forecast accuracy, deep learning-based temperature forecasting has received more and more attention. Based on the atmospheric temperature, dew point temperature, relative humidity, air pressure, and cumulative wind speed data of five cities in China from 2010 to 2015 in the UCI database, multivariate time series atmospheric temperature forecast models based on recurrent neural networks (RNN) are established. Firstly, the temperature forecast modeling of five cities in China is established by RNN for five different model configurations; secondly, the neural network training process is controlled by using the Ridge Regularizer (L2) to avoid overfitting and underfitting; and finally, the Bayesian optimization method is used to adjust the hyper-parameters such as network nodes, regularization parameters, and batch size to obtain better model performance. The experimental results show that the atmospheric temperature prediction error based on LSTM RNN obtained a minimum error compared to using the base models, and these five models obtained are the best models for atmospheric temperature prediction in the corresponding cities. In addition, the feature selection method is applied to the established models, resulting in simplified models with higher prediction accuracy.

Predicting the Price of Bitcoin using LSTM Recurrent Neural Network

In this paper, we tried to estimate the Bitcoin price precisely taking into consideration various parameters that affect the Bitcoin value. In our work, we pointed to understand and identify daily changes in the Bitcoin market while obtaining insight into most appropriate features surrounding Bitcoin price. We will predict the daily price change with highest possible accuracy. The market capitalization of publicly traded cryptocurrencies is currently above $230 billion. Bitcoin, the most valuable cryptocurrency, serves primarily as a digital store of value, and its price predictability has been well-studied. For the first phase of our investigation, we aim to understand and identify daily trends in the Bitcoin market while gaining insight into optimal features surrounding Bitcoin price. Our data set consists of various features relating to the Bitcoin price and payment network over the course of five years, recorded daily. For the second phase of our investigation, using the available information, we will predict the sign of the daily price change with highest possible accuracy with deep learning algorithm such as long short term memory for greater accuracy. Compared with benchmark results for daily price prediction, we achieve a better performance, with the highest accuracies of the statistical methods and deep learning algorithms. Deep Learning models includes Long Short-Term Memory in RNN for Bitcoin price prediction are superior to statistical methods

Status Quo and Influence of Martial Arts Fitness in Pursuit of Health Using LSTM Recurrent Neural Network Algorithm

With the acceleration of the socialist modernization process, people’s living standards continue to improve, and the modernization of cities is also developing rapidly. In the process of urban construction, a series of problems such as how to deal with the relationship between citizens and the surrounding environment, how to carry out citizens' health activities, and how to organize citizens to exercise healthily are becoming more and more obvious. With the introduction of the National Sports Outline, as an important sports activity, Chinese martial arts has a long history and profound cultural heritage, and it still plays an irreplaceable role in promoting the national spirit and building a harmonious society today. Therefore, this paper introduced the LSTM recurrent neural network algorithm to collect and extract the current data on martial arts fitness in the pursuit of health. By constructing the LSTM neuron structure and building the LSTM recurrent neural network structure, a state estimation based on LSTM feature extraction was proposed. The algorithm used the memory ability of LSTM to extract the relevant features of the data stream, which significantly improved the accuracy of the data stream collection. The experimental results showed that the time required by the method in this paper were 12.7, 10.2, 12.4 s and 11.8 min respectively, and the accuracy rates were 98.78, 98.26, 99.03 and 97.89%, respectively, which were greatly improved compared with the existing methods.

Early Identification and Localization Algorithm for Weak Seedlings Based on Phenotype Detection and Machine Learning

It is important to propose the correct decision for culling and replenishing seedlings in factory seedling nurseries to improve the quality of seedlings and save resources. To solve the problems of inefficiency and subjectivity of the existing traditional manual culling and replenishment of seeds, this paper proposes an automatic method to discriminate the early growth condition of seedlings. Taking watermelon plug seedlings as an example, Azure Kinect was used to collect data of its top view three times a day, at 9:00, 14:00, and 19:00. The data were collected from the time of germination to the time of main leaf growth, and the seedlings were manually determined to be strong or weak on the last day of collection. Pre-processing, image segmentation, and point cloud processing methods were performed on the collected data to obtain the plant height and leaf area of each seedling. The plant height and leaf area on the sixth day were predicted using an LSTM recurrent neural network for the first three days. The R squared for plant height and leaf area prediction were 0.932 and 0.901, respectively. The dichotomous classification of normal and abnormal seedlings was performed using six machine learning classification methods, such as random forest, SVM, and XGBoost, for day six data. The experimental results proved that random forest had the highest classification accuracy of 84%. Finally, the appropriate culling and replenishment decisions are given based on the classification results. This method can provide some technical support and a theoretical basis for factory seedling nurseries and transplanting robots.

Uncertainty-aware and explainable machine learning for early prediction of battery degradation trajectory

We present an interpretable uncertainty-aware machine learning model to predict battery degradation trajectories. Using LSTM Recurrent Neural Networks, we reach an RMSE of 106 and MAPE of 10.6%.

Retracted: Design and Implementation of Human Motion Recognition Information Processing System Based on LSTM Recurrent Neural Network Algorithm.

[This retracts the article DOI: 10.1155/2021/3669204.].

Forecasting Steel Production in the World—Assessments Based on Shallow and Deep Neural Networks

Forecasting algorithms have been used to support decision making in companies, and it is necessary to apply approaches that facilitate a good forecasting result. The present paper describes assessments based on a combination of different neural network models, tested to forecast steel production in the world. The main goal is to find the best machine learning model that fits the steel production data in the world to make a forecast for a nine-year period. The study is important for understanding the behavior of the models and sensitivity to hyperparameters of convolutional LSTM and GRU recurrent neural networks. The results show that for long-term prediction, the GRU model is easier to train and provides better results. The article contributes to the validation of the use of other variables that are correlated with the steel production variable, thus increasing forecast accuracy.

Research on quantitative trading strategy based on prediction and dynamic programming model

Today, the digital currency market represented by bitcoin has become an important part of the capital market. As the COVID-19 pandemic has increased volatility in capital markets in the past two years, effective trend forecasting and risk-averse investment strategies can bring substantial profits to companies and individuals and reduce the possibility of financial risks. To achieve the above objectives, we established the following models: (1)price prediction models of gold and bitcoin; (2)trading market risk judgment model; (3)revenue valuation model based on dynamic programming. This paper uses ARIMA and LSTM neural network to establish the prediction model. After checking the historical data, the best parameter fitting values of ARIMA model were obtained by using the exhaustive method according to BIC criterion, which were ARIMA (4,1,3) (gold) and ARIMA (2,1,2) (bitcoin) respectively. Then LSTM recurrent neural network is used to modify the forecast results, and the price forecast graph with high fitting degree with the real data is obtained. Then, this paper combines the technical indicators commonly used in the financial market to form a market risk judgment model. On the basis of the eight laws of buying and selling, the weight index is introduced to put forward the bull and bear market judgment rules of gold and bitcoin trading, and the tendency distribution map of the market environment in the past five years is obtained. At the same time, the buying risk coefficient is quantified by combining the maximum retraction rate. On the basis of the first two sub-models, this paper establishes a revenue valuation model based on the idea of dynamic programming. By setting the policy trigger threshold to control the risk, the problem that local optimal solution may appear in the model is improved.

GLCM and LSTM Recurrent Neural Networks Integrated with Machine Learning Techniques to Identify Plant Disease

Plant diseases are very impactful towards the overall effectiveness and quality management of the agricultural sector. In recent years, deep learning methods have been used as a way to identify these diseases, based on neural networks. The study presents GLCM and LSTM Recurrent Neural Networks Integrated with Machine Learning towards the identification of plant diseases. It has been found that the process is very accurate and can assess diverse plants disease characteristics dataset as well.

Short-term traffic prediction using physics-aware neural networks

In this work, we propose an algorithm performing short-term predictions of the flow and speed of vehicles on a stretch of road, using past measurements of these quantities. This algorithm is based on a physics-aware recurrent neural network. A discretization of a macroscopic traffic flow model (using the so-called Traffic Reaction Model) is embedded in the architecture of the network and yields traffic state estimations and predictions for the flow and speed of vehicles, which are physically-constrained by the macroscopic traffic flow model and based on estimated and predicted space–time dependent traffic parameters. These parameters are themselves obtained using a succession of LSTM recurrent neural networks. The algorithm is tested on raw flow measurements obtained from loop detectors.

Forecasting Crude Oil Consumption in Poland Based on LSTM Recurrent Neural Network

Primary fuels, i.e., crude oil, natural gas, and power coal, dominate the total global demand for primary energy. Among them, crude oil plays a particularly important role due to the universality of applications and the practical lack of substitutes in transport. Crude oil is also one of the main sources of primary energy in Poland and accounts for around 30% of the energy consumed. Poland covers only 3% of its needs from domestic deposits. The rest is imported from Russia, Saudi Arabia, Nigeria, Great Britain, Kazakhstan, and Norway. Due to such a high import of raw material, Poland must anticipate future demand. On the one hand, this article aims to analyze the current (2020) and future (2040) crude oil consumption on the Polish market. The study analyzes the geopolitical and economic foundations of the functioning of the energy raw-materials market, the crude oil supply, the structure of Poland’s energy mix, and assumptions about the energy policy until 2040. On the other hand, conclusions from the research were used to build a model of crude oil consumption for the internal market. It has been also shown that the consumption of crude oil on the Polish market is a nonlinear phenomenon with a small set of statistical data, which makes it difficult to build an accurate model. This paper proposes a new model based on artificial neural networks that includes long-term memory (LSTM). The accuracy of the constructed model was assessed using the MSE, Theil, and Janus coefficients. The results show that LSTM models can be used to forecast crude oil consumption, and they cope with the nonstationary and nonlinear time series. Many important contemporary problems posed in the field of energy economy are also discussed, and it is proposed to solve them with the use of modern machine-learning tools.

Speaker Recognition of Fiber-Optic External Fabry-Perot Interferometric Microphone Based on Deep Learning

In order to realize speaker recognition in flammable, explosive and strong electromagnetic interference environments, the speaker recognition technology of fiber-optic external Fabry-Perot interferometric (EFPI) microphone based on Deep learning is studied in this paper. The combination of convolutional neural network and long short-term memory (CNN-LSTM) model is proposed for speaker recognition in EFPI acoustic sensing system. The intensity self-compensation method is used to demodulate the voice signals of different speakers, then, the spectral features of voice signals are divided into training set and test set. The training set are input into the established CNN-LSTM model for model training, and the test set are input into the trained model for testing. The experimental results show that the CNN-LSTM model proposed in this paper has good training accuracy and test accuracy, reaching 100% and 94.0% respectively. Comparing with CNN and LSTM recurrent neural network (LSTM-RNN), the test accuracy of this model is 11.1% and 13.4% higher respectively.

Forecasting NIFTY 50 in Volatile Markets Using RNN–LSTM: A Study on the Performance of Neural Network Models During the COVID-19 Pandemic

The COVID-19 pandemic has shown us how the market can be highly uncertain and volatile at certain times. This brings a new level of challenges to all the investors and active traders in the market, as they have not seen such a movement in the past. However, as technology is evolving, highly sophisticated tools and techniques are being used by hedge funds and other investment banks to track down these movements and turn this into an opportunity. In this paper, we try to analyse how recurrent neural network (RNN) with long- and short-term memory architecture performs under volatile market conditions. For this study, we tried to perform a comparative analysis between two models within two successive time periods, where one is trained in a volatile market condition and the other in a relatively low volatile market condition. The results showed that the RNN model is less accurate in predicting the prices in a volatile market compared to a relatively low volatile market. We also compared these two models to a separate model where we trained using the combined data from the two successive time periods. Even though the addition in data points for the neural network produced a better result compared to the model trained under volatile conditions, it did not significantly perform better than the model, which was trained in the low volatile period.

Ensemble Machine Learning Model for Accurate Air Pollution Detection Using Commercial Gas Sensors.

This paper presents the results on developing an ensemble machine learning model to combine commercial gas sensors for accurate concentration detection. Commercial gas sensors have the low-cost advantage and become key components of IoT devices in atmospheric condition monitoring. However, their native coarse resolution and poor selectivity limit their performance. Thus, we adopted recurrent neural network (RNN) models to extract the time-series concentration data characteristics and improve the detection accuracy. Firstly, four types of RNN models, LSTM and GRU, Bi-LSTM, and Bi-GRU, were optimized to define the best-performance single weak models for CO, O3, and NO2 gases, respectively. Next, ensemble models which integrate multiple single weak models with a dynamic model were defined and trained. The testing results show that the ensemble models perform better than the single weak models. Further, a retraining procedure was proposed to make the ensemble model more flexible to adapt to environmental conditions. The significantly improved determination coefficients show that the retraining helps the ensemble models maintain long-term stable sensing performance in an atmospheric environment. The result can serve as an essential reference for the applications of IoT devices with commercial gas sensors in environment condition monitoring.

A real time prediction methodology for hurricane evolution using LSTM recurrent neural networks

A real time prediction methodology for hurricane evolution using LSTM recurrent neural networks

Employing a Long-Short-Term Memory Neural Network to Improve Automatic Sleep Stage Classification of Pharmaco-EEG Profiles

An increasing problem in today’s society is the spiraling number of people suffering from various sleep disorders. The research results presented in this paper support the use of a novel method that employs techniques from the classification of sleep disorders for more accurate scoring. Applying this novel method will assist researchers with better analyzing subject profiles for recommending prescriptions or to alleviate sleep disorders. In biomedical research, the use of animal models is required to experimentally test the safety and efficacy of a drug in the pre-clinical stage. We have developed a novel LSTM Recurrent Neural Network to process Pharmaco-EEG Profiles of rats to automatically score their sleep–wake stages. The results indicate improvements over the current methods; for the case of combined channels, the model accuracy improved by 1% and 3% in binary or multiclass classifications, respectively, to accuracies of 93% and 82%. In the case of using a single channel, binary and multiclass LSTM models for identifying rodent sleep stages using single or multiple electrode positions for binary or multiclass problems have not been evaluated in prior literature. The results reveal that single or combined channels, and binary or multiclass classification tasks, can be applied in the automatic sleep scoring of rodents.

ISeeU2: Visually interpretable mortality prediction inside the ICU using deep learning and free-text medical notes

Accurate mortality prediction allows Intensive Care Units (ICUs) to adequately benchmark clinical practice and identify patients with unexpected outcomes. Traditionally, simple statistical models have been used to assess patient death risk, many times with sub-optimal performance. On the other hand Deep Learning holds promise to positively impact clinical practice by leveraging medical data to assist diagnosis and prediction, including mortality prediction. However, as the question of whether powerful Deep Learning models attend correlations backed by sound medical knowledge when generating predictions remains open, additional interpretability tools are needed to foster trust and encourage the use of AI by clinicians. In this work we show an interpretable Deep Learning model trained on MIMIC-III to predict mortality inside the ICU using raw nursing notes, together with visual explanations for word importance based on the Shapley Value. Our model reaches a ROC of 0.8629 (±0.0058), outperforming the traditional SAPS-II score and a LSTM recurrent neural network baseline while providing enhanced interpretability when compared with similar Deep Learning approaches. Supporting code can be found at https://github.com/williamcaicedo/ISeeU2.

A Current Sharing State Estimation Method of Redundant Switched-Mode Power Supply Based on LSTM Neural Network

Redundant Switched-mode Power supplies (SMPSs) are commonly used to improve electronic systems’ reliability, and accurate estimation of the current sharing state is significant for evaluating the system’s health. Currently, the current sharing state estimation is mainly realized by using current sensors to detect each branch’s current, and the deployment and maintenance costs are high. In this paper, a method for power supply current sharing state estimation based on LSTM recurrent neural network is proposed. By taking advantage of subtle differences in the inherent spectral characteristics of SMPSs, this method only needs to detect the voltage ripple at the switching frequency of the load terminal to estimate the output current of each power supply branch. The verification experiment on the three-power redundant experimental platform shows that the estimation error is less than 10%. The method has the characteristics of simple structure, non-invasion, convenient deployment and maintenance, so it has high application and promotion value.