Convolutional Neural Network LSTM Research Articles

Forecasting Bitcoin volatility using machine learning techniques

This paper studies the Bitcoin volatility forecasting performance between popular traditional econometric models and machine learning techniques. We compare the 1-day to 2-month ahead forecasting performance of the Long Short-Term Memory (LSTM) and a hybrid Convolutional Neural Network-LSTM (CNN-LSTM) model to the traditional models. We find that neural networks outperform Generalised Autoregressive Conditional Heteroskedasticity (GARCH) models for all forecasting horizons. Furthermore, the LSTM model outperforms the Heterogeneous Autoregressive (HAR) model and by integrating the Markov Transition Field (MTF) into the CNN-LSTM model, we achieve superior forecasting results in the short-term, particularly for the 7-day forecasts.

Data-driven strategy for state of health prediction and anomaly detection in lithium-ion batteries

This study addresses the crucial challenge of monitoring the State of Health (SOH) of Lithium-Ion Batteries (LIBs) in response to the escalating demand for renewable energy systems and the imperative to reduce CO2 emissions. The research introduces deep learning (DL) models, namely Encoder-Long Short-Term Memory (E-LSTM) and Convolutional Neural Network-LSTM (CNN–LSTM), each designed to forecast battery SOH. E-LSTM integrates an encoder for dimensionality reduction and an LSTM model to capture data dependencies. CNN–LSTM, on the other hand, employs CNN layers for encoding followed by LSTM layers for precise SOH estimation. Significantly, we prioritize model explainability by employing a game-theoretic approach known as SHapley Additive exPlanations (SHAP) to elucidate the output of our models. Furthermore, a method based on pattern mining was developed, synergizing with the model, to identify patterns contributing to abnormal SOH decrease. These insights are presented through informative plots. The proposed approach relies on the battery dataset from the Massachusetts Institute of Technology (MIT) and showcases promising results in accurately estimating SOH values, in which the E-LSTM model outperformed the CNN–LSTM model with a Mean Absolute Error (MAE) of less than 1%.

Application of Attention-Based LSTM Hybrid Models for Stock Price Prediction

The stock market plays a pivotal role in the national economy, while the application of artificial intelligence (AI) in stock price prediction has gained traction. This paper evalu-ates the performance of five advanced deep learning (DL) models: Long Short-Term Memory (LSTM), Self-attention, Convolutional Neural Network-LSTM with attention (CNN-LSTM-attention), Gated Recurrent Unit-LSTM with attention (GRU-LSTM-attention), and CNN-Bidirectional LSTM-GRU with attention (CNN-BiLSTM-GRU-attention), utilizing a decade of data on Amazons closing prices. Our results show that the CNN-BiLSTM-GRU-attention model exhibits superior performance, achieving a root mean square error (RMSE) of 1.054589 and a coefficient of determination (R2) of 0.970123, indicative of its proficiency in handling intricate financial data. This papers significance lies in its validation of the effectiveness of attention-based ensemble models in stock market prediction, as well as the introduction of the innovative application of the CNN-BiLSTM-GRU-attention model in financial forecast-ing, which holds potential for wide-ranging applicability.

A Time Series Prediction Model for Wind Power Based on the Empirical Mode Decomposition–Convolutional Neural Network–Three-Dimensional Gated Neural Network

In response to the global challenge of climate change and the shift away from fossil fuels, the accurate prediction of wind power generation is crucial for optimizing grid operations and managing energy storage. This study introduces a novel approach by integrating the proportional–integral–derivative (PID) control theory into wind power forecasting, employing a three-dimensional gated neural (TGN) unit designed to enhance error feedback mechanisms. The proposed empirical mode decomposition (EMD)–convolutional neural network (CNN)–three-dimensional gated neural network (TGNN) framework starts with the pre-processing of wind data using EMD, followed by feature extraction via a CNN, and time series forecasting using the TGN unit. This setup leverages proportional, integral, and differential control within its architecture to improve adaptability and response to dynamic wind patterns. The experimental results show significant improvements in forecasting accuracy; the EMD–CNN–TGNN model outperforms both traditional models like autoregressive integrated moving average (ARIMA) and support vector regression (SVR), and similar neural network approaches, such as EMD–CNN–GRU and EMD–CNN–LSTM, across several metrics including mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE), and coefficient of determination (R2). These advancements substantiate the model’s effectiveness in enhancing the precision of wind power predictions, offering substantial implications for future renewable energy management and storage solutions.

Short-Term Electric Load Forecasting Based on Signal Decomposition and Improved TCN Algorithm

In the realm of power systems, short-term electric load forecasting is pivotal for ensuring supply–demand balance, optimizing generation planning, reducing operational costs, and maintaining grid stability. Short-term load curves are characteristically coarse, revealing high-frequency data upon decomposition that exhibit pronounced non-linearity and significant noise, complicating efforts to enhance forecasting precision. To address these challenges, this study introduces an innovative model. This model employs complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) to bifurcate the original load data into low- and high-frequency components. For the smoother low-frequency data, a temporal convolutional network (TCN) is utilized, whereas the high-frequency components, which encapsulate detailed load history information yet suffer from a lower fitting accuracy, are processed using an enhanced soft thresholding TCN (SF-TCN) optimized with the slime mould algorithm (SMA). Experimental tests of this methodology on load forecasts for the forthcoming 24 h across all seasons have demonstrated its superior forecasting accuracy compared to that of non-decomposed models, such as support vector regression (SVR), recurrent neural network (RNN), gated recurrent unit (GRU), long short-term memory (LSTM), convolutional neural network-LSTM (CNN-LSTM), TCN, Informer, and decomposed models, including CEEMDAN-TCN and CEEMDAN-TCN-SMA.

Deep learning and tree-based models for earth skin temperature forecasting in Malaysian environments

Predicting the Earth Skin Temperature (TS) using artificial intelligence (AI) has the potential to offer valuable insights into environmental changes and their impacts. TS has significant nonlinearity due to several meteorological parameters, including average temperature, maximum temperature, minimum temperature, relative humidity, surface pressure, wind speed, and wind direction. This study introduced four machine learning (ML) models, namely solo Long Short-Term Memory (LSTM), Autoencoder-LSTM, Classification and Regression Tree (CART), and a hybrid Convolutional Neural Network-LSTM (CNN-LSTM), to predict daily averaged TS at different locations over the Malaysian region. In the first stage of the modeling development, Pearson Correlation (PC) was adopted to measure the strength and direction of the relationship between input and output variables. The statistical analysis and visual interpretations demonstrated that the Autoencoder-LSTM model outperformed the CNN-LSTM, LSTM, and CART models for each simulated city. The Autoencoder-LSTM model showed outstanding performance at Kuantan in comparison with the seven cities, achieving a coefficient of determination (R2 = 0.965), Root Mean Square Error (RMSE = 0.183), Mean Absolute Error (MAE = 0.142), and Nash-Sutcliffe Efficiency (NSE = 0.96). The research findings suggested that the coordinates of each station are crucial in determining the level of data randomness, which ultimately affects the learning process of the ML models.

Carbon trading price forecasting based on parameter optimization VMD and deep network CNN–LSTM model

To meet carbon peak and neutrality targets, accurate carbon trading price forecasting is very important for enterprises making emission reduction decisions. By fusing convolutional neural network (CNN) and long short-term memory network (LSTM), the CNN–LSTM model is constructed. After variational mode decomposition (VMD), several intrinsic mode functions (IMFs) components are obtained and input into the CNN–LSTM model, thus constructing the combined sooty tern optimization algorithm (STOA)–VMD–CNN–LSTM forecasting model. To test this model, the carbon trading prices of the carbon emission trading markets of Hubei, Guangdong and Shenzhen were forecast. The prediction performance of the STOA–VMD–CNN–LSTM model is compared with ARIMA, BP, CNN and LSTM benchmark models and models combining different decomposition technologies. The international carbon trading price (EUR and CER) is used for prediction. Compared with other methods, the developed model makes fewer errors and achieves superior performance. Several important implications are provided for investors and risk managers involved in carbon financial products.

Pavement freezing depth estimation using hybrid deep-learning models

Predicting pavement temperature by depth is crucial for road design, analysis, and maintenance. However, current methods predominantly utilize regression and/or open-form solutions focusing on highways. Additionally, most machine-learning models focus on asphalt layers and do not extend to deeper pavement layers. Therefore, this study provides deep-learning models using weather parameters to predict pavement temperature from surface to sublayers and estimate pavement freezing depth for developing massive apartment complexes. Temperature-by-depth data collected from thin pavements from three locations in South Korea were used. Comparative analyses of long short-term memory (LSTM), convolutional neural network-LSTM (CNN-LSTM), and convolutional LSTM were performed. Results showed that CNN-LSTM model performed better with coefficients of determination ( R2) of 0.965, 0.987, and 0.981. Additionally, the CNN-LSTM predicted freezing depth with 0.3%–13.1% error margins outperforming the LSTM, Aldrich’s, and Korean Ministry of Transport approaches. The proposed approach shows that deep-learning models better estimate the freezing depth of pavements than existing approaches.

Hydrogen leakage location prediction at hydrogen refueling stations based on deep learning

Accurate and efficient localization of hydrogen leakage is crucial for ensuring the safe and stable operation of hydrogen refueling stations. In this paper, a hybrid model (CEEMDAN–CNN–LSTM) based on data noise reduction and deep learning is proposed to predict the location of hydrogen leakage in hydrogen refueling stations. The model employs the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) algorithm to effectively reduce noise and process 720 hydrogen leakage condition datasets. The CNN-LSTM model is constructed by synergistically combining the spatiotemporal feature extraction capabilities of the convolutional neural network (CNN) with the temporal processing capabilities of the long-short-term memory neural network (LSTM) while optimizing hyperparameter combinations through GridSearch. Compared with other models, the proposed CEEMDAN–CNN–LSTM model demonstrates superior prediction performance, achieving an accuracy of 99.54 %, precision of 99.42 %, recall of 99.54 %, and F1-score of 99.46 %. Moreover, when utilizing a 5-s duration of leakage data as input, the model attains an impressive accuracy of 97.64 %. These results underscore the favorable application prospects of the CEEMDAN–CNN–LSTM model in the field of gas leak localization.

Prediction of railroad track geometry change using a hybrid CNN-LSTM spatial-temporal model

Track geometry is highly important for ensuring railroad safety. Predicting track geometry degradation can support preventive maintenance by identifying and prioritizing track segments that are more prone to potential track geometry defects. This paper develops a novel machine learning approach that can simultaneously geospatially align track geometry data from multiple inspections accounting for positional errors, and also predict foot-by-foot track geometry change over time. The proposed position correction method considers multiple geometry parameters to provide highly accurate positional information for geometry measurements, which is a prerequisite for foot-by-foot track geometry prediction. A hybrid CNN (convolutional neural network)-LSTM (long short-term memory neural network) machine learning model is developed to account for spatial–temporal dependence with respect to foot-by-foot track geometry change. Specifically, CNN is used to incorporate spatial dependence of track geometry on adjacent segments. LSTM is applied to learn dynamic changes in track geometry data over time to consider the temporal dependence. The hybrid CNN-LSTM model is validated using track inspection data provided by one major freight railroad in the United States. The results show that our positional error correction method can reduce the relative positional error to less than 1 foot with a 99% confidence interval. Our proposed CNN-LSTM model outperforms four other models, including a naïve model (using the last observation), multi-layer perceptron (MLP), plain CNN, and plain LSTM, for both short-term and long-term prediction periods. The proposed hybrid machine learning methodology can be adapted to various other freight and passenger rail lines. The predictive track geometry change information can be used by the industry to plan and prioritize resources for preventative maintenance, yielding benefits in safety and operational efficiency.

A CNN–LSTM Machine-Learning Method for Estimating Particulate Organic Carbon from Remote Sensing in Lakes

As particulate organic carbon (POC) from lakes plays an important role in lake ecosystem sustainability and carbon cycle, the estimation of its concentration using satellite remote sensing is of great interest. However, the high complexity and variability of lake water composition pose major challenges to the estimation algorithm of POC concentration in Class II water. This study aimed to formulate a machine-learning algorithm to predict POC concentration and compare their modeling performance. A Convolutional Neural Network–Long Short-Term Memory (CNN–LSTM) algorithm based on spectral and time sequences was proposed to construct an estimation model using the Sentinel 2 satellite images and water surface sample data of Chaohu Lake in China. As a comparison, the performances of the Backpropagation Neural Network (BP), Generalized Regression Neural Network (GRNN), and Convolutional Neural Network (CNN) models were evaluated for remote sensing inversion of POC concentration. The results show that the CNN–LSTM model obtained higher prediction precision than the BP, GRNN, and CNN models, with a coefficient of determination (R2) of 0.88, a root mean square error (RMSE) of 3.66, and residual prediction deviation (RPD) of 3.03, which are 6.02%, 22.13%, and 28.4% better than the CNN model, respectively. This indicates that CNN–LSTM effectively combines spatial and temporal information, quickly captures time-series features, strengthens the learning ability of multi-scale features, is conducive to improving estimation precision of remote sensing models, and offers good support for carbon source monitoring and assessment in lakes.

Spatio-temporal heterogeneous graph using multivariate earth observation time series: Application for drought forecasting

Accurate forecasting is required for the effective risk management of drought disasters. Many machine learning- and deep learning-based models have been proposed for drought forecasting, however, they cannot handle the temporal and/or spatial dependencies in the input data, causing unexpected forecasting results. In order to solve the challenging issue, in this paper we proposed the Heterogeneous Spatio-Temporal Graph (HetSPGraph), for drought forecasting. It includes three major layers: spatial aggregations including inter and intra aggregations, temporal aggregation, and a forecasting network. The main function of HetSPGraph is to learn the dynamic spatiotemporal correlations between the regions and to further predict the drought in different regions, based on which accurate drought forecasting can be achieved. Experimental forecasting results of the Standardized Precipitation Evapotranspiration Index (SPEI) in China indicated that the HetSPGraph model outperformed the traditional baseline methods including the Long Short-Term Memory model (LSTM), Convolutional Neural Network-LSTM (CNN-LSTM), Gated Recurrent Unit (GRU), Spatio-Temporal Graph Convolutional Networks (STGCN) and Geographic-Semantic-Temporal Hypergraph Convolutional Network (GST-HCN). Even for long-term forecasting (12 months), more accurate forecasting results, with the coefficient of determination R2 higher than 0.89, can also be obtained by HetSPGraph compared to the other three models. The proposed HetSPGraph model has the potential for wider use in forecasting drought and other natural disasters.

Weather image-based short-term dense wind speed forecast with a ConvLSTM-LSTM deep learning model

Short-term wind speed predication is of great significance for scholars (e.g., understanding wind profiles), practitioners (e.g., building energy management), regulators (e.g., urban microclimate regulation), and even the general public. Current wind speed forecasting methods either generate sparse predictions or occur high cost. This paper reports a novel, inexpensive framework to forecast urban local dense wind speed. The central tenet is a convolutional long short-term memory (ConvLSTM) and LSTM combinatorial deep learning model to learn the features of input historical weather image series coupled with spatial-temporal correlations. The model was trained and tested using Hong Kong datasets. The feasibility and effectiveness of the proposed model are verified and compared with parallel models under different criteria, including mean absolute error (MAE), root mean square error (RMSE) and R-squared (R2). The experimental results show that: (1) the proposed ConvLSTM-LSTM deep learning model can effectively forecast wind speed regardless of location; (2) the overall MAE, RMSE, and R2 value of the proposed model are improved by 14.84%, 15.04%, and 7.51%, respectively, compared to the ConvLSTM-full connected (ConvLSTM-FC) model, and by 22.12%, 22.80%, and 12.24%, respectively, compared to the convolutional neural network-LSTM (CNN-LSTM) model; and (3) compared with parallel models, the proposed model has better performance in predicting wind speed series with large amplitude variations and rapid frequency changes.

Data-Driven Parameter Prediction of Water Pumping Station

In the construction process of an intelligent pumping station, the parameter calibration of the pumping station unit is very important. In actual engineering, the working parameters of the pumping station are affected by complex working conditions and natural factors, so that it is difficult to establish a traditional physical model for the pumping station. This paper uses a data-driven method to apply the hybrid model of the convolutional neural network (CNN) and long-term short-term memory network (LSTM) to water level prediction in pumping stations and adds self-attention mechanism feature selection and a bagging optimization algorithm. Then, after an error analysis of the hybrid model, a performance comparison experiment with the separate model was conducted. The historical data of the pumping station project provided by the Tuancheng Lake Management Office of Beijing South-to-North Water Diversion Project was used to train and verify the proposed pumping station water level prediction model. The results show that the CNN–LSTM model based on the self-attention mechanism has higher accuracy than the separate CNN model and LSTM model, with a correlation coefficient (R2) of 0.72 and a mean absolute error (MAE) of 19.14. The model can effectively solve the problem of water level prediction in the front and rear pools under complex pumping station conditions.

Deep learning coupled model based on TCN-LSTM for particulate matter concentration prediction

In this study, we combined the Temporal Convolutional Network (TCN) model with the Long Short-Term Memory (LSTM) network model and applied it to prediction of atmospheric particulate matter (PM) concentrations. The study area is Xi'an City, Shaanxi Province, and the study period is from January 2015 to July 2022. During this period, Xi'an exceeded China's National Ambient Air Quality Grade Ⅰ standard for PM for up to 70% of the days. The prediction results of the TCN-LSTM model were compared with those of deep learning models (Convolutional Neural Network-LSTM, TCN, and LSTM) and machine learning models (Support Vector Regression and Random Forest). The R2 values of the TCN-LSTM model were all >0.88, indicating better performance than that of the other five models, and the errors of the TCN-LSTM model were all lower than those of the other five models. The results showed that high-accuracy PM predictions using deep learning models can improve air quality monitoring by compensating for problems in the environmental monitoring process such as pollutant monitoring errors caused by instrument failures. Additionally, sensitivity analysis helps to identify the key factors influencing the behavior of PM. A sensitivity analysis of PM for different periods of COVID-19 found that PM2.5 is more sensitive to O3, while PM10 is mainly influenced by PM2.5. The sensitivity analysis for the whole period showed that PM was closely related to CO. Removing variables that do not contribute to the model output based on the sensitivity analysis results improves modeling efficiency while reducing operating costs and improving environmental monitoring activities and management strategies.

Multi-UAV autonomous collision avoidance based on PPO-GIC algorithm with CNN–LSTM fusion network

This paper is concerned with the autonomous effective collision avoidance strategy for multiple unmanned aerial vehicles (multi-UAV) in limited airspace under the framework of proximal policy optimization (PPO) algorithm. An end-to-end deep reinforcement learning (DRL) control strategy and a potential-based reward function are designed. Next, the CNN-LSTM (CL) fusion network is constructed by fusing the convolutional neural network (CNN) and the long short-term memory network (LSTM), which realizes the feature interaction among the information of multi-UAV. Then, a generalized integral compensator (GIC) is introduced into the actor-critic structure, and the CLPPO-GIC algorithm is proposed by combining CL and GIC. Finally, we validate the learned policy in various simulation environments by performance evaluation. The simulation results show that the introduction of the LSTM network and GIC can further improve the efficiency of collision avoidance, and the robustness and accuracy of the algorithm are verified in different environments.

Dual-path recommendation algorithm based on CNN and attention-enhanced LSTM

ABSTRACT To recommend useful information to users more efficiently, this paper proposes a dual-path recommendation algorithm which combines multilayer Convolutional Neural Network (CNN) and attention-enhanced long short-term memory network (Attention-LSTM). Firstly, the matrix factorisation technique is used for learning the long-term preferences of users. Secondly, a dual-path network based on CNN and LSTM is constructed to perform feature extraction on the rating matrix. The dual-path network can learn the long-term preferences of users while capturing their dynamic preferences in changing preferences. The algorithm is tested on the public dataset MovieLens-1M, and the MAE value reflects the accuracy of the algorithm.

An Imperceptible Method to Monitor Human Activity by Using Sensor Data with CNN and Bi-directional LSTM

Deep learning (DL) algorithms have substantially increased research in recognizing day-to-day human activities All methods for recognizing human activities that are found through DL methods will only be useful if they work better in real-time applications. Activities of elderly people need to be monitored to detect any abnormalities in their health and to suggest healthy life style based on their day to day activities. Most of the existing approaches used videos, static photographs for recognizing the activities. Those methods make the individual to feel anxious that they are being monitored. To address this limitation we utilized the cognitive outcomes of DL algorithms and used sensor data as input to the proposed model which is collected from smart home dataset for recognizing elderly people activity, without any interference in their privacy. At early stages human activities the input for human activity recognition by DL models are done using single sensor data which are static and lack in recognizing dynamic and multi sensor data. We propose a DL architecture based on the blend of deep Convolutional Neural Network (CNN) and Bi-directional Long Short-Term Memory (Bi-LSTM) in this research which replaces human intervention by automatically extracting features from multifunctional sensing devices to reliably recognize the activities. During the entire investigation process we utilized Tulum, a benchmark dataset that contains the logs of sensor data. We exhibit that our methodology outperforms by marking its accuracy as 98.76% and F1 score as 0.98.

Efficacy of novel attention-based gated recurrent units transformer for depression detection using electroencephalogram signals.

Depression is a global challenge causing psychological and intellectual problems that require efficient diagnosis. Electroencephalogram (EEG) signals represent the functional state of the human brain and can help build an accurate and viable technique for the early prediction and treatment of depression. An attention-based gated recurrent units transformer (AttGRUT) time-series model is proposed to efficiently identify EEG perturbations in depressive patients. Statistical, spectral and wavelet features were first extracted from the 60-channel EEG signal data. Then, two feature selection techniques, recursive feature elimination and the Boruta algorithm, both with Shapley additive explanations, were utilised for selecting essential features. The proposed model outperformed the two baseline and two hybrid time-series models-long short-term memory (LSTM), gated recurrent units (GRU), convolutional neural network-LSTM (CNN-LSTM), and CNN-GRU-achieving an accuracy of up to 98.67%. Feature selection considerably increased the performance across all time-series models. Based on the obtained results, novel feature selection greatly affected the results of the baseline and hybrid time-series models. The proposed AttGRUT can be implemented and tested in other domains by using different modalities for prediction. The online version contains supplementary material available at 10.1007/s13755-022-00205-8.

Condition monitoring of wind turbine based on deep learning networks and kernel principal component analysis

Condition monitoring is frequently hampered by the abominable working conditions of wind turbines (WTs). The present study proposes a new approach for condition monitoring of WT, which involves the convolutional neural network (CNN) that cascades to the long- and short-term memory network (LSTM) with kernel principal component analysis (KPCA). First, the density-based spatial clustering of applications with noise (DBSCAN) method was used to filter data from supervisory control and data acquisition (SCADA) to improve the effectiveness of the data. Then, the KPCA algorithm was applied for performance monitoring and fault prediction of WT by selecting input variables, and the KPCA–CNN–LSTM model was established. Finally, taking a wind farm as an example, the established prediction model was used to analyze multiple components of the WT. The experimental outcomes demonstrate that the proposed model can help not only in monitoring the state of the WT but also in predicting the abnormal operation state of the WT at an early stage, thereby verifying the effectiveness of the proposed method.