LSTM Model Research Articles

Forecasting the Volatility of CSI 300 Index with a Hybrid Model of LSTM and Multiple GARCH Models

Forecasting the Volatility of CSI 300 Index with a Hybrid Model of LSTM and Multiple GARCH Models

Research on information leakage in time series prediction based on empirical mode decomposition.

Time series analysis predicts the future based on existing historical data and has a wide range of applications in finance, economics, meteorology, biology, engineering, and other fields. Although the combination of decomposition techniques and machine learning algorithms can effectively solve the problem of predicting nonstationary sequences, this kind of decomposition-integration-prediction strategy of the prediction method has serious defects. After the decomposition of the division of the training set and the test set, the information of the test set in the process of decomposition of the information leakage ultimately shows a high accuracy of the prediction of the illusionary. This paper proposes three improvement strategies for this type of "information leakage" problem: sliding window decomposition (SW-EMD), single training and multiple decomposition (STMP-EMD), and multiple training and multiple decomposition (MTMP-EMD). They are combined with a bidirectional multiscale temporal convolutional network (MSBTCN), bidirectional long- and short-term memory network (BiLSTM), and attention mechanism (DMAttention), which introduces a dependency matrix based on cosine similarity to be applied to water quality prediction. The experimental results show that the model achieves good performance in the prediction of three water quality indicators (pH, DO and KMnO4), and the accuracies of the three models proposed in this paper are improved by 1.958% and 0.853% in terms of the RMSE and MAPE, respectively, compared with those of the mainstream LSTM models. The key contributions of this study include the following: (1) three methods are proposed to improve the class EMD decomposition, which can effectively solve the problem of "information leakage" that exists in the current models via class EMD decomposition; (2) the CEEMDAN-MSBTCN-BiLSTM-DMAttention model structure is innovated by combining improved class EMD decomposition methods; and (3) the three improved decomposition methods proposed in this paper can effectively solve the problem of "information leakage" and optimize the prediction model at the same time. This study provides an effective experimental method for water quality prediction and can effectively address the problem of "overfitting" models via class EMD decompositions during model training and testing.

Machine Learning-Based Morel Mushroom Environmental Monitoring System

As living standards improve, the demand for edible fungi increases, making their efficient cultivation crucial for food supply and agricultural productivity. Traditional manual monitoring has limited accuracy, but the introduction of IoT and embedded technology enhances the intelligence and precision of agricultural monitoring, saving both labor and resources. This study designs a machine learning-based environmental monitoring and prediction system for edible fungi, utilizing low-power ZigBee networks and Kalman filtering to improve data accuracy. The system ensures real-time data transmission via the MQTT protocol and enhances security and scalability through cloud storage. By comparing Transformer, LSTM, and LSTM-Attention models, it was found that the Transformer model performed best in predicting environmental parameters, enabling proactive regulation, boosting yield and quality, and promoting the development of intelligent agriculture.

Comparative Evaluation of Sentiment Analysis Methods: From Traditional Techniques to Advanced Deep Learning Models

Abstract. Sentiment evaluation plays a crucial role in deciphering public perception and consumer responses in today's digital landscape. This investigation offers a thorough assessment of diverse sentiment evaluation techniques, contrasting conventional machine learning methodologies with cutting-edge deep learning frameworks. In particular, the research scrutinizes the efficacy of Bidirectional Encoder Representations from Transformers (BERT)-derived architectures (BERT-Base and Robustly Optimized BERT Pretraining Approach (RoBERTa)), Convolutional Neural Networks (CNN), Long Short-Term Memory Networks (LSTM), Support Vector Machines (SVM), and Naive Bayes classifiers. The study gauges these approaches based on their precision, recall, F1-metric, overall accuracy, and computational efficiency using an extensive sentiment evaluation dataset. The results reveal that BERT-based models, particularly RoBERTa, achieve the highest accuracy (87.44%) and F1-score (0.8746), though they also require the longest training time (approximately 3 hours). CNN and LSTM models strike a balance between performance and efficiency, while traditional methods like SVM and Naive Bayes offer faster training and deployment with moderate accuracy. The insights gained from this study are valuable for both researchers and practitioners, highlighting the trade-offs between model performance, computational demands, and practical deployment considerations in sentiment analysis applications.

Research on Medium- and Long-Term Hydropower Generation Forecasting Method Based on LSTM and Transformer

Hydropower generation is influenced by various factors such as precipitation, temperature, and installed capacity, with hydrometeorological factors exhibiting significant temporal variability. This study proposes a hydropower generation forecasting method based on Transformer and SE-Attention for different provinces. In the model, the outputs of the Transformer and SE-Attention modules are fed into an LSTM layer to capture long-term data dependencies. The SE-Attention module is reintroduced to enhance the model’s focus on important temporal features, and a linear layer maps the hidden state of the last time step to the final output. The proposed Transformer-LSTM-SE model was tested using provincial hydropower generation data from Yunnan, Sichuan, and Chongqing. The experimental results demonstrate that this model achieves high accuracy and stability in medium- and long-term hydropower forecasting at the provincial level, with an average accuracy improvement of 33.79% over the LSTM model and 24.30% over the Transformer-LSTM model.

Global Warming: Predictive Models and Correlation Analysis

In recent years, the Earth's rising temperatures and the shrinking Antarctic ice cover have become increasingly evident. Understanding the factors driving global warming and developing strategies to mitigate it are critical. By applying the Pettitt change-point detection method, we analyzed March temperatures from the past decade and identified a significant shift occurring in March 2015. Forecasting models, including ARIMA and LSTM, predict similar temperature trends, with the LSTM model demonstrating slightly higher accuracy. Through grey correlation analysis and multiple linear regression, we identified CO2 as the most influential factor affecting global temperatures. Therefore, reducing CO2 emissions is crucial to mitigating global warming.

An Enhanced Detection Method of Hardware Trojan Based on CNN- Attention-LSTM

The security issues caused by the insertion of hardware Trojans seriously threaten the security and reliability of the entire hardware device. This article constructs a detection model that combines convolutional neural networks (CNN) and long short-term memory networks (LSTM), and introduces attention mechanism to enhance the model's ability to recognize complex circuits. This method can automatically learn and optimize feature extraction and classification models, reduce reliance on manual experience through training on large amounts of data, and improve the intelligence level of detection. Especially, by combining attention mechanisms and LSTM models, it is possible to more effectively capture small anomalies in circuit design and improve the accuracy and efficiency of hardware Trojan detection. The experimental results show that the proposed CNN-Attention-LSTM model exhibits superior Trojan detection performance and good generalization ability on different datasets, with an precision of 96.3%, a recall rate of 94.7%, and an F1 score of 95.5%.

Spatial heterogeneity response of soil salinization inversion cotton field expansion based on deep learning

Soil salinization represents a significant challenge to the ecological environment in arid areas, and digital mapping of soil salinization as well as exploration of its spatial heterogeneity with crop growth have important implications for national food security and salinization management. However, the machine learning models currently used are deficient in mining local information on salinity and do not explore the spatial heterogeneity of salinity impacts on crops. This study developed soil salinization inversion models using CNN (Convolutional Neural Network), LSTM (Long Short-Term Memory Network), and RF (Random Forest) models based on 97 field samples and feature variables extracted from Landsat-8 imagery. By evaluating the accuracy, the best-performing model was selected to map soil salinity at a 30m resolution for the years 2013 and 2022, and to explore the relationship between soil electrical conductivity (EC) values and the expansion of cotton fields as well as their spatial correlation. The results indicate that:(1) The CNN performs best in prediction, with an R2 of 0.84 for the training set and 0.73 for the test set, capable of capturing more local salinity information. (2) The expansion of cotton fields has reduced the level of soil salinization, with the area of severely salinized and saline soils in newly added cotton fields decreasing from 177.91 km2 and 381.46 km2 to 19.49 km2 and 1.12 km2, respectively. (3) Regions with long-term cotton cultivation and newly reclaimed cotton fields exhibit high sensitivity and vulnerability to soil salinity. This study explores the excellent performance of deep learning in salinity mapping and visualizes the spatial distribution of cotton fields that are highly sensitive to soil salinity, providing a scientific theoretical basis for accurate salinity management.

A New Method of Intelligent Fault Diagnosis of Ship Dual-Fuel Engine Based on Instantaneous Rotational Speed

Ship engine misfire faults not only pose a serious threat to the safe operation of ships but may also cause major safety accidents or even lead to ship paralysis, which brings huge economic losses. Most traditional fault diagnosis methods rely on manual experience, with limited feature extraction capability, low diagnostic accuracy, and poor adaptability, which make it difficult to meet the demand for high-precision diagnosis. To this end, a fusion intelligent diagnostic model—ResNet–BiLSTM—is proposed based on a residual neural network (ResNet) and a bidirectional long short-term memory network (BiLSTM). Firstly, a multi-scale decomposition of the instantaneous rotational speed signal of a ship’s engine is carried out by using the continuous wavelet transform (CWT), and features containing misfire fault information are extracted. Subsequently, the extracted features are fed into the ResNet–BiLSTM model for learning. Finally, the intelligent diagnosis of ship dual-fuel engine misfire faults is realized by the classifier. The model combines the advantages of ResNet18 in image feature extraction and the capability of BiLSTM in temporal information processing, which can efficiently capture the time-frequency features and dynamic changes in the fault signal. Through comparison experiments with fusion models AlexNet–BiLSTM, VGG–BiLSTM, and the existing AlexNet–LSTM and VGG–LSTM models, the results show that the ResNet–BiLSTM model outperforms the other models in terms of diagnostic accuracy, robustness, and generalization ability. This model provides an effective new method for intelligent diagnosis of ship dual-fuel engine misfire faults to solve the traditional diagnostic methods’ limitations.

Collaborative Research for the Development of Localized Solar PV Output Forecasting Models for the Philippines Using Geospatial Data

Abstract. This paper presents a collaborative effort to develop localized solar photovoltaic (PV) power output (PPV) forecasting models for the Philippines using geospatial data. It underlines the importance of solar energy in the country and discusses the opportunities and challenges associated with PPV forecasting. Project SINAG, a two-year research project, aimed to develop solar PV output forecasting models through a collaborative approach with academic institutions, solar energy industries, and government agencies. Actual PPV data from 43 solar PV installations were analyzed alongside meteorological data from the PAGASA weather bureau, ERA5, AHI-8, and FY- 4A. These datasets were filtered based on a one-year period to ensure quality. The study employed SARIMAX, LSTM, and XGBoost models individually and in hybrid models to develop the forecasting models. Model performance was evaluated using root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). In a case study in Baguio City, the SARIMAX model exhibited strong seasonal dependence, providing more accurate forecasts in dry seasons than in wet seasons. Additionally, the forecasting accuracy of each model (SARIMAX, LSTM, and XGBoost) varied based on the month and location of the installation, emphasizing the need for local and season-based PPV forecasting models. Despite implementation challenges, such as collaboration arrangements, bureaucratic barriers, and budget constraints, the project produced thirteen research publications and provided data for three student theses. This paper also demonstrated diverse engagements and contributions that emphasize the significance of collaborative research in conducting nationwide-scale data-driven projects.

Research on stock prediction based on CED-PSO-StockNet time series model

To tackle the challenge of low accuracy in stock prediction within high-noise environments, this paper innovatively introduces the CED-PSO-StockNet time series model. Initially, the model decomposes raw stock data using the Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) technique and reconstructs the components by estimating their frequencies via the extreme point method. This process enhances component stability and mitigates noise interference. Subsequently, an Encoder-Decoder framework equipped with an attention mechanism is employed for precise prediction of the reconstructed components, facilitating more effective extraction and utilization of data features. Furthermore, this paper utilizes an Improved Particle Swarm Optimization (IPSO) algorithm to optimize the model parameters. On the Pudong Bank dataset, through ablation experiments and comparisons with baseline models, various optimization strategies incorporated into the proposed CED-PSO-StockNet model were effectively validated. Compared to the standalone LSTM model, CED-PSO-StockNet achieved a remarkable 45.59% improvement in the R2 metric. To further assess the model’s generalization capability, this paper also conducted comparative experiments on the Ping An Bank dataset, and the results underscored the significant advantages of CED-PSO-StockNet in the domain of stock prediction.

Hydropower Station Status Prediction Using RNN and LSTM Algorithms for Fault Detection

In 2019, more than 16% of the globe’s total production of electricity was provided by hydroelectric power plants. The core of a typical hydroelectric power plant is the turbine. Turbines are subjected to high levels of pressure, vibration, high temperatures, and air gaps as water passes through them. Turbine blades weighing several tons break due to this surge, a tragic accident because of the massive damage they cause. This research aims to develop predictive models to accurately predict the status of hydroelectric power plants based on real stored data for all factors affecting the status of these plants. The importance of having a typical predictive model for the future status of these plants lies in avoiding turbine blade breakage and catastrophic accidents in power plants and the resulting damages, increasing the life of these plants, avoiding sudden shutdowns, and ensuring stability in the generation of electrical energy. In this study, artificial neural network algorithms (RNN and LSTM) are used to predict the condition of the hydropower station, identify the fault before it occurs, and avoid it. After testing, the LSTM algorithm achieved the greatest results with regard to the highest accuracy and least error. According to the findings, the LSTM model attained an accuracy of 99.55%, a mean square error (MSE) of 0.0072, and a mean absolute error (MAE) of 0.0053.

Research on traffic flow prediction method based on LSTM model and PSO-LSTM model

Abstract. With the acceleration of urbanization, the number of cars owned by residents has also significantly increased. The contradiction between the number of cars and the road carrying capacity has become increasingly severe, resulting in very serious congestion. This paper selects road data from 0:00 to 10:00 every morning in Beijing from April 2nd to April 12th, 2016, and uses the average speed of vehicles as a variable to measure road congestion. Based on these data, this article uses LSTM models to predict the speed of vehicles on two roads representing main and non main roads. Research has found that LSTM model has good predictive performance for the speed of vehicles on two roads and is very accurate in predicting the data trend. However, the accuracy of LSTM model in predicting non periodic and highly discrete data is not ideal. This article also uses the PSO-LSTM model to predict one of the roads, and the results show that the model is more accurate than the LSTM model in predicting non periodic data. The prediction of short-term traffic flow will greatly help the transportation department coordinate and manage resources, alleviate traffic congestion, and facilitate residents' travel.

Noise‐robust structural response estimation method using short‐time Fourier transform and long short‐term memory

AbstractStructural response estimation based on deep learning can suffer from reduced estimation performance owing to discrepancies between the training and test data as the noise level in the test data increases. This study proposes a short‐time Fourier transform‐based long short‐term memory (STFT‐LSTM) model to improve estimation performance in the presence of noise and ensure estimation robustness. This model enables robust estimations in the presence of noise by positioning an STFT layer before feeding the data into the LSTM layer. The output transformed into the time‐frequency domain by the STFT layer is learned by the LSTM model. The robustness of the proposed model was validated using a numerical model with three degrees of freedom at various signal‐to‐noise ratio levels, and its robustness against impulse and periodic noise was verified. Experimental validation assessed the estimation robustness under impact load and verified the robustness against environmental noise in the acquired acceleration response.

Research on oil and gas pipeline leak detection method based on 1DCNN-DBO-LSTM

Abstract Detecting leaks in oil and gas pipelines is crucial for the safe operation of these systems. Current techniques for identifying oil and gas pipeline breaches are insufficient and fail to utilize the temporal attributes of the leak signal. This research proposes a fusion recognition model that integrates a long short-term memory network (LSTM) optimized using a one-dimensional convolutional neural network (1DCNN) with the dung beetle optimization algorithm (DBO). The model initially tackles the issue of excessive depth in the 1DCNN network, which paradoxically results in decreased accuracy and increased computational load. It develops a lightweight 1DCNN network via structural experiments that adaptively extracts data features. Subsequently, to mitigate the effects of the LSTM model's high stochasticity and the challenges associated with manual hyperparameter selection on detection accuracy, an enhanced LSTM model is proposed. This model is optimized through the integration of the Dung Beetle Optimization Algorithm (DBO) to refine the hyperparameters in LSTM. A 1DCNN-DBO-LSTM recognition model is developed, whereby the data features retrieved adaptively from the 1DCNN are fed into the enhanced DBO-LSTM for classification, and the T-SNE dimensionality reduction technique is utilized to visualize the network at each output layer. The model is assessed utilizing the precision and duration measures. The suggested model enhances recognition accuracy and reduces detection time compared to existing advanced models. The approach presented in this work can extract pipeline data features more swiftly and accurately, thus enhancing classification precision.

Energy Market Price Forecasting and Financial Technology Risk Management Based on Generative AI

Abstract: The volatility of global energy markets, particularly electricity prices, plays a crucial role in influencing international economic activities. With the ongoing global energy transition and the push for low-carbon development, predicting electricity prices has become increasingly important for policymakers and market participants. This paper explores the forecasting capabilities of the ARIMA and LSTM models in analyzing electricity prices in the United States, drawing from data spanning 2001 to 2024.ARIMA, a traditional time series model, is valued for its simplicity and effectiveness in capturing linear trends, while LSTM, a deep learning-based model, excels at handling long-term dependencies in complex datasets. This study reveals that while both models offer valuable insights, each exhibits limitations. ARIMA struggles with non-linear patterns and volatility, whereas LSTM tends to underestimate extreme price values. The findings highlight the potential of hybrid models that combine traditional and machine learning approaches to enhance forecasting accuracy in the increasingly dynamic energy market. This research provides essential guidance for improving decision-making processes in the context of the global shift towards clean energy.

Polish Speech and Text Emotion Recognition in a Multimodal Emotion Analysis System

Emotion recognition by social robots is a serious challenge because sometimes people also do not cope with it. It is important to use information about emotions from all possible sources: facial expression, speech, or reactions occurring in the body. Therefore, a multimodal emotion recognition system was introduced, which includes the indicated sources of information and deep learning algorithms for emotion recognition. An important part of this system includes the speech analysis module, which was decided to be divided into two tracks: speech and text. An additional condition is the target language of communication, Polish, for which the number of datasets and methods is very limited. The work shows that emotion recognition using a single source—text or speech—can lead to low accuracy of the recognized emotion. It was therefore decided to compare English and Polish datasets and the latest deep learning methods in speech emotion recognition using Mel spectrograms. The most accurate LSTM models were evaluated on the English set and the Polish nEMO set, demonstrating high efficiency of emotion recognition in the case of Polish data. The conducted research is a key element in the development of a decision-making algorithm for several emotion recognition modules in a multimodal system.

Study on the impact of shoulder flexibility training on smash speed mechanics in badminton using machine learning

The smash stroke in badminton is a key attack style that makes the opponent player miss the strike; the smash requires speed, agility, strength, and precision. The smash demands a high level of shoulder flexibility from the players, which increases the Range of Motion (ROM) during the backswing and forward swing phases. The shoulder flexibility provides excellent energy storage and transfer, improving smash speed. The biomechanical efficiency of Shoulder Flexibility Training (FLT) on smash speed efficiency is still under study. This lack of study leads to modelling training program limitations, which may increase the risk of injury. Examining the process by which smash speed mechanics are impacted by Shoulder Flexibility Training (SFT) programs is the primary goal of the present investigation. It is approximately a 6-week training program for Amateur Players (AP) and National Players (NP), which uses core shoulder motions like flexion, abduction, and rotation as its basis. Motion capture systems and radar sensors investigated joint motion and smash speeds. To address the shortage of study evidence on the subject, a hybrid CNN + LSTM model was applied to predict smash speed concerning improved shoulder flexibility. As reported by the research results, students’ smash speed and shoulder flexibility improved significantly during training. There was a 4.35% boost to smash speed at contact and a 4.69% gain in shoulder internal rotation compared to non-contact athletes. Additionally, there was a 9.83% boost in smash speed at contact and a 9.76% boost in shoulder internal rotation for the best athletes. Considering post-training illnesses, the CNN + LSTM model successfully predicted smash speed, with R³ scores of 0.99 for NP and 0.97 for AP.

Lower Limb Motion Recognition Based on sEMG and CNN-TL Fusion Model.

To enhance the classification accuracy of lower limb movements, a fusion recognition model integrating a surface electromyography (sEMG)-based convolutional neural network, transformer encoder, and long short-term memory network (CNN-Transformer-LSTM, CNN-TL) was proposed in this study. By combining these advanced techniques, significant improvements in movement classification were achieved. Firstly, sEMG data were collected from 20 subjects as they performed four distinct gait movements: walking upstairs, walking downstairs, walking on a level surface, and squatting. Subsequently, the gathered sEMG data underwent preprocessing, with features extracted from both the time domain and frequency domain. These features were then used as inputs for the machine learning recognition model. Finally, based on the preprocessed sEMG data, the CNN-TL lower limb action recognition model was constructed. The performance of CNN-TL was then compared with that of the CNN, LSTM, and SVM models. The results demonstrated that the accuracy of the CNN-TL model in lower limb action recognition was 3.76%, 5.92%, and 14.92% higher than that of the CNN-LSTM, CNN, and SVM models, respectively, thereby proving its superior classification performance. An effective scheme for improving lower limb motor function in rehabilitation and assistance devices was thus provided.

Comparing machine learning and deep learning models to predict cognition progression in Parkinson's disease.

Cognitive decline in Parkinson's disease (PD) varies widely. While models to predict cognitive progression exist, comparing traditional probabilistic models to deep learning methods remains understudied. This study compares sequential modeling techniques to identify cognitive progression in individuals with and without PD. Using data from the Parkinson's Progression Marker Initiative, shallow Markov, deep recurrent (long short-term memory [LSTM]), and nonrecurrent (temporal fusion transformer [TFT]) models were compared to predict cognitive status over time. Cognitive status was categorized into normal cognition (NC), mild cognitive impairment (MCI), and dementia. Predictions were made annually for up to 3 years using clinical data, including demographics, cognitive assessments, PD severity, and medical history. Each approach was evaluated using inverse probability weighted (IPW-) F1 scores. An ensemble method combined outputs from the Markov, LSTM, and TFT models. The dataset included 917 individuals (53% PD; 30% at risk for PD; 17% Healthy Controls). The TFT model outperformed others across all annual periods (IPW-F1 = 0.468) compared to the Markov (IPW-F1 = 0.349) and LSTM (IPW-F1 = 0.414) models, with improved performance using an ensemble approach (IPW-F1 = 0.502). For MCI and dementia predictions, which were rarer occurrences compared to NC status (ratios: 50:8:1), the TFT model consistently outperformed competing models, achieving IPW-F1 scores of 0.496 and 0.533 for MCI and dementia, respectively. In conclusion, sequential deep learning models like TFT, which mitigate long-term memory loss and can interpret complex, high-dimensional data, perform best overall in predicting clinically important cognitive transitions. These methods should be further explored for predicting degenerative conditions.