Short-term Memory Network Research Articles

A Systematic Literature Review on The Role of LSTM Networks in Capturing Temporal Dependencies in Data Mining Algorithms

Data mining plays a crucial role in extracting meaningful insights from large and complex data sets, with broad applications in sectors like finance, health care, and market analysis. Traditional techniques—such as classification, clustering, association rule mining, regression, and anomaly detection—are effective for analyzing structured data but struggle with sequential data due to the challenges of modeling temporal dependencies. Long Short Term Memory (LSTM) networks, a specialized form of Recurrent Neural Networks (RNNs), provide a solution to these challenges. By incorporating memory cells and gating mechanisms, LSTM effectively manage long-term dependencies and address issues like vanishing and exploding gradients. This paper reviews the impact of LSTM networks on data mining, analyzing over 60 key publications. By synthesizing concepts and recent advancements, the review underscores how LSTMs enhance the ability of data mining algorithms to capture and predict temporal patterns, reflecting current research trends and innovations

Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image and Video Forgeries

This paper introduces a hybrid architecture combining Long Short-Term Memory (LSTM) networks and an encoderdecoder model for the detection and localization of image and video forgeries. The proposed system leverages resampling features and LSTM cells to identify manipulation patterns such as splicing and retouching in multimedia content. By utilizing a combination of spatial and temporal features, the model achieves high precision in detecting forged regions. Extensive testing on diverse datasets demonstrates the robustness of the proposed method.

A Novel Deep Learning Model for Stock Prediction Integrating Historical Prices, Core Volatility Trends and Financial News

Accurate stock price forecasting is crucial for investors and analysts aiming to make informed decisions and mitigate risks in financial markets. While deep learning models have proven effective in identifying complex patterns in quantitative data, they often struggle to account for non-quantitative factors such as corporate events, market sentiment, and regulatory changes that can cause unpredictable stock price movements. To address this challenge, an ensemble model is introduced that integrates historical price data, financial news, and stock fundamentals to predict next-day stock prices. The model combines one-dimensional Convolutional Neural Networks (1D CNNs), Long Short-Term Memory networks (LSTMs), and Gated Recurrent Units (GRUs), applied independently to each data stream and concurrently as a unified ensemble, to capture both short-term and long-term market behaviors. NVIDIA (NVDA) stock was selected for testing due to its high volatility and rapid price fluctuations, presenting a challenging case for predictive modeling. Using data spanning from April 2014 to March 2024, the model achieved a coefficient of determination (R²) of 0.983 and a mean absolute error (MAE) of $12.72. These results suggest that the ensemble approach effectively captures both quantitative and qualitative factors, making it a promising tool for improving stock price prediction in volatile market conditions.

Design and development of an ai-driven communication model for stimulating memory recall in patients with Alzheimer’s and other neurodegenerative diseases

This study presents the development and evaluation of an AI-driven communication tool designed to stimulate memory recall in patients with Alzheimer’s disease and other neurodegenerative disorders. The tool integrates machine learning algorithms, including Random Forest and Long Short-Term Memory (LSTM) networks, with mass communication theories to deliver personalized and adaptive communication strategies. The AI models were trained and tested on patient interaction data, enabling the tool to dynamically adjust its prompts based on real-time cognitive states. Performance metrics such as accuracy, precision, and AUC-ROC were used to assess model efficacy, with the Random Forest model achieving the highest performance across all metrics. Clinical trials demonstrated significant improvements in memory recall and emotional engagement in patients using the AI tool compared to standard cognitive stimulation therapy. Caregivers also reported higher satisfaction with the tool's usability and impact on patient interaction quality. These findings highlight the potential of AI-driven, personalized communication tools to revolutionize Alzheimer's care, offering scalable, adaptive interventions that improve cognitive and emotional health. Academically, this work advances the integration of AI and health communication, emphasizing culturally sensitive and patient-centered approaches. In healthcare, it presents a promising non-pharmacological intervention that can be easily scaled for widespread use, potentially reducing caregiver burden and improving patient outcomes.

Fortifying cloud environments against data breaches: A novel AI-driven security framework

As cloud computing continues to dominate the modern technological landscape, organizations face growing challenges in preventing data breaches and sophisticated cyber threats. The increasing complexity and scale of cloud environments require advanced security mechanisms to address evolving threats. This paper introduces "SecureCloudAI," a cutting-edge AI-driven security framework designed to fortify sensitive data within cloud infrastructures. SecureCloudAI leverages a hybrid approach that combines machine learning models like Random Forest and deep learning techniques, including Long Short-Term Memory (LSTM) networks, to detect, classify, and respond to potential security breaches in real-time. The system offers robust malware detection, network traffic analysis, and web intrusion detection while maintaining scalability and efficiency across large cloud environments. Experimental results demonstrate SecureCloudAI's high accuracy, with malware detection at 94.78% and network traffic classification at 90.92%, ensuring the system can handle both complex and emerging threats. This AI-driven solution marks a significant advancement in cloud security, providing organizations with an adaptive and scalable tool to safeguard their data against the ever-growing cyber threat landscape.

Optimization of Hybrid Energy Systems Based on MPC-LSTM-KAN: A Case Study of a High-Altitude Wind Energy Work Umbrella Control System

This paper presents an optimization method for hybrid energy systems based on Model Predictive Control (MPC), Long Short-Term Memory (LSTM) networks, and Kolmogorov–Arnold Networks (KANs). The proposed method is applied to a high-altitude wind energy work umbrella control system, where it aims to enhance the stability and efficiency of energy utilization. The work umbrella system integrates wind and solar energy sources, with energy stored in a battery and used to control the umbrella’s operations. The MPC framework is employed to optimize control actions by solving a finite-horizon optimization problem, ensuring the battery State of Charge (SOC) remains within an optimal range. The LSTM network provides accurate predictions of environmental conditions, including wind speed and solar irradiance, which are essential for MPC’s decision-making process. To address complex nonlinearities in the system, the KAN is utilized to model and approximate these dynamics, refining the LSTM predictions. The integration of these advanced control strategies enables the system to handle varying operational conditions and maintain optimal performance. The case study demonstrates the effectiveness of the MPC-LSTM-KAN approach, revealing improvements in the SOC stability, energy efficiency, and operational endurance of the high-altitude wind energy work umbrella system. The results indicate that this hybrid optimization method offers a robust solution for managing hybrid energy systems in dynamic environments.

Modeling takeover time for non-driving related tasks using hybrid deep neural networks in conditionally automated vehicles

To enhance the safety of the transition from autonomous to manual driving control, one of the key factors is quantifying the safe takeover transition, namely the takeover time. Previous studies have focused on factors influencing takeover time and predicting takeover time using traditional methods, but there is a lack of research utilizing deep learning networks and driver physiological factors for takeover time prediction. Toward this end, we propose a hybrid deep learning network model dedicated to predicting drivers’ takeover time with a dataset obtained from a bench experiment involving 46 drivers. Firstly, to address the potential information loss commonly associated with traditional manual feature extraction techniques, a convolutional neural network (CNN) is developed to automatically extract physiological spatial representations from drivers. Additionally, a double-layer bidirectional long short-term memory network (DBiLSTM) is incorporated to model the physiological time representation dimension, enabling a more refined representation of physiological signals. Then, to tackle the complexities introduced by individual driver variations, a self-attention mechanism is enabling the model to automatically prioritize critical features, thereby enhancing the accuracy and reliability of its predictions. The results demonstrate that the proposed approach surpasses traditional machine learning network models in predictive accuracy for takeover time prediction, thereby exhibiting novelty in the field of vehicle takeover control research. This provides a more reliable technical foundation for anticipating the durations of vehicle takeovers.

Influence of Ocean Current Features on the Performance of Machine Learning and Dynamic Tracking Methods in Predicting Marine Drifter Trajectories

Accurately and rapidly predicting marine drifter trajectories under conditions of information scarcity is critical for addressing maritime emergencies and conducting marine surveys with resource-limited unmanned vessels. Machine learning-based tracking methods, such as Long Short-Term Memory networks (LSTM), offer a promising approach for trajectory prediction in such scenarios. This study combines satellite observations and idealized simulations to compare the predictive performance of LSTM with a resource-dependent dynamic tracking method (DT). The results indicate that when driven solely by historical drifter paths, LSTM achieves better trajectory predictions when trained and tested on relative trajectory intervals rather than the absolute positions of individual trajectory points. In general, LSTM provides a more accurate geometric pattern of trajectories at the initial stages of forecasting, while DT offers superior accuracy in predicting specific trajectory positions. The velocity and curvature of ocean currents jointly influence the prediction quality of both methods. In regions characterized by active sub-mesoscale dynamics, such as the fast-flowing and meandering Kuroshio Current and Kuroshio Current Extension, DT predicts more reliable trajectory patterns but lacks precision in detailed position estimates compared to LSTM. However, in areas dominated by the fast but relatively straight North Equatorial Current, the performance of the two methods reverses. The two methods also demonstrate different tolerances for noise and sampling intervals. This study establishes a baseline for selecting machine learning methods for marine drifter prediction and highlights the limitations of AI-based predictions under data-scarce and resource-constrained conditions.

The CNN-LSTM-attention Model for short term prediction of the Polar Motion

Abstract The accuracy of polar motion prediction significantly impacts the fields of coordinate frame transformation, satellite orbit determination, and deep space exploration. The present study develops two short term forecasting models based on the EOP 14C04 series. One hybrid approach incorporates convolutional neural networks (CNN) and long short-term memory networks (LSTM), augmented with an attention mechanism; whereas another baseline model comprises CNN and LSTM. The first model, in contrast to the second model, incorporates an attention mechanism module for a more comprehensive integration of temporal information at each time step. In the initial short-term forecasting experiment, we conducted 360 repeated predictions, and the findings revealed that the parameters suitable for PMX forecasting may not necessarily be applicable to PMY forecasting. In the second experiment, the two models generated a total of 500 forecasts, each encompassing short-term predictions ranging from 1 to 30 days. The experimental results demonstrate that the first model exhibits mean absolute error (MAE) range of 0 ~ 7.72 mas for PMX and 0 ~ 4.73 mas for PMY, while the second model shows MAE range of 0 ~ 7.88 mas for PMX and 0 ~ 4.78 mas for PMY. After two exploratory experiments, we discovered the following results: the first model exhibits marginally superior predictive accuracy compared to the second model. Furthermore, this study substantiates the robustness of both models in short-term prediction and affirms the significance of assigning distinct weights to past temporal intervals in forecasting, thereby offering a novel perspective for polar motion prediction research.

Prediction of the Remaining Useful Life of Bearings Through CNN-Bi-LSTM-Based Domain Adaptation Model

Predicting the remaining useful life (RUL) of mechanical bearings is crucial in the industry. Estimating the RUL enables the assessment of health bearing, maintenance planning, and significant cost reduction, thereby fostering industrial development. Existing models rely on traditional feature engineering with feature changes because operating conditions pose a major challenge to the generalization of RUL prediction models. This study focuses on neural network-based feature engineering and the downstream prediction of the RUL, eliminating the need for specific prior knowledge and simplifying the development and maintenance of models. Initially, a convolutional neural network (CNN) model is employed for feature engineering. Subsequently, a bidirectional long short-term memory network (Bi-LSTM) model is used to capture the time-series degradation characteristics of the engineered features and predict the RUL through regression. Finally, the study examines the influence of operating conditions in the model and integrates domain adaptation to minimize differences in feature distribution, thereby enhancing the model’s generalizability for the RUL prediction.

Deep Learning in Finance: A Survey of Applications and Techniques

Machine learning (ML) has transformed the financial industry by enabling advanced applications such as credit scoring, fraud detection, and market forecasting. At the core of this transformation is deep learning (DL), a subset of ML that is robust in processing and analyzing complex and large datasets. This paper provides a comprehensive overview of key deep learning models, including Convolutional Neural Networks (CNNs), Long Short-Term Memory networks (LSTMs), Deep Belief Networks (DBNs), Transformers, Generative Adversarial Networks (GANs), and Deep Reinforcement Learning (Deep RL). Beyond summarizing their mathematical foundations and learning processes, this study offers new insights into how these models are applied in real-world financial contexts, highlighting their specific advantages and limitations in tasks such as algorithmic trading, risk management, and portfolio optimization. It also examines recent advances and emerging trends in the financial industry alongside critical challenges such as data quality, model interpretability, and computational complexity. These insights can guide future research directions toward developing more efficient, robust, and explainable financial models that address the evolving needs of the financial sector.

Flow Field Analysis and Development of a Prediction Model Based on Deep Learning

The velocity of ocean currents significantly affects the trajectory prediction of ocean drifters and the safe navigation of intelligent vessels. Currently, most ocean current predictions focus on time-based forecasts at specific fixed points. In this study, deep learning based on the flow field prediction model (CNNs–MHA–BiLSTMs) is proposed, which predicts the changes in ocean currents by learning from historical flow fields. Unlike conventional models that focus on single-point current velocity data, the CNNs–MHA–BiLSTMs model focuses on the ocean surface current information within a specific area. The CNNs–MHA–BiLSTMs model integrates multiple convolutional neural networks (CNNs) in parallel, multi-head attention (MHA), and bidirectional long short-term memory networks (BiLSTMs). The model demonstrated exceptional modelling capabilities in handling spatiotemporal features. The proposed model was validated by comparing its predictions with those predicted by the MIKE21 flow model of the ocean area within proximity to Dalian Port (which used a commercial numerical model), as well as those predicted by other deep learning algorithms. The results showed that the model offers significant advantages and efficiency in simulating and predicting ocean surface currents. Moreover, the accuracy of regional flow field prediction improved with an increase in the number of sampling points used for training. The proposed CNNs–MHA–BiLSTMs model can provide theoretical support for maritime search and rescue, the control or path planning of Unmanned Surface Vehicles (USVs), as well as protecting offshore structures in the future.

Enhancing streamflow prediction in the Wujiang River basin: a two-stage decomposition approach with deep learning integration

ABSTRACT Global climate change and human activities have profoundly impacted the geological and hydrological processes in watersheds, increasing the challenges in streamflow prediction. In this study, we propose a streamflow prediction model based on deep learning and dual-mode decomposition specifically tailored for the Wujiang River basin, a significant tributary on the southern bank of the upper Yangtze River. This model effectively addresses the prediction error issue caused by high-frequency components through dual-mode decomposition of time-series data. The results demonstrate that employing the coupled complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN)–variational mode decomposition (VMD)–temporal convolutional network (TCN)–long short-term memory network (LSTM) model reduces prediction errors by at least 35% compared to single decomposition models. Furthermore, compared to individual TCN or LSTM models, the TCN–LSTM coupled model exhibits greater stability and higher prediction accuracy during training, with reductions in mean absolute error and root mean square error by 43.13 and 24.57%, respectively. This model holds promising prospects for application and can provide crucial insights for water resource management.

Predicting diabetes prognosis using machine learning techniques in a Hungarian clinical database

Abstract Background Chronic conditions such as type 2 diabetes mellitus have great impact on patients’ quality of life. Although clinical databases provide a perfect base for research aimed at improving diabetes care, analyzing such databases requires extensive pre-processing, mainly due to the large amount of unstructured data. Our study aimed to present the steps of generating a dataset from a large clinical database, and to apply machine learning-based analytical techniques regarding. Methods Data of the Clinical Center of University of Debrecen was used. To structure the unstructured data, regular expressions and natural language processing methods were used. The main machine learning models were as follows: Gradient Boosting Machines to predict the risk of complication development; Long Short-Term Memory Networks to forecast future health outcomes. All analysis and procedures were done using Python. Results The database contains approximately 1600 tables, with a total size of 1.9 terabytes, where the largest table is 21.07 gigabytes with approximately 44.64 million rows. The final dataset consisted of 40,332 patients. Most variables originate from the unstructured data, including complications and comorbidities of diabetes, as well as physical and laboratory parameters. Related to laboratory parameters, the number of measurements and the median value for every half-year were available. The diagnosis time of the complications’ occurrence is also presented. Machine learning methods were more accurate compared with traditional statistical methods in predicting the prognosis (p < 0.05). Discussion Our research highlights the importance of clinical data in chronic disease management. There are challenges in pre-processing and managing datasets, but machine learning-based methods are very efficient not only in extracting useful information from unstructured data but also in predicting the prognosis and identifying potential intervention points for better care. Key messages • Natural language processing can be used to obtain useful information from unstructured clinical data. • Using machine learning techniques on clinical data could improve diabates care.

Time Series Forecasting of Thermal Systems Dispatch in Legal Amazon Using Machine Learning

This paper analyzes time series forecasting methods applied to thermal systems in Brazil, specifically focusing on diesel consumption as a key determinant. Recognizing the critical role of thermal systems in ensuring energy stability, especially during low rain seasons, this study employs bagged, boosted, and stacked ensemble learning methods for time series forecasting focusing on exploring consumption patterns and trends. By leveraging historical data, the research aims to predict future diesel consumption within Brazil’s thermal energy sector. Based on the bagged ensemble learning approach a mean absolute percentage error of 0.089% and a coefficient of determination of 0.9752 were achieved (average considering 50 experiments), showing it to be a promising model for the short-time forecasting of thermal dispatch for the electric power generation system. The bagged model results were better than for boosted and stacked ensemble learning methods, long short-term memory networks, and adaptive neuro-fuzzy inference systems. Since the thermal dispatch in Brazil is closely related to energy prices, the predictions presented here are an interesting way of planning and decision-making for energy power systems.

Trend Prediction of Vibration Signals for Pumped-Storage Units Based on BA-VMD and LSTM

Under “dual-carbon” goals and rapid renewable energy growth, increasing start-stop frequency poses new challenges to safe operations of pumped-storage power plant equipment. Ensuring equipment safety and predictive maintenance under complex conditions urgently requires vibration warnings and trend forecasting for pumped-storage units. In this study, the measured vibration-signal characteristics of pumped-storage units in a strong background-noise environment are obtained using a noise-reduction method that integrates BA-VMD and wavelet thresholding. We monitored the vibration-signal data of hydroelectric units over a long period of time, and the measured vibration-signal characteristics of pumped-storage units in a strong background-noise environment are accurately obtained using a noise-reduction method that integrates BA-VMD and wavelet thresholding. In this paper, a BP neural network prediction model, a support vector machine (SVM) prediction model, a convolutional neural network (CNN) prediction model, and a long short-term memory network (LSTM) prediction model are used to predict the trend of vibration signals of the pumped-storage unit under different operating conditions. The model prediction effect is analyzed by using the different error evaluation functions, and the prediction results are compared with the predicted results of the four different methods. By comparing the prediction effects of the four different methods, it is concluded that LSTM has higher prediction accuracy and can predict the vibration trends of hydropower units more accurately.

Data-Driven Approaches for State-of-Charge Estimation in Battery Electric Vehicles Using Machine and Deep Learning Techniques

One of the most important functions of the battery management system (BMS) in battery electric vehicle (BEV) applications is to estimate the state of charge (SOC). In this study, several machine and deep learning techniques, such as linear regression, support vector regressors (SVRs), k-nearest neighbor, random forest, extra trees regressor, extreme gradient boosting, random forest combined with gradient boosting, artificial neural networks (ANNs), convolutional neural networks, and long short-term memory (LSTM) networks, are investigated to develop a modeling framework for SOC estimation. The purpose of this study is to improve overall battery performance by examining how BEV operation affects battery deterioration. By using dynamic response simulation of lithium battery electric vehicles (BEVs) and lithium battery packs (LIBs), the proposed research provides realistic training data, enabling more accurate prediction of SOC using data-driven methods, which will have a crucial and effective impact on the safe operation of electric vehicles. The paper evaluates the performance of machine and deep learning algorithms using various metrics, including the R2 Score, median absolute error, mean square error, mean absolute error, and max error. All the simulation tests were performed using MATLAB 2023, Anaconda platform, and COMSOL Multiphysics.

Road Pothole Detection Model Based on Local Attention Resnet18-CNN-LSTM

Abstract. In response to the low detection accuracy and slow speed of existing road pothole detection methods, a road pothole classification detection model based on local attention Resnet18-CNN-LSTM (Long Short-Term Memory network) is proposed. On the basis of Resnet18, a local attention mechanism and a CNN-LSTM combined model are added to propose a road pothole detection model based on local attention Resnet18-CNN-LSTM. The local attention mechanism is used to accurately extract specific target feature values, CNN is used to extract the spatial features of the input data, and LSTM enhances the detection model's extraction of sequential features and performs classification, thereby improving the accuracy of the road and pothole model. Experimental results show that on the training set, the accuracy of the local attention mechanism-based ResNet18-CNN-LSTM model reached 99.2188%, which is an increase of 0.7813% and 2.3438% compared to the ResNet34-CNN-LSTM and ResNet50-CNN-LSTM models under the same conditions, respectively. On the test set, the model's accuracy was 93.4437%, an increase of 0.5437% and 1.9867% compared to the ResNet34-CNN-LSTM and ResNet50-CNN-LSTM models, respectively. After dealing with overfitting issues through early stopping, the detection accuracy of this model has significantly improved compared to the detection models based on ResNet34 and ResNet50, with an increase of 1.2% and 1.49% respectively. The model shows faster processing speed in identification time, effectively retains the correlation and sequence features of the data, overcomes the problem of gradient disappearance in deep networks, and thereby enhances the extraction capability of local target features of road pothole images. The above results indicate that the local attention mechanism-based ResNet18-CNN-LSTM model shows superior performance in road pothole detection.

Improving efficiency and sustainability via supply chain optimization through CNNs and BiLSTM

Supply chain management is changing rapidly due to increasing complexity, uncertain demand, and the requirement for sustainable methods. Advanced technologies like Bidirectional Long Short-Term Memory networks (BiLSTM) and Convolutional Neural Networks (CNNs) can enhance supply chain processes. This paper proposes integrating CNNs and BiLSTM models to improve supply chain efficiency and sustainability. The proposed model employs CNNs to optimize resource allocation, uncover trends, and evaluate supply chain spatial linkages. Using BiLSTM models to capture temporal correlations allows accurate demand forecasting and proactive decision-making. Combining these models explains supply chain dynamics. CNNs and BiLSTM models' adaptive learning and real-time monitoring boost efficiency by responding quickly to changing situations. Predictive analytics optimizes inventory, lowers stock outs, and cuts lead times. Sustainability factors include transportation route optimization, carbon footprint minimization, and intelligent green-sourcing decision assistance. The proposed Hybrid Model achieved 94.65 % Specificity, 96.57 % Accuracy, 95.67 % Sensitivity and 0.85 % MCC. The result analysis demonstrates that the proposed model significantly improved the accuracy level. This research sheds light on supply chain difficulties from all sides. CNNs and BiLSTM models can boost operational efficiency and link supply chain practices with sustainability goals to produce a more sustainable global supply network.

Stacking BSRG-PLS: A physical and data-driven real-time stability safety analysis of arch dams during operation

Anti-sliding stability is the foundation for the normal operation of arch dams. In recent years, extreme weather has occurred frequently. It is important to grasp the anti-slide stability of arch dam (ASSAD) under complex load conditions in time. Currently, the ASSAD safety factor is primarily analyzed through the finite element method (FEM), which are time-consuming, labor-intensive, and lack timeliness. To address this, this paper proposes a real-time ASSAD analysis method during operation based on the Stacking BSRG-PLS model, which integrates a Bidirectional long short-term memory network, Residual neural network, Support vector machine, Gaussian process regression model and Partial Least Squares regression method. Firstly, based on the Stacking BSRG-PLS model and measured temperature combing with FEM, the transient temperature field of arch dam is established. Subsequently, a sample dataset containing the load data and the corresponding ASSAD safety factors calculated by FEM is constructed. Whereafter, using the Stacking BSRG-PLS model again, the real-time ASSAD safety factor is obtained based on the sample dataset. Case studies indicate that this method is effective and feasible, and has high analysis precision. It provides an effective way to quickly evaluate the ASSAD safety based on the measured data.