Short-term Memory Research Articles

Enhancing water quality prediction with advanced machine learning techniques: An extreme gradient boosting model based on long short-term memory and autoencoder

Achieving carbon neutrality in the pulp and paper industry necessitates effectively recycling pulp and papermaking wastewater, where continuous monitoring of effluent quality indices is crucial. This study suggests a novel machine learning-based model named LSTMAE-XGBOOST that integrates the feature extraction capabilities of autoencoder, the sequential feature learning capabilities of LSTM, and the high prediction accuracy of XGBOOST. This model is capable of extracting the complex relationships, non-Gaussian characteristics, and time series features from the papermaking wastewater data, and it demonstrates superior predictive performance. Compared to traditional machine learning models, the proposed model exhibits higher prediction accuracy. Specifically, when contrasted with partial least squares regression, LSTMAE-XGBOOST achieves a 40% increase in R2 and a 35% reduction in RMSE. Further comparative assessments against other machine learning-based hybrid models with similar structures confirm the superiority of integrating LSTM and XGBOOST within the hybrid model approach. This study contributes a compelling methodology for modeling effluent quality indices, offering significant implications for environmental management in the pulp and paper industry.

Non-destructive estimation for Kyoho grape shelf-life using Vis/NIR hyperspectral imaging and deep learning algorithm

Grape shelf-life estimation is a substantial challenge for the grape industry. The objective of this study is to investigate the potential of grape shelf-life estimation using HSI technique and a deep learning algorithm. The visible and near-infrared (400.68–1001.61 nm) hyperspectral reflectance images data of grape samples was acquired and preprocessed with different spectral preprocessing methods. Additionally, a stacked denoising autoencoder (SDAE)-based deep learning algorithm was developed to extract deep features from pixel-level hyperspectral data of grapes, and then these features were used as inputs to establish support vector machine (SVM) models for estimating grape shelf-life. Furthermore, SVM, one-dimensional convolutional neural network (1D CNN) and long short-term memory (LSTM) models were used as traditional machine learning and end to end models for comparison. The results demonstrated that the SDAE-SVM model achieved reasonable recognition accuracy of 100 % and 98.125 % for the shelf-life of grapes in the training and test sets, respectively. The overall results suggested that SDAE-based deep learning method can be used as a powerful tool to deal with large-scale hyperspectral data as well as this research confirms the feasibility of non-destructive estimation for grapes shelf-life by the combination of HSI technique and deep learning method, which would provide a valuable guidance for shelf-life estimation of other postharvest fruit.

GWO-ANFIS-RD3PG: A reinforcement learning approach with dynamic adjustment and dual replay mechanism for building energy forecasting

Reliable prediction of building energy consumption is essential for effective and sustainable energy management. Traditional forecasting methods, however, face challenges when dealing with sudden changes in energy consumption. To address the above-mentioned issue, we introduce the GWO-ANFIS-RD3PG prediction model in this paper. This model aims to ensure global forecasting accuracy while minimizing prediction errors at sudden change points. A Grey Wolf Optimization (GWO) algorithm with the Adaptive Neuro-Fuzzy Inference System (ANFIS) is proposed to predict the energy consumption type and its fluctuation magnitude for the next time step. Within the Recurrent Dual Experience Replay Buffer DDPG (RD3PG) prediction module, we employ Long Short-Term Memory (LSTM), as the actor network in the Deep Deterministic Policy Gradient (DDPG), to account for long-term dependencies in time series data. Notably, we introduce an adaptive action modification mechanism that allows the agent to optimize actions based on the output of the aforementioned ANFIS, enabling real-time adjustments during sudden change. To further enhance the model’s performance, we adopt a dual experience replay mechanism to store data samples from sudden change points, capturing insight information during these anomalies. Experimental results on two real-world datasets demonstrate that this method reduces MAE by 5.81% and 13.47%, and RMSE by 5.69% and 15.21% compared to the state-of-art models, showing the significance of the proposed method for energy efficiency improvement in real-world building operations.

A hybrid model based on complementary ensemble empirical mode decomposition, sample entropy and long short-term memory neural network for the prediction of time series signals in NPPs

Accurate and reliable predictions are fundamental for the condition monitoring and maintenance of components and systems in nuclear power plants (NPPs). In this work, we propose a hybrid condition prediction approach based on the combination of complementary ensemble empirical mode decomposition (CEEMD), sample entropy (SampEn) and optimized long short-term memory (LSTM) neural network. Firstly, the CEEMD decomposes the time series signals into multiple subsequences called intrinsic mode functions (IMFs), by doing so, the complexity of the time series signals can be reduced, and this facilitates the accurate prediction of the original signals. Then, in order to reduce the calculation cost of prediction models for subsequences, SampEn is used to measure the complexities of the IMFs, and the IMFs whose values of SampEn are lower than the average are aggregated into a new component. Finally, the LSTM with the hyperparameters optimized by the Bayesian optimization algorithm (BOA) is used to perform the prediction of each component. The prediction results of the original signals are reconstructed by synthesizing the predictions of all components. The proposed hybrid prediction model is utilized on the time series signals collected from an NPP. The results obtained show that the proposed approach can capture the characteristics of the signals and has better performance in prediction accuracy than other models.

Core temperature estimation of lithium-ion battery based on numerical model fusion deep learning

Temperature has a critical impact on the lifespan and safety of lithium batteries. This paper proposes a battery core temperature estimation method based on numerical model fused with long short-term memory (LSTM) neural network. The proposed technique extracts features from the numerical model, estimates the volume-averaged temperature by electrochemical impedance spectroscopy (EIS), uses an LSTM neural network to learn thermodynamic parameters and complex calculations, which takes advantage of the strengths of each method, and achieves accurate core temperature estimation. The effects of state of charge (SOC) and temperature on EIS are explored, impedance properties are selected on the criteria of robustness and rapidity, and the estimation of the volume-averaged temperature is achieved using the imaginary part of the impedance. The proposed method can achieve root mean squared error (RMSE) of less than 0.28 °C and mean absolute error (MAE) of less than 0.23 °C. The proposed method has advantages of high estimation accuracy and does not require an electrothermal model. It also considers the effect of ambient temperature and has a good generalization capability.

Cross-sectional analysis of timber boards using convolutional long short-term memory neural networks

This paper proposes a one-dimensional convolutional long short-term memory (1D-CNN-LSTM) model for estimating the pith position and average ring width in Norway spruce timber boards. The model predicts these cross-sectional parameters by processing sequences of light-intensity signals derived from optical scans of the board’s four surfaces. The dataset used for training the model consists of synthetic boards sawn from simulated 3D logs. The model was evaluated on a dataset consisting of 552 end cross-sections from actual Norway spruce boards. Comparisons between the automatic and manual pith and ring width estimations demonstrated a very good accuracy. The computational speed of the model was more than twice as fast as the quickest method available in the literature. A large set of boards was then used to determine the advantages of incorporating the automatically determined average ring width in formulating indicating properties for machine strength grading. This evaluation revealed that the average ring width could, in certain situations, compensate for unknown variables such as density or resonance frequency in predicting the tensile and bending strength of Norway spruce boards.

Probing an LSTM-PPO-Based reinforcement learning algorithm to solve dynamic job shop scheduling problem

With the growth of personalized demand and the continuous improvement in social productivity, the large-scale and few-variety centralized production model is gradually transitioning towards a personalized model of small batches and multiple varieties, which makes the manufacturing process of the job shop increasingly complex. Furthermore, disruptive events such as machinery failures and rush orders in the job shop increase the uncertainty and variability of the production environment. Traditional scheduling methods are usually based on fixed rules and heuristic algorithms, which are difficult to adapt to constantly changing production environments and demands. This may lead to inaccurate scheduling decisions and hinder the optimal allocation of job shop resources. To solve the dynamic job shop scheduling problem (JSP) more effectively, this paper proposes a Reinforcement Learning (RL) optimization algorithm integrating long short-term memory (LSTM) neural network and proximal policy optimization (PPO). It can dynamically adjust scheduling strategies according to the changing production environment, achieving comprehensive status awareness of the job shop environment to make optimal scheduling decisions. First, a state-aware network framework based on LSTM-PPO is proposed to achieve real-time perception of job shop state changes. Then, the state and action space of the job shop are described within the context of the state-aware network framework. Finally, an experimental environment is established to verify the algorithm’s effectiveness. Training the LSTM-PPO algorithm makes it feasible to achieve better performance than other scheduling methods. By comparing the initial planning time with the actual completion time of the rescheduling decision under different dynamic disturbances, the efficiency of the proposed algorithm is verified for the dynamic JSP

Assessing forecast performance of daily reference evapotranspiration: A comparison of equations, machine and deep learning using weather forecasts

An accurate forecast of short-term reference evapotranspiration (ET0) is crucial for effective farm and irrigation scheduling management. The techniques of forecasting ET0 have progressed from empirical equations to artificial intelligence-based models. However, gaps have remained in comprehensively evaluating the ET0 forecast performance discrepancies between various methods based on weather forecasts in addition to historical meteorological data, as well as investigating the effect of the amount of input variables on ET0 forecast accuracy. Here, the study evaluated the forecast accuracy of five conventional equations calibrated and six machine learning (ML) and deep learning (DL) models in predicting daily ET0 for a lead time of 1–7 days in the North China Plain (NCP). Comparative evaluations indicate that the Analytical Penman-Monteith (APM) is the optimum empirical equation for forecasting ET0, but its accuracy decreases as lead times increase. Conversely, ML and DL models consistently outperform empirical equations, characterized by lower mean absolute error (MAE) and root mean square error (RMSE) as well as correlation coefficient and Nash-Sutcliffe efficiency coefficient approaching 1 across all lead times. Notably, the Bidirectional Long short-term memory (Bi-LSTM) model outperforms other models, maintaining robust and effective forecast performance even on day 7. Moreover, with the increasing input variables, combining weather forecasts and historical meteorological data, the forecast performance of all models is improved, with significant enhancement observed in the Bi-LSTM model. This work provides valuable insights for choosing appropriate models to forecast ET0, holding essential in regional managing farm and irrigation systems.

Classification of Internal and External Distractions in an Educational VR Environment Using Multimodal Features.

Virtual reality (VR) can potentially enhance student engagement and memory retention in the classroom. However, distraction among participants in a VR-based classroom is a significant concern. Several factors, including mind wandering, external noise, stress, etc., can cause students to become internally and/or externally distracted while learning. To detect distractions, single or multi-modal features can be used. A single modality is found to be insufficient to detect both internal and external distractions, mainly because of individual variability. In this work, we investigated multi-modal features: eye tracking and EEG data, to classify the internal and external distractions in an educational VR environment. We set up our educational VR environment and equipped it for multi-modal data collection. We implemented different machine learning (ML) methods, including k-nearest-neighbors (kNN), Random Forest (RF), one-dimensional convolutional neural network - long short-term memory (1 D-CNN-LSTM), and two-dimensional convolutional neural networks (2D-CNN) to classify participants' internal and external distraction states using the multi-modal features. We performed cross-subject, cross-session, and gender-based grouping tests to evaluate our models. We found that the RF classifier achieves the highest accuracy over 83% in the cross-subject test, around 68% to 78% in the cross-session test, and around 90% in the gender-based grouping test compared to other models. SHAP analysis of the extracted features illustrated greater contributions from the occipital and prefrontal regions of the brain, as well as gaze angle, gaze origin, and head rotation features from the eye tracking data.

A novel ensemble machine learning exposure model system for ground-level ozone at the national scale: A case of mainland China from 2013 to 2020

In epidemiological research, accurate estimation of historical ground-level ozone (O3) concentrations with enhanced spatiotemporal resolution is crucial for effective exposure assessment. The current state-of-the-art for estimating air pollutant concentrations is a two-stage ensemble method that integrates outputs from multiple machine learning algorithms. Despite its effectiveness, opportunities exist to refine this approach for more precise O3 estimation. In this study, we propose an enhanced ensemble method that incorporates four key strategies. First, we employ high-resolution spatiotemporal predictors derived from prior machine learning studies for refined secondary learning. Second, we use sophisticated algorithms, including categorical gradient boosting, deep neural network, random forest, stochastic variable Gaussian process, transformer, and a combination of convolutional neural network and long short-term memory neural network, as sublearners to enhance learning capabilities. Third, we spatiotemporally split the sample set and then train submodels separately on each subset to eliminate the unobserved spatiotemporal heterogeneity. Finally, we apply a complex machine learning algorithm, rather than the generalized additive model, for integrating sublearner predictions, enabling the capture of intricate nonlinear relationships beyond basic spatiotemporal linear weights. To validate these improvements, we estimated daily maximum 8-h moving average O3 concentrations ([O3]MDA8) across Chinese mainland from 2013 to 2020 at a 1 km spatial resolution. The proposed method demonstrated notable accuracy, achieving an out-of-station determination coefficient (R2) of 0.943 and a root-mean-square error (RMSE) of 10.197 μg/m3. This performance marks a nearly 15% improvement over the best existing Chinese O3 exposure model based on a single algorithm and also surpasses previous studies utilizing traditional ensemble methods for other air pollutants. Our enhanced ensemble approach significantly bolsters the reliability and robustness of future environmental epidemiological studies by further mitigating “misclassification” errors.

A Systematic Literature Review on The Role of LSTM Networks in Capturing Temporal Dependencies in Data Mining Algorithms

Data mining plays a crucial role in extracting meaningful insights from large and complex data sets, with broad applications in sectors like finance, health care, and market analysis. Traditional techniques—such as classification, clustering, association rule mining, regression, and anomaly detection—are effective for analyzing structured data but struggle with sequential data due to the challenges of modeling temporal dependencies. Long Short Term Memory (LSTM) networks, a specialized form of Recurrent Neural Networks (RNNs), provide a solution to these challenges. By incorporating memory cells and gating mechanisms, LSTM effectively manage long-term dependencies and address issues like vanishing and exploding gradients. This paper reviews the impact of LSTM networks on data mining, analyzing over 60 key publications. By synthesizing concepts and recent advancements, the review underscores how LSTMs enhance the ability of data mining algorithms to capture and predict temporal patterns, reflecting current research trends and innovations

Integrating Twitter Sentiment with ML Models for Enhanced Stock Forecasting

This research investigates the integration of Twitter sentiment analysis with machine learning models, specifically Long Short-Term Memory (LSTM), Facebook Prophet, and Neural Prophet, to enhance the accuracy of stock market predictions. While these models are accurate in capturing long-term trends and seasonality, they often fail when exploring the short-term volatility in stock prices. By using real-time social media sentiments, we aim to improve short-term prediction accuracy. Sentiment data from Twitter, extracted and classified using NLP techniques, was integrated with the models to predict stock prices for Apple Inc. (AAPL), Paramount Global (PARA), and BP plc (BP) over the period 2016-2024. The study compares the performance of the models using Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). Results show that Neural Prophet stood the best in terms of its accuracy and efficiency in stock predictions when compared to LSTM and Facebook Prophet, reducing RMSE by 25% and MAE by 20%. While the integration of sentiment analysis improved prediction accuracy, limitations remain in predicting the scale of sudden market changes. The findings highlight the potential of sentiment analysis in stock prediction but also call for further model enhancements to fully address short-term market volatility.

Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image and Video Forgeries

This paper introduces a hybrid architecture combining Long Short-Term Memory (LSTM) networks and an encoderdecoder model for the detection and localization of image and video forgeries. The proposed system leverages resampling features and LSTM cells to identify manipulation patterns such as splicing and retouching in multimedia content. By utilizing a combination of spatial and temporal features, the model achieves high precision in detecting forged regions. Extensive testing on diverse datasets demonstrates the robustness of the proposed method.

Design and development of an ai-driven communication model for stimulating memory recall in patients with Alzheimer’s and other neurodegenerative diseases

This study presents the development and evaluation of an AI-driven communication tool designed to stimulate memory recall in patients with Alzheimer’s disease and other neurodegenerative disorders. The tool integrates machine learning algorithms, including Random Forest and Long Short-Term Memory (LSTM) networks, with mass communication theories to deliver personalized and adaptive communication strategies. The AI models were trained and tested on patient interaction data, enabling the tool to dynamically adjust its prompts based on real-time cognitive states. Performance metrics such as accuracy, precision, and AUC-ROC were used to assess model efficacy, with the Random Forest model achieving the highest performance across all metrics. Clinical trials demonstrated significant improvements in memory recall and emotional engagement in patients using the AI tool compared to standard cognitive stimulation therapy. Caregivers also reported higher satisfaction with the tool's usability and impact on patient interaction quality. These findings highlight the potential of AI-driven, personalized communication tools to revolutionize Alzheimer's care, offering scalable, adaptive interventions that improve cognitive and emotional health. Academically, this work advances the integration of AI and health communication, emphasizing culturally sensitive and patient-centered approaches. In healthcare, it presents a promising non-pharmacological intervention that can be easily scaled for widespread use, potentially reducing caregiver burden and improving patient outcomes.

Fortifying cloud environments against data breaches: A novel AI-driven security framework

As cloud computing continues to dominate the modern technological landscape, organizations face growing challenges in preventing data breaches and sophisticated cyber threats. The increasing complexity and scale of cloud environments require advanced security mechanisms to address evolving threats. This paper introduces "SecureCloudAI," a cutting-edge AI-driven security framework designed to fortify sensitive data within cloud infrastructures. SecureCloudAI leverages a hybrid approach that combines machine learning models like Random Forest and deep learning techniques, including Long Short-Term Memory (LSTM) networks, to detect, classify, and respond to potential security breaches in real-time. The system offers robust malware detection, network traffic analysis, and web intrusion detection while maintaining scalability and efficiency across large cloud environments. Experimental results demonstrate SecureCloudAI's high accuracy, with malware detection at 94.78% and network traffic classification at 90.92%, ensuring the system can handle both complex and emerging threats. This AI-driven solution marks a significant advancement in cloud security, providing organizations with an adaptive and scalable tool to safeguard their data against the ever-growing cyber threat landscape.

A Novel Deep Learning Model for Stock Prediction Integrating Historical Prices, Core Volatility Trends and Financial News

Accurate stock price forecasting is crucial for investors and analysts aiming to make informed decisions and mitigate risks in financial markets. While deep learning models have proven effective in identifying complex patterns in quantitative data, they often struggle to account for non-quantitative factors such as corporate events, market sentiment, and regulatory changes that can cause unpredictable stock price movements. To address this challenge, an ensemble model is introduced that integrates historical price data, financial news, and stock fundamentals to predict next-day stock prices. The model combines one-dimensional Convolutional Neural Networks (1D CNNs), Long Short-Term Memory networks (LSTMs), and Gated Recurrent Units (GRUs), applied independently to each data stream and concurrently as a unified ensemble, to capture both short-term and long-term market behaviors. NVIDIA (NVDA) stock was selected for testing due to its high volatility and rapid price fluctuations, presenting a challenging case for predictive modeling. Using data spanning from April 2014 to March 2024, the model achieved a coefficient of determination (R²) of 0.983 and a mean absolute error (MAE) of $12.72. These results suggest that the ensemble approach effectively captures both quantitative and qualitative factors, making it a promising tool for improving stock price prediction in volatile market conditions.

The Comprehensive Investigation for Covid-19 Trend Prediction Through Machine Learning and Deep Learning

Corona Virus Disease (Covid-19) has surely been a challenging problem to solve for the past few years. Due to the diversity in the form of dataset, it is essential to obtain accurate predictive results of Covid-19 trends. This paper analyzes different artificial intelligence methods used in Covid-19 trend prediction, including several machine learning and deep learning methods. More specifically, this work investigates linear regression, random forest, and decision trees in terms of machine learning and delves into Artificial Neural Network (ANN) as well as Long Short-Term Memory (LSTM) for deep learning. By comparing various past works, the effectiveness of machine learning and deep learning methods is achieved by their hidden algorithms, such as the Multiple Linear Regression (MLR) model for linear regression analysis. Incorporation with other models or methods is applied in deep learning. For example, Ensemble Empirical Mode Decomposition (EEMD) is included in ANN structure to decrease the noises within the Covid-19 datasets. Furthermore, the paper also inquiries into potential improvement of some drawbacks in predictive results for Covid-19 trends by reviewing related works of expert system and transfer learning as well as domain adaptation. The machine learning and deep learning models could provide accurate predictive results as a reference for related organizations to consider or establish insightful policies.

An efficient deep learning approach for ultimate bond strength estimations of corroded bar and concrete

Abstract The corrosion behavior of the reinforced concrete structure depends heavily on the interfacial bond behavior between steel and concrete. Over the years, the deterioration of integrated reinforced steel has weakened this bond and potentially led to structural problems. Conventional methods of bond strength evaluation, such as pullout and bond beam tests, is frequently intrusive and tedious. Therefore, there is a growing need for non-intrusive, effective, and reliable forecasting algorithms capable of assessing bond deterioration caused by corrosion. Traditional algorithms for predicting bond strength make it difficult to capture the complex nature of steel-concrete bonds. The present study proposes two different deep learning algorithms, i.e., convolutional neural networks (CNN) and long short-term memory (LSTM), for predicting maximum bond strength in the presence of corrosion. The predictive model is based on a comprehensive dataset comprising 218 datasets from previous studies encompassing diverse input and output variables for predicting the models. The models were trained and tested using the given data to improve early predictions of corrosion-induced bond degradations. The predictive model’s effectiveness was assessed by applying various performance metrics. From this study, the CNN model exhibits higher accuracy and efficiency with mean absolute error, root mean square error, and mean absolute percentage error of 0.25, 0.28, and 95.72, respectively, for predicting ultimate bond strength estimations. The findings of this study provide an accurate and robust prediction model to improve the reliability and safety of the concrete structure by enhancing the residual load-bearing capacity of the concrete structure that has undergone corrosion.

Artificial Intelligence-based Enhanced Brain Tumor Prediction: A Comprehensive Investigation of Machine Learning and Deep Learning

Brain tumors pose a significant threat to patients’ quality of life and survival rates, with traditional diagnostic methods often falling short due to their time-consuming nature and susceptibility to high misdiagnosis rates. Recent progress in Artificial Intelligence (AI), especially in Machine Learning (ML) and Deep Learning (DL), offer promising alternatives for the prediction and diagnosis of brain tumors. AI models, including Support Vector Machine (SVM), Random Forests, Logistic Regression, Artificial Neural Network (ANN), Convolutional Neural Network (CNN), and Long Short-Term Memory (LSTM) models networks, have demonstrated superior performance in processing complex medical data, thereby enhancing predictive accuracy and aiding clinical decision-making. This review systematically evaluates the application of these AI techniques in brain tumor prediction, highlighting their strengths and limitations, as well as the challenges faced in terms of model interpretability, data applicability, and patient privacy. Furthermore, this paper explored future prospects for improving model interpretability, transfer learning and domain adaptation for enhancing model applicability, and federated learning for preserving patient privacy. By addressing these issues, Artificial Intelligence can significantly advance brain tumor diagnosis and treatment, ultimately improving patient outcomes.

Single-Channel Sleep EEG Classification Method Based on LSTM and Hidden Markov Model

Background: The single-channel sleep EEG has the advantages of convenient collection, high-cost performance, and easy daily use, and it has been widely used in the classification of sleep stages. Methods: This paper proposes a single-channel sleep EEG classification method based on long short-term memory and a hidden Markov model (LSTM-HMM). First, the single-channel EEG is decomposed using wavelet transform (WT), and multi-domain features are extracted from the component signals to characterize the EEG characteristics fully. Considering the temporal nature of sleep stage changes, this paper uses a multi-step time series as the input for the model. After that, the multi-step time series features are input into the LSTM. Finally, the HMM improves the classification results, and the final prediction results are obtained. Results: A complete experiment was conducted on the Sleep-EDFx dataset. The results show that the proposed method can extract deep information from EEG and make full use of the sleep stage transition rule. The proposed method shows the best performance in single-channel sleep EEG classification; the accuracy, macro average F1 score, and kappa are 82.71%, 0.75, and 0.76, respectively. Conclusions: The proposed method can realize single-channel sleep EEG classification and provide a reference for other EEG classifications.