LSTM-based Model Research Articles

Mealtime prediction using wearable insulin pump data to support diabetes management

Many patients with diabetes struggle with post-meal high blood glucose due to missed or untimely meal-related insulin doses. To address this challenge, our research aims to: (1) study mealtime patterns in patients with type 1 diabetes using wearable insulin pump data, and (2) develop personalized models for predicting future mealtimes to support timely insulin dose administration. Using two independent datasets with over 45,000 meal logs from 82 patients with diabetes, we find that the majority of people (∼\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sim$$\\end{document} 60%) have irregular and inconsistent mealtime patterns that change notably through the course of each day and across months in their own historical data. We also show the feasibility of predicting future mealtimes with personalized LSTM-based models that achieve an average F1 score of > 95% with less than 0.25 false positives per day. Our research lays the groundwork for developing a meal prediction system that can nudge patients with diabetes to administer bolus insulin doses before meal consumption to reduce the occurrence of post-meal high blood glucose.

Rockburst early-warning method based on time series prediction of multiple acoustic emission parameters

Rockburst early-warning is crucial for ensuring safety in deep underground engineering. Existing methods primarily focus on classifying rockburst grades, making it challenging to provide timely warnings. This paper proposes a novel rockburst early-warning framework based on time series prediction of acoustic emission (AE) parameters. Six AE parameters (rise time, count, duration, amplitude, absolute energy, and peak frequency) were identified as potential indicators for rockburst early-warning based on rockburst tests. A sliding window method was applied to process normalized AE data, calculating statistical parameters of the local duration. An LSTM-based time series prediction model was developed to forecast the future evolution of these AE parameters. This, in turn, enabled the establishment of a comprehensive multi-indicator early-warning system. The Isolation Forest (IF) algorithm, an outlier detection method, was used to determine the warning thresholds for each indicator. The CRITIC weighting method was employed to integrate the six rockburst indicators into a single early-warning coefficient (EC), with EC=100 signifying the warning trigger condition. The results demonstrate that the proposed framework effectively captures the evolution trends of AE parameters, enabling proactive early warnings. This approach addresses the limitations of existing methods, such as reliance on experience for threshold determination, lack of a clear basis for multi-indicator weights, and difficulty in quantifying early-warning trigger conditions. The framework provides a new perspective for rockburst early-warning systems.

An Advisor Neural Network framework using LSTM-based Informative Stock Analysis

In the past years, the widespread diffusion of Artificial Intelligence (AI) in the finance domain transformed different services, with particular attention to the stock market. Although different AI-based approaches have been proposed for stock forecasting, they are focused on news content or sentiment without considering fundamental features and vice versa. In turn, other approaches rely on handmade rules or ones based on technical indicators for providing advice without considering contextual information that can strongly affect the stock market. In this paper, we propose an Advisor Neural Network framework using Long Short-Term Memory (LSTM)-based Informative Stock Analysis for Daily investment Advice. Specifically, the forecasting unit relies on a LSTM-based model, which combines technical indicators, contextual information, and financial data for stock forecasting. Successively, the advice unit provides next-day advice based on predicted information in conjunction with the proposed Heuristic Stocks Selection algorithm. This framework has been evaluated on the Stock and Cryptocurrencies markets, considering a subset of 417 stocks and 67 cryptocurrencies over three years, respectively. We compared the proposed framework with several state-of-the-art approaches, showing how it outperforms the baseline in both markets. Furthermore, we achieved a financial gain greater than 41%, despite the downward trend of the NASDAQ market in the quarter under review, and we obtained a 39.38% return on investment for the Cryptocurrencies market.

Javanese part-of-speech tagging using cross-lingual transfer learning

<span dir="ltr">Large datasets that are publicly available for POS tagging do not always exist </span><span dir="ltr">for some languages. </span><span dir="ltr">One of those languages is Javanese, a local language in </span><span dir="ltr">Indonesia, which is considered as a low-resource language. This research aims </span><span dir="ltr">to examine the effectiveness of cross-lingual transfer learning for Javanese POS </span><span dir="ltr">tagging by fine-tuning the state-of-the-art Transformer-based models (such as </span><span dir="ltr">IndoBERT, mBERT, and XLM-RoBERTa) using different kinds of source lan</span><span dir="ltr">guages that have a higher resource (such as Indonesian, English, Uyghur, Latin, </span><span dir="ltr">and Hungarian languages), and then fine-tuning it again using the Javanese lan</span><span dir="ltr">guage as the target language. We found that the models using cross-lingual trans</span><span dir="ltr">fer learning can increase the accuracy of the models without using cross-lingual </span><span dir="ltr">transfer learning by 14.3%–15.3% over LSTM-based models, and by 0.21%–</span><span dir="ltr">3.95% over Transformer-based models. Our results show that the most accurate </span><span dir="ltr">Javanese POS tagger model is XLM-RoBERTa that is fine-tuned in two stages </span><span dir="ltr">(the first one using Indonesian language as the source language, and the second </span><span dir="ltr">one using Javanese language as the target language), capable of achieving an </span><span dir="ltr">accuracy of 87.65%</span>

A non-linear Lasso and explainable LSTM approach for estimating tail risk interconnectedness

ABSTRACT Tail risk inter-connectivity is a significant aspect and a risk indicator that should be focused on. Many of the previous works have shown potential non-linearity in tail risk contagion. With the recent advancements in deep learning, Long-Short Term Memory (LSTM) networks have played an important role in sequential data prediction. We experiment with LASSO-based neural networks and interpretative LSTM model along with other machine learning approaches for investigating tail risk interconnectedness among the public banks of Japan. We also investigate the risk reception from large overseas banks in United States finding that medium-sized banks are more likely to receive international risks. Our studies show that LSTM-based model is an excellent fit for the scenario and total connectedness goes up during an economic crisis. The banks having larger market capitalization are more prone to emission and reception of tail risks. This is accompanied by exhibiting the impact of some major economic distresses on Japanese banking system. These results provide important information to regulators and policy makers.

LSTM-Based Multimodal Model for Self-Harm Prediction Combining Mental Health Case Reports and Biometric Signals

LSTM-Based Multimodal Model for Self-Harm Prediction Combining Mental Health Case Reports and Biometric Signals

A novel deep learning-based intrusion detection system for IoT DDoS security

Intrusion detection systems (IDS) for IoT devices are critical for protecting against a wide range of possible attacks when dealing with Distributed Denial of Service (DDoS) attacks. These attacks have become a primary concern for IoT networks. Intelligent decision-making techniques are required for DDoS attacks, which pose serious threats. The range of devices connected to the IoT ecosystem is growing, and the data traffic they generate is continually changing; the need for models more resistant to new attack types and existing attacks is of research interest. Motivated by this gap, this paper provides an effective IDS powered by deep learning models for IoT networks based on the recently published CICIoT2023 dataset. In this work, we improved the detection and mitigation of potential security threats in IoT networks. To increase performance, we performed preprocessing operations on the dataset, such as random subset selection, feature elimination, duplication removal, and normalization. A two-level IDS using deep-learning models containing binary and multiclass classifiers has been designed to identify DDoS attacks in IoT networks. The effectiveness of several deep-learning models in real-time and detection performance has been evaluated. We trained fully connected, convolutional, and LSTM-based deep learning models for detecting DDoS attacks and sub-classes. According to the results on a partially balanced sub-dataset, two staged models performed better than baseline models such as DNN (Deep Neural Networks), CNN (Convolutional Neural Networks), LSTM (Long Short Term Memory), RNN (Recurrent Neural Network).

LSTM-Based Forum Topic Classification Model

With the rapid development of information technology, forums have become an important platform for information exchange. However, the manual classification of forum topics consumes a significant amount of human resources and is prone to classification errors. To address this issue, this study proposes a forum topic classification model based on Long Short-Term Memory (LSTM) networks. By leveraging LSTM’s capability in text processing, the accuracy and efficiency of topic classification are significantly improved. This study used approximately 68,000 entries from 29 topic categories, scraped from Zhihu, for the experiments. Preprocessing steps such as text cleaning, tokenization, and word vectorization were performed, and a classification model with LSTM and Dropout layers was designed. The experimental results indicate that the model performs well in most topic classifications, although overfitting remains an issue for certain categories. The paper concludes by summarizing the advantages and limitations of the model and discusses the potential for improving classification accuracy through increasing data volume and optimizing the model in the future.

Prediction of time-dependent concrete mechanical properties based on advanced deep learning models considering complex variables

Evaluating the mechanical properties such as compressive strength, tensile strength, and modulus of elasticity (MOE) of concrete is crucial for the design, construction, and quality control of engineering projects. Developing a comprehensive theoretical-based numerical model to predict the concrete mechanical properties is considered challenging, owing to the diversity of admixtures and curing environment affecting the performance of modern concrete. The deep learning (DL) model gives a feasible approach as data-driven methods for accurate prediction of various material properties these years. However, predicting time-dependent mechanical properties of modern concrete with complex mix constituents by DL algorithms is still confined so far. In this article, artificial neuron network (ANN) and long short-term memory (LSTM) based DL models were established to predict concrete compressive strength, tensile strength, and MOE at different ages, considering the mix design, curing condition, curing time and tension testing method as input variables. LSTM outperformed ANN in terms of the statistical indicators for accuracy evaluation with high training and testing time costs. Bidirectional modification and multi-head attention mechanism were implemented to further improve the prediction accuracy of LSTM. It is shown that the multi-head attention bidirectional LSTM had more precise predicting results (R2=0.9355 for compressive strength, R2=0.8930 for tensile strength, R2=0.9584 for MOE) and fitting of the mechanical property-age curves. The LSTM-based models provided a promising prediction tool for time-dependent concrete mechanical properties with complex input parameters.

LSTM-Based Hybrid Model and Refractive Index Fault Detection for Terpolymerization in CSTR

LSTM-Based Hybrid Model and Refractive Index Fault Detection for Terpolymerization in CSTR

M5c-iDeep: 5-Methylcytosine sites identification through deep learning

5-Methylcytosine (m5c) is a modified cytosine base which is formed as the result of addition of methyl group added at position 5 of carbon. This modification is one of the most common PTM that used to occur in almost all types of RNA. The conventional laboratory methods do not provide quick reliable identification of m5c sites. However, the sequence data readiness has made it feasible to develop computationally intelligent models that optimize the identification process for accuracy and robustness. The present research focused on the development of in-silico methods built using deep learning models. The encoded data was then fed into deep learning models, which included gated recurrent unit (GRU), long short-term memory (LSTM), and bi-directional LSTM (Bi-LSTM). After that, the models were subjected to a rigorous evaluation process that included both independent set testing and 10-fold cross validation. The results revealed that LSTM-based model, m5c-iDeep, outperformed revealing 99.9 % accuracy while comparing with existing m5c predictors. In order to facilitate researchers, m5c-iDeep was also deployed on a web-based server which is accessible at https://taseersuleman-m5c-ideep-m5c-ideep.streamlit.app/.

Bidirectional Long Short-Term Memory Development for Aircraft Trajectory Prediction Applications to the UAS-S4 Ehécatl

The rapid advancement of unmanned aerial systems in various civilian roles necessitates improved safety measures during their operation. A key aspect of enhancing safety is effective collision avoidance, which is based on conflict detection and is greatly aided by accurate trajectory prediction. This paper represents a novel data-driven trajectory prediction methodology based on applying the Long Short-Term Memory (LSTM) prediction algorithm to the UAS-S4 Ehécatl. An LSTM model was designed as the baseline and then developed into a Staked LSTM to better capture complex and hierarchical temporal trajectory patterns. Next, the Bidirectional LSTM was developed for a better understanding of the contextual trajectories from both its past and future data points, and to provide a more comprehensive temporal perspective that could enhance its accuracy. LSTM-based models were evaluated in terms of mean absolute percentage errors. The results reveal the superiority of the Bidirectional LSTM, as it could predict UAS-S4 trajectories more accurately than the Stacked LSTM. Moreover, the developed Bidirectional LSTM was compared with other state-of-the-art deep neural networks aimed at aircraft trajectory prediction. Promising results confirmed that Bidirectional LSTM exhibits the most stable MAPE across all prediction horizons.

A Study on FGSS outlier Detection using an LSTM-based Auto Encoder Learning Model on Gas-powered Ships

A Study on FGSS outlier Detection using an LSTM-based Auto Encoder Learning Model on Gas-powered Ships

Construction of a mental health risk model for college students with long and short-term memory networks and early warning indicators

Abstract With the increasing social pressure and academic competition, the mental health (for convenience, abbreviated as MH) problems of college students are becoming increasingly prominent, but there are often challenges that are difficult to accurately predict and intervene in a timely manner. The aim of this article is to address the early warning needs of college students’ MH problems and construct a model that can timely identify the MH problems of college students. The experiment collected MH related data from college students in S city, and analyzed and trained these data using the Long short-term memory (LSTM) network model. By changing the number of hidden layers, learning rate, batch size, and epoch times, the most suitable training effect was achieved. By using the time-series characteristics of the LSTM model, the selected parameters from the experiment can better capture the changing trends of college students’ MH status, thereby improving prediction accuracy. Finally, three stage indicators of low, medium, and high were set up for early warning of the predicted results, in order to effectively and timely take measures. The research results indicated that the constructed model achieved a minimum regularization loss of 0.0674 after training. Finally, the adjusted model was used to predict the test set, with an average accuracy of 0.852 and an average accuracy of 0.906. The LSTM-based MH risk model performed well in predicting college students’ MH problems and could identify potential risk factors in a timely manner.

Machine learning-based predictive control of an electrically-heated steam methane reforming process

Hydrogen plays a crucial role in improving sustainability and offering a clean and efficient energy carrier that significantly reduces greenhouse gas emissions. However, the primary method of industrial hydrogen production, steam methane reforming (SMR), relies on the combustion of hydrocarbons as the heating source for the reforming reactions, resulting in significant carbon emissions. To address this issue, an experimental setup of an electrically-heated steam methane reformer (e-SMR) has been constructed at UCLA, and a lumped first-principle dynamic process model was built based on parameters estimated from the experimental data in a previous study. Subsequently, the first-principle dynamic process model was implemented into the computational model predictive control (MPC) scheme, successfully driving the hydrogen production rate to the desired setpoint. While these works are important and pave the way for developing MPC for large-scale e-SMR processes, the first-principle process model may not accurately reflect the actual process behavior, particularly as the process behavior changes with time. Therefore, the development and establishment of an adaptive data-driven approach for implementing model predictive control in the e-SMR process is necessary. To address this need, the present work investigates the construction of recurrent neural network (RNN) models for an e-SMR process in-depth, utilizing data from an experimentally-validated first-principle model. Specifically, a long short-term memory (LSTM) layer was utilized in the RNN model to effectively capture the complex correlations present in long-term sequential data. Subsequently, this LSTM-based RNN process model was employed to design an MPC, and its performance was evaluated through comparison with proportional–integral (PI) control. To address potential disturbances and variability in a typical e-SMR process, three distinct approaches were developed: MPC with an integrator, MPC with real-time online retraining (transfer learning), and offset-free MPC. These approaches effectively eliminated the offset caused by disturbances. Overall, this study underscores the effectiveness of utilizing RNN models to capture process dynamics in an experimental e-SMR process. It also outlines strategies for employing RNN-based control and multiple approaches to address disturbances in general processes with partially infrequent and delayed measurement feedback. This approach is particularly valuable in scenarios where developing first-principle models for a new process may be challenging.

LSTM Based New Probability Features Using Machine Learning to Improve Network Attack Detection

This project focuses on improving the detection of network attacks by using a machine learning technique known as Long Short-Term Memory (LSTM) networks. LSTM networks are a type of neural network that excels at analyzing sequences of data, making them well-suited for identifying patterns associated with network intrusions. To enhance the LSTM model's effectiveness, we introduce new probability features that help the model better distinguish between normal and malicious activities. Our approach includes collecting network data, preprocessing it to make it suitable for training, and then using this data to train the LSTM model. We evaluate the model's performance using a range of metrics to ensure its accuracy and reliability. The results indicate that our method significantly improves the detection rate of network attacks while also reducing the number of false alarms. This means that our LSTM-based model not only catches more real threats but also makes fewer mistakes in identifying normal activities as attacks. Overall, this project showcases the potential of advanced machine learning techniques, like LSTM networks, to enhance cyber security measures and protect against network threats more effectively.

Symmetry-Enhanced LSTM-Based Recurrent Neural Network for Oscillation Minimization of Overhead Crane Systems during Material Transportation

This paper introduces a novel methodology for mitigating undesired oscillations in overhead crane systems used in material handling operations in the industry by leveraging Long Short-Term Memory (LSTM)-based Recurrent Neural Networks (RNNs). Oscillations during material transportation, particularly at the end location, pose safety risks and prolong carrying times. The methodology involves collecting sensor data from an overhead crane system, preprocessing the data, training an LSTM-based RNN model that incorporates symmetrical features, and integrating the model into a control algorithm. The control algorithm utilizes swing angle predictions from the symmetry-enhanced LSTM-based RNN model to dynamically adjust crane motion in real time, minimizing oscillations. Symmetry in this framework refers to the balanced and consistent handling of oscillatory data, ensuring that the model can generalize better across different scenarios and load conditions. The LSTM-based RNN model accurately predicts swing angles, enabling proactive control actions to be taken. Experimental validation demonstrates the effectiveness of the proposed approach, achieving an accuracy of approximately 98.6% in swing angle prediction. This innovative approach holds promise for transforming material transportation processes in industrial settings, enhancing operational safety, and optimizing efficiency.

Comparative Performance of Neural Network Architectures in Gold Price Prediction: a Study of GRU, N-BEATS, LSTM and Hybrid Models

This study assesses the efficacy of a variety of neural network models in predicting gold prices, with a particular emphasis on GRU, N-BEATS, N-BEATS-GRU, N-BEATS-LSTM, and LSTM. We evaluated the models using a comprehensive dataset and metrics such as MSE, MAE, MAPE, and RMSE. The GRU model demonstrated the highest MAPE of 3.10%, as indicated by the results. N-BEATS was the second-best-performing model. In stark contrast to expectations, hybrid models, notably N-BEATS-LSTM, underperformed. Compared to LSTM-based models, visual analysis demonstrated that GRU and N-BEATS more effectively captured short-term and long-term trends. The results indicate that simplified models, such as GRU, may be more effective than complex hybrids in predicting gold prices.

An empirical study on KDIGO-defined acute kidney injury prediction in the intensive care unit.

Acute kidney injury (AKI) is a syndrome that affects a large fraction of all critically ill patients, and early diagnosis to receive adequate treatment is as imperative as it is challenging to make early. Consequently, machine learning approaches have been developed to predict AKI ahead of time. However, the prevalence of AKI is often underestimated in state-of-the-art approaches, as they rely on an AKI event annotation solely based on creatinine, ignoring urine output. We find that optimal performance is achieved by using GBDT with the time-based stacking technique (AUPRC = 65.7%, compared with the LSTM-based model's AUPRC = 62.6%), which is motivated by the high relevance of time since ICU admission for this task. Both models show mildly reduced performance in the limited training data setting, perform fairly across different subcohorts, and exhibit no issues in gender transfer. The code to reproduce the findings of our manuscript can be found at: https://github.com/ratschlab/AKI-EWS.

A Framework of Digital Twins for Improving Respiratory Health and Healthcare Measures

The investigation describes an inventive use of digital twin technology and LSTM-based machine learning models for real-time patient lung disease monitoring and nutrition planning. The suggested application uses various patient healthcare data, treatment processes, dietary habits, and real-time sensor information to construct digital twins, which are virtual reproductions of specific patients. The LSTM model is trained on this large dataset to predict patient health improvements and dietary needs. For each patient’s digital twin, the program provides personalized treatment plans and nutritional advice, enabling proactive interventions and optimizing patient care. Using performance measures, the trained LSTM model achieves high scores for accuracy (92%), precision (89%), recall (93%), and F1 score (91%), proving its usefulness in generating credible health predictions. Patient feedback on the program shows that patients (98.8%) agree on the accuracy and importance of health feedback, as well as the convenience of access to health information (95.4%). The application’s response rate study reveals an average response rate of 85.87%, assuring prompt feedback. To secure patient information, the study emphasizes data privacy and security, adopting multilayered authentication and data encryption. The outcomes of this study demonstrate the application’s potential to revolutionize patient-centered healthcare by providing data-driven, personalized solutions to patients and healthcare professionals.