Performance Of Long Short-term Memory Research Articles

Towards Fault Detection in Industrial Equipment through Energy Consumption Analysis: Integrating Machine Learning and Statistical Methods

Accurately forecasting the energy consumption of industrial equipment and linking these forecasts to equipment health has become essential in modern manufacturing. This capability is crucial for advancing predictive maintenance strategies to reduce energy consumption and greenhouse gas emissions. In this study, we propose a hybrid model that combines Long Short-Term Memory (LSTM) for energy consumption prediction with a statistical change-point detection algorithm to identify significant shifts in consumption patterns. These shifts are then correlated with the equipment’s health status, providing a comprehensive overview of energy usage and potential failure points. In our case study, we began by evaluating the prediction model to confirm the performance of LSTM, comparing it with several machine learning models commonly used in the literature, such as Random Forest, Support Vector Machines (SVM), and GRU. After assessing different loss functions, the LSTM model achieved the strongest prediction accuracy, with an RMSE of 0.07, an MAE of 0.0188, and an R2 of 92.7%. The second part of the model, which focuses on detecting change points in consumption patterns, was evaluated by testing several cost functions combined with binary segmentation and dynamic programming. Applied to a real-world case, it successfully detected a change point two months before equipment failure, offering the potential to reduce energy consumption by 27,052 kWh. This framework not only clarifies the relationship between equipment health and CO2 emissions but also provides actionable insights into emission reduction, contributing to both economic and environmental sustainability.

Performance of recurrent neural networks with Monte Carlo dropout for predicting pharmacokinetic parameters from dynamic contrast‐enhanced magnetic resonance imaging data

AbstractPurposeTo quantitatively evaluate the performance of two types of recurrent neural networks (RNNs), long short‐term memory (LSTM) and gated recurrent units (GRU), using Monte Carlo dropout (MCD) to predict pharmacokinetic (PK) parameters from dynamic contrast‐enhanced magnetic resonance imaging (DCE‐MRI) data.MethodsDCE‐MRI data for simulation studies were synthesized using the extended Tofts model and a population‐averaged arterial input function (AIF). The ranges of PK parameters for training the RNNs were determined from data of patients with brain tumors. The effects of the number of training samples, number of hidden units, dropout rate (DR), and bolus arrival time delay and dispersion in AIF on the accuracy of the PK parameters were investigated, and the uncertainties for different DRs and peak signal‐to‐noise ratios (PSNRs) were quantified. For comparison, PK parameters were estimated using the nonlinear least‐squares method. In the clinical studies, the PK parameter and uncertainty images were generated by applying the trained RNNs to DCE‐MRI data.ResultsCompared with GRU, the computational cost for training the LSTM was significantly higher. The prediction accuracy of GRU decreased with decreasing numbers of training samples and hidden units, whereas the performance of LSTM remained stable. Despite an increased computational cost, MCD reduced the prediction error at low PSNR and improved the quality of PK parameter images. The simulation results recommended using a DR of 0.25–0.5 at low PSNR and ≤ 0.25 for other PSNRs. The clinical studies recommended using a DR of 0.25 and 0.5 for LSTM and GRU, respectively.ConclusionsMCD is effective in quantifying uncertainty in PK parameter prediction from DCE‐MRI data and improves their performance, particularly at low PSNR; however, at the expense of increased computational cost. This study helps deepen our understanding of RNNs with MCD and select suitable hyperparameters for creating an RNN architecture for DCE‐MRI studies.

Bi-Level Interturn Short-Circuit Fault Monitoring for Wind Turbine Generators With Benchmark Dataset Development

Abstract Flourished wind energy market pushes the latest wind turbines (WTs) to further and harsher inland and offshore environment. Increased operation and maintenance cost calls for more reliable and cost effective condition monitoring systems. In this article, a bi-level condition monitoring framework for interturn short-circuit faults (ITSCFs) in WT generators is proposed. A benchmark dataset, consisting of 75 ITSCF scenarios and generator current signals of a specific WT, has been created and made publicly available on Zenodo. The data are simulated at a rate of 4 kHz. Based on the time and frequency features extracted from data processing, machine learning-based severity estimation and faulty phase identification modules can provide valuable diagnostic information for wind farm operators. Specifically, the performance of long short-term memory (LSTM) networks, gated recurrent unit (GRU) networks, and convolutional neural networks (CNNs) are analyzed and compared for severity estimation and faulty phase identification. For test-bed experimental reference, various numbers of scenarios for training the models are analyzed. Numerical experiments demonstrate the computational efficiency and robust denoising capability of the CNN algorithm. The GRU network, however, achieves the highest accuracy. The overall system performance improves significantly, from 87.76% with 16 training scenarios to 99.95% with 52 training scenarios, when tested on a set containing all 76 scenarios from an unforeseen period.

Evaluating Performances of LSTM, SVM, GPR, and RF for Drought Prediction in Norway: A Wavelet Decomposition Approach on Regional Forecasting

A serious natural disaster that poses a threat to people and their living spaces is drought, which is difficult to notice at first and can quickly spread to wide areas through subtle progression. Numerous methods are being explored to identify, prevent, and mitigate drought, and distinct metrics have been developed. In order to contribute to the research on measures to be taken against drought, the Standard Precipitation Evaporation Index (SPEI), one of the drought indices that has been developed and accepted in recent years and includes a more comprehensive drought definition, was chosen in this study. Machine learning and deep learning algorithms, including support vector machine (SVM), random forest (RF), long short-term memory (LSTM), and Gaussian process regression (GPR), were used to model the droughts in six regions of Norway: Bodø, Karasjok, Oslo, Tromsø, Trondheim, and Vadsø. Four distinct model architectures were employed for this goal, and as a novel approach, the models’ output was enhanced by using discrete wavelet decomposition/transformation (WT). The model outputs were evaluated using the correlation coefficient (r), Nash–Sutcliffe efficiency (NSE), and root mean square error (RMSE) as performance evaluation criteria. When the findings were analyzed, the GPR model (W-GPR), which was acquired after WT, typically produced the best results. Furthermore, it was discovered that, out of all the recognized models, M04 had the most effective model structure. Consequently, the most successful outcomes were obtained with W-SVM-M04 for Bodø and W-GPR-M04 for Karasjok, Oslo, Tromsø, Trondheim, and Vadsø. Furthermore, W-GPR-M04 in the Oslo region had the best results across all regions (r: 0.9983, NSE: 0.9966 and RMSE:0.0539).

OP03 Redesigning the patient management pathway through prediction of flares in axial spondyloarthritis using a machine learning approach to improve care

Abstract Background and aims Treatment of axial spondyloarthritis (axSpA) involves the management of flares which can be unpredictable. ASAS developed a data-driven definition of flares/disease worsening as an increase in ASDAS-CRP of at least 0.9 points, for use in axSpA clinical trials (1). With the current pressures on clinic capacity, there is a challenge to see flare patients in a timely manner. Artificial intelligence (AI) and natural language processing (NLP) methods provide algorithms for learning systems to recognise disease-related terms and classify clinical phenotypes using large datasets, which may support early identification of patients with a risk of having a flare. In previous work, we presented an approach using NLP to identify patients who may have rheumatic disease with predictors to discriminate patients with diagnosed inflammatory arthritis from controls with non-inflammatory arthritis. With the same aim, we extended the panel of predictors by combining clinical and laboratory data to predict flares in axSpA. Methods 282 axSpA patients seen between January 2016 to May 2024 were included in the study. There are 100 flare and 182 non-flare patients studied. We utilised patient blood test results, demographic information, electronic patient-reported outcomes scores, comorbidity, weight and height. We developed a time series dataset and applied Long Short-Term Memory (LSTM) networks and neural networks (NN) to forecast the risk of flare before their future clinics. Results We conducted predictive modelling to forecast flare and non-flare events using data collected prior to patients upcoming clinic appointments. The dataset was split into training and testing sets based on observations taken 2 months before the clinic visit. Our LSTM model achieved 90.25% accuracy, 0.89 sensitivity, 0.96 specificity, 0.93 precision and 0.98 Area Under the Receiver Operating Characteristic Curve (AUROC) predicting flare and non-flare events 2 months before the clinics. The results are summarized in Table 1. OP03 Table 1.Performance of Long Short-Term Memory (LSTM) and neural networks (NN) on data from 2 months before the clinic visit.AVG ACCURACYAVG SENSITIVITYAVG SPECIFICITYAVG PRECISIONAVG AUROCLSTM (2 months before the next clinics)90.25%0.890.960.930.98NEURAL NETWORK (2 months before the next clinics)71.90%0.910.010.610.5 Conclusions The presented combination of clinical and laboratory data shows promising results for the prediction of flare in axSpA and may in the future assist clinicians in seeing the right patient in the right clinic. A study is planned not only to confirm the results but also to refine the machine learning (ML) models developed and specify disease attributes. In the service redesign, patients identified as at risk of flares will be booked and seen in routine clinics at the right time to prevent flares. This process in the long-term will reduce the reliance on flare and urgent clinics.

Enhanced Fault Detection in Satellite Attitude Control Systems Using LSTM-Based Deep Learning and Redundant Reaction Wheels

Reliable fault detection in satellite attitude control systems stands as a critical aspect of ensuring the safety and success of space missions. Central to these systems, reaction wheels (RWs), despite being the most frequently used actuators, present a vulnerability given their susceptibility to faults—a factor with the potential to precipitate catastrophic failures such as total satellite loss. In light of this, we introduce a fault detection methodology grounded in deep learning techniques specifically designed for satellite attitude control systems. Our proposed method utilizes a Long Short-Term Memory (LSTM) model adept at learning temporal patterns inherent to both healthy and faulty system behaviors. Incorporated into our model is a torque allocation algorithm designed to circumvent specific velocities known to induce torque disturbances, a factor known to influence LSTM performance adversely. To bolster the robustness of our fault detection technique, we also incorporated denoising autoencoders within the LSTM framework, thereby enabling the model to identify temporal patterns in healthy and faulty system behavior, even amidst the noise. The method was evaluated using cross-validation on simulated satellite data comprising 1000 time series samples and across different fault scenarios, such as stiction and resonance at varying intensities (90%, 50%, and 30%). The results confirm achieving performance metrics such as Mean Squared Error for accurate fault identification. This research underscores a stride in the evolution of fault detection and control strategies for satellite attitude control systems, holding promise to boost the reliability and efficiency of future space missions.

Analysis of athletes’ technical action based on deep learning

In order to raise athletes’ technical proficiency and overall performance, this paper uses deep learning technology to precisely assess table tennis technical activities. The paper builds a hybrid neural network model for technical action analysis of table tennis players, based on Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) in deep learning. First, CNN extracts the spatial characteristics from the video frames. After that, an LSTM processes these data in a time series to accurately recognize and classify the movements of athletes. Lastly, through trials, the model is trained and evaluated using the Table Tennis Tracking Network (TTNET) dataset. The experimental findings demonstrate that: (1) In table tennis technical action analysis, the suggested deep learning model performs better than other comparative models. The efficacy of combining convolution with sequence learning is fully demonstrated by the CNN-LSTM hybrid model, which performs best in all indicators when compared to the CNN, LSTM, Multi-Objective Function (MOF), and Convolutional Neural Network—Long Short-Term Memory (CNN-LSTM) combined models. Its accuracy rate, precision rate, recall rate, and F1 score are 0.923, 0.918, 0.925, and 0.921, respectively. In contrast, the performances of LSTM and CNN are also excellent, but the performance of MOF model is relatively low; (2) In the classification of technical actions, the model has the highest classification accuracy of service, reaching 0.930, and the classification accuracy of other technical actions such as forehand stroke, backhand stroke and spike is also above 0.9, and the time sequence consistency index also maintains a high level, indicating that the model can effectively identify and analyze table tennis technical actions. In addition, the performance evaluation of real-time feedback shows that the model can achieve low processing time and feedback delay in video data processing with different lengths, which ensures the real-time and reliability in practical applications. These results show that the proposed model can not only provide accurate technical action recognition, but also provide timely and effective feedback in practical application, which has high practical value. The results of this paper prove the potential of deep learning technology in the analysis of athletes’ technical actions, and provide scientific basis and effective tools for the technical training and optimization of table tennis players.

Investigating the effect of non-resonant background variation on the CARS data analysis of bacteria samples and classification using machine learning

Non-resonant background (NRB) plays a significant role in coherent anti-Stokes Raman scattering (CARS) spectroscopic applications. All the recent works primarily focused on removing the NRB using different deep learning methods, and only one study explored the effect of NRB. Hence, in this work, we systematically investigated the impact of NRB variation on Raman signal retrieval. The NRB is simulated as a linear function with different strengths relative to the resonant Raman signal, and the variance also changes for each NRB strength. The resonant part of nonlinear susceptibility is extracted from real experimental Raman data; hence, the simulated CARS data better approximate the experimental CARS spectra. Then, the corresponding Raman signal is retrieved by four different methods: maximum entropy method (MEM), Kramers-Kronig (KK), convolutional neural network (CNN), and long short-term memory (LSTM) network. Pearson correlation measurements and principal component analysis combined with linear discriminant analysis modeling revealed that MEM and KK methods have an edge over LSTM and CNN for higher NRB strengths. It is also demonstrated that normalizing the input data favors LSTM and CNN predictions. In contrast, background removal from the predictions significantly influenced Pearson correlation but not the classification accuracies for MEM and KK. Further, the LSTM performance is found to be limited and can only be applied for low NRB strengths. This comprehensive study has the potential to impact the CARS spectroscopy and microscopy applications in different areas.

NH4 Modelling with ARIMA and LSTM

AI-Mining is a prototype designed to detect various environmental gases, including CO2, NH3, NH4, and hydrogen, alongside temperature, pressure, and humidity. This study emphasizes the importance of modeling NH4 time series data due to its critical role in environmental and health monitoring. Accurate NH4 predictions facilitate early pollution detection and timely greenhouse gas control interventions. The study investigates the effectiveness of AI-Mining in detecting and predicting gas levels, focusing on data collection and analysis. Initial data analysis employed the Autoregressive Moving Average (ARIMA) model, specifically ARIMA (1,1,1), described by the equation yt = 0.0311 - 0.0750yt-1 + 0.3842εt-1. Despite its use, ARIMA's Root Mean Square Error (RMSE) performance was found lacking compared to more advanced methods. Given the classification of the obtained data as big data and time series, the Long Short-Term Memory (LSTM) method was also applied. The LSTM model initially used two layers with tanh and relu activation functions, and its performance was further explored by adding a third layer and varying the number of neurons (64, 128, and 256). The Adam optimizer was consistently used across all LSTM variations. Results indicated that increasing layers and neurons did not significantly impact LSTM's performance, with RMSE values around 0.023. However, LSTM consistently outperformed ARIMA in prediction accuracy, highlighting its robustness and reliability. Consequently, the study recommends using LSTM for predicting other recorded data in AI-Mining, underscoring its superiority in handling complex environmental datasets.

An open innovative inventory management based demand forecasting approach for the steel industry

This research focuses on accurately forecasting steel wire mesh demand to ensure timely order fulfillment. Various univariate time series forecasting methods were employed, including Prophet, Support Vector Regression (SVR), 1-Dimensional Convolutional Neural Network (1D-CNN), Recurrent Neural Network (RNN), Gated Recurrent Unit (GRU), and Long Short-Term Memory (LSTM). These RNN-based models were implemented to enhance the dynamics of open innovation. The study also employed the split test method to assess the impact of data size on model performance. Following a comprehensive comparative analysis, LSTM outperformed other models at 80:20 data split ratio, achieving the lowest RMSE of 51,473 and MAPE of 1.43. The robust performance of LSTM underscores its ability to expertly predict complex demand patterns. Moreover, forecast accuracy was enhanced using ensemble models, with RNN-LSTM delivering the best results (RMSE of 45,931 and MAPE of 1.20) at the same data split ratio. This accurate forecasting supports steel wire mesh companies in making informed decisions regarding inventory management. The study additionally demonstrated the integration of forecasting results into inventory strategies to improve decision-making for cost optimization.

Comparison of Machine Learning-Based Predictive Models of the Nutrient Loads Delivered from the Mississippi/Atchafalaya River Basin to the Gulf of Mexico

Predicting nutrient loads is essential to understanding and managing one of the environmental issues faced by the northern Gulf of Mexico hypoxic zone, which poses a severe threat to the Gulf’s healthy ecosystem and economy. The development of hypoxia in the Gulf of Mexico is strongly associated with the eutrophication process initiated by excessive nutrient loads. Due to the complexities in the excessive nutrient loads to the Gulf of Mexico, it is challenging to understand and predict the underlying temporal variation of nutrient loads. The study was aimed at identifying an optimal predictive machine learning model to capture and predict nonlinear behavior of the nutrient loads delivered from the Mississippi/Atchafalaya River Basin (MARB) to the Gulf of Mexico. For this purpose, monthly nutrient loads (N and P) in tons were collected from US Geological Survey (USGS) monitoring station 07373420 from 1980 to 2020. Machine learning models—including autoregressive integrated moving average (ARIMA), gaussian process regression (GPR), single-layer multilayer perceptron (MLP), and a long short-term memory (LSTM) with the single hidden layer—were developed to predict the monthly nutrient loads, and model performances were evaluated by standard assessment metrics—Root Mean Square Error (RMSE) and Correlation Coefficient (R). The residuals of predictive models were examined by the Durbin–Watson statistic. The results showed that MLP and LSTM persistently achieved better accuracy in predicting monthly TN and TP loads compared to GPR and ARIMA. In addition, GPR models achieved slightly better test RMSE score than ARIMA models while their correlation coefficients are much lower than ARIMA models. Moreover, MLP performed slightly better than LSTM in predicting monthly TP loads while LSTM slightly outperformed for TN loads. Furthermore, it was found that the optimizer and number of inputs didn’t show effects on the LSTM performance while they exhibited impacts on MLP outcomes. This study explores the capability of machine learning models to accurately predict nonlinearly fluctuating nutrient loads delivered to the Gulf of Mexico. Further efforts focus on improving the accuracy of forecasting using hybrid models which combine several machine learning models with superior predictive performance for nutrient fluxes throughout the MARB.

Explainable AI and optimized solar power generation forecasting model based on environmental conditions.

This paper proposes a model called X-LSTM-EO, which integrates explainable artificial intelligence (XAI), long short-term memory (LSTM), and equilibrium optimizer (EO) to reliably forecast solar power generation. The LSTM component forecasts power generation rates based on environmental conditions, while the EO component optimizes the LSTM model's hyper-parameters through training. The XAI-based Local Interpretable and Model-independent Explanation (LIME) is adapted to identify the critical factors that influence the accuracy of the power generation forecasts model in smart solar systems. The effectiveness of the proposed X-LSTM-EO model is evaluated through the use of five metrics; R-squared (R2), root mean square error (RMSE), coefficient of variation (COV), mean absolute error (MAE), and efficiency coefficient (EC). The proposed model gains values 0.99, 0.46, 0.35, 0.229, and 0.95, for R2, RMSE, COV, MAE, and EC respectively. The results of this paper improve the performance of the original model's conventional LSTM, where the improvement rate is; 148%, 21%, 27%, 20%, 134% for R2, RMSE, COV, MAE, and EC respectively. The performance of LSTM is compared with other machine learning algorithm such as Decision tree (DT), Linear regression (LR) and Gradient Boosting. It was shown that the LSTM model worked better than DT and LR when the results were compared. Additionally, the PSO optimizer was employed instead of the EO optimizer to validate the outcomes, which further demonstrated the efficacy of the EO optimizer. The experimental results and simulations demonstrate that the proposed model can accurately estimate PV power generation in response to abrupt changes in power generation patterns. Moreover, the proposed model might assist in optimizing the operations of photovoltaic power units. The proposed model is implemented utilizing TensorFlow and Keras within the Google Collab environment.

Modeling continental US stream water quality using long-short term memory and weighted regressions on time, discharge, and season

The temporal dynamics of solute export from catchments are challenging to quantify and model due to confounding hydrological and biogeochemical processes and sparse measurements. Conventionally, the concentration-discharge relationship (C-Q) and statistical approaches to describe it, such as the Weighted Regressions on Time, Discharge and Seasons (WRTDS), have been widely used. Recently, deep learning (DL) approaches, especially Long-Short-Term-Memory (LSTM) models, have shown predictive capability for discharge, temperature, and dissolved oxygen. However, it is not clear if such advances can be expanded to water quality variables driven by complex subsurface biogeochemical processes. This work evaluates the performance of LSTM and WRTDS for 20 water quality variables across ~500 catchments in the continental US. We find that LSTM does not markedly outperform WRTDS in our dataset, potentially limited by the current measurement capabilities of water quality across CONUS. Both models present similar performance patterns across water quality variables, with the LSTM displaying better performance for nutrients compared to weathering-derived solutes. Additionally, the LSTM does not benefit from flexibility in the inputs. For example, incorporation of climate data that constrains streamflow generation, does not significantly improve the LSTM performance. We also find that data availability is not a straightforward predictor of LSTM model performance, although higher availability tends to stabilize performance. To fully assess the potential of the LSTM model, it may be necessary to use a higher frequency dataset across the CONUS, which does not exist today. To evaluate the dynamics of C-Q patterns relative to model performance, we introduce a “simplicity index” considering both the seasonality in the concentration pattern and the linearity in the C-Q relationship, or the C-Q-t pattern. The simplicity index is strongly correlated with model performance and differentiates the underlying controls on water quality dynamics. Further DL experiments and model-intercomparison highlight the strengths and deficiencies of existing frameworks, pointing to the need for further hydrogeochemical theories that are amenable to complex basins and solutes.

An integrated framework for driving risk evaluation that combines lane-changing detection and an attention-based prediction model

Objective In recent years, the increase in traffic accidents has emerged as a significant social issue that poses a serious threat to public safety. The objective of this study is to predict risky driving scenarios to improve road safety. Methods On the basis of data collected from naturalistic driving real-vehicle experiments, a comprehensive framework for identifying and analyzing risky driving scenarios, which combines an integrated lane-changing detection model and an attention-based long short-term memory (LSTM) prediction model, is proposed. The performance of the 4 machine learning methods on the CULane data set is compared in terms of model running time and running speed as evaluation metrics, and the ultrafast network with the best performance is selected as the method for lane line detection. We compared the performance of LSTM and attention-based LSTM on the basis of the prediction accuracy, recall, precision, and F1 value and selected the better model (attention-based LSTM) for risky scenario prediction. Furthermore, Shapley additive explanation analysis (SHAP) is used to understand and interpret the prediction results of the model. Results In terms of algorithm efficiency, the running time of the ultrafast lane detection network only requires 4.1 ms, and the average detection speed reaches 131 fps. For prediction performance, the accuracy rate of attention-based LSTM reaches 96%, the precision rate is 98%, the recall rate is 96%, and the F1 value is 97%. Conclusions The improved attention-based LSTM model is significantly better than the LSTM model in terms of convergence speed and prediction accuracy and can accurately identify risky scenarios that occur during driving. The importance of factors varies by risky scenario. The characteristics of the yaw rate, speed stability, vehicle speed, acceleration, and lane change significantly influence the driving risk, among which lane change has the greatest impact. This study can provide real-time risky scenario prediction, warnings, and scientific decision guidance for drivers.

Deep learning the Hurst parameter of linear fractional processes and assessing its reliability

AbstractThis research explores the reliability of deep learning, specifically Long Short‐Term Memory (LSTM) networks, for estimating the Hurst parameter in fractional stochastic processes. The study focuses on three types of processes: fractional Brownian motion (fBm), fractional Ornstein–Uhlenbeck (fOU) process, and linear fractional stable motions (lfsm). The work involves a fast generation of extensive datasets for fBm and fOU to train the LSTM network on a large volume of data in a feasible time. The study analyses the accuracy of the LSTM network's Hurst parameter estimation regarding various performance measures like root mean squared error (RMSE), mean absolute error (MAE), mean relative error (MRE), and quantiles of the absolute and relative errors. It finds that LSTM outperforms the traditional statistical methods in the case of fBm and fOU processes; however, it has limited accuracy on lfsm processes. The research also delves into the implications of training length and valuation sequence length on the LSTM's performance. The methodology is applied to estimating the Hurst parameter in li‐ion battery degradation data and obtaining confidence bounds for the estimation. The study concludes that while deep learning methods show promise in parameter estimation of fractional processes, their effectiveness is contingent on the process type and the quality of training data.

Runoff Simulation in Data-Scarce Alpine Regions: Comparative Analysis Based on LSTM and Physically Based Models

Runoff simulation is essential for effective water resource management and plays a pivotal role in hydrological forecasting. Improving the quality of runoff simulation and forecasting continues to be a highly relevant research area. The complexity of the terrain and the scarcity of long-term runoff observation data have significantly limited the application of Physically Based Models (PBMs) in the Qinghai–Tibet Plateau (QTP). Recently, the Long Short-Term Memory (LSTM) network has been found to be effective in learning the dynamic hydrological characteristics of watersheds and outperforming some traditional PBMs in runoff simulation. However, the extent to which the LSTM works in data-scarce alpine regions remains unclear. This study aims to evaluate the applicability of LSTM in alpine basins in QTP, as well as the simulation performance of transfer-based LSTM (T-LSTM) in data-scarce alpine regions. The Lhasa River Basin (LRB) and Nyang River Basin (NRB) were the study areas, and the performance of the LSTM model was compared to that of PBMs by relying solely on the meteorological inputs. The results show that the average values of Nash–Sutcliffe efficiency (NSE), Kling–Gupta efficiency (KGE), and Relative Bias (RBias) for B-LSTM were 0.80, 0.85, and 4.21%, respectively, while the corresponding values for G-LSTM were 0.81, 0.84, and 3.19%. In comparison to a PBM- the Block-Wise use of TOPMEDEL (BTOP), LSTM has an average enhancement of 0.23, 0.36, and −18.36%, respectively. In both basins, LSTM significantly outperforms the BTOP model. Furthermore, the transfer learning-based LSTM model (T-LSTM) at the multi-watershed scale demonstrates that, when the input data are somewhat representative, even if the amount of data are limited, T-LSTM can obtain more accurate results than hydrological models specifically calibrated for individual watersheds. This result indicates that LSTM can effectively improve the runoff simulation performance in alpine regions and can be applied to runoff simulation in data-scarce regions.

Generalized Performance of LSTM in Time-Series Forecasting

ABSTRACT Optimizing the time-series forecasting performance is a multi-objective problem which enables the comparison of general applicability of methods across multiple use cases such as finance and demographics. Libra, a time-series forecasting framework which shifts the problem of optimization from minimizing single to multiple evaluation measures and use cases, is used as a benchmark to evaluate the performance of the Long Short-Term Memory (LSTM) neural network. LSTMs with parameter tuning have been shown to perform well with time-series forecasting. This paper applies LSTMs (mostly with standard parameters and variations of some of them) to the Libra framework and concludes that due to data characteristic variance and without increased hardware and time constraints LSTMs do not outperform the median measures of Libra.

Comparative Study of Conventional Machine Learning versus Deep Learning-Based Approaches for Tool Condition Assessments in Milling Processes

This evaluation of deep learning and traditional machine learning methods for tool state recognition in milling processes aims to automate furniture manufacturing. It compares the performance of long short-term memory (LSTM) networks, support vector machines (SVMs), and boosting ensemble decision trees, utilizing sensor data from a CNC machining center. These methods focus on the challenges and importance of feature selection, data preprocessing, and the application of tailored machine learning models to specific industrial tasks. Results show that SVM, with an accuracy of 96%, excels in handling high-dimensional data and robust feature extraction. In contrast, LSTM, which is appropriate for sequential data, is constrained by limited training data and the absence of pre-trained networks. Boosting ensemble decision trees also demonstrate efficacy in reducing model bias and variance. Conclusively, selecting an optimal machine learning strategy is crucial, depending on task complexity and data characteristics, highlighting the need for further research into domain-specific models to improve performance in industrial settings.

Traffic crash prediction in Kano, Nigeria: multivariate long short-term memory method

Accurate traffic crash prediction is crucial for implementing effective road safety measures. In this study, the performance of long short-term memory (LSTM) and multivariate LSTM (MLSTM) models in forecasting total crash count data in Kano state, Nigeria were compared. Human and vehicle factors, including speed violations, tyre bursts, brake failures, sign/light violations and phone use while driving, were incorporated as covariates in the MLSTM model. An autoregressive integrated moving average with exogenous variables (Arimax) model was used to investigate the effects of the covariates. The MLSTM model outperformed both the basic LSTM model and individual covariate models, emphasising the synergistic effect of considering a broad range of factors. The Arimax model results revealed that speed violation is significantly positively correlated with total crashes; the other covariates showed positive correlations but did not reach the statistical significance. The findings underscore the importance of a multivariate approach in enhancing traffic crash prediction. The MLSTM model's superior performance highlights the value of considering a comprehensive range of factors that influence crash occurrence to achieve more accurate predictions. Practical applications of these models could involve leveraging them for proactive traffic safety measures, including increased enforcement of traffic rules, improvements to road infrastructure and targeted driver education and campaigns.

Emotion Recognition in Reddit Comments Using Recurrent Neural Networks

Background: Reddit comments are a valuable source of natural language data where emotion plays a key role in human communication. However, emotion recognition is a difficult task that requires understanding the context and sentiment of the texts. In this paper, we aim to compare the effectiveness of four recurrent neural network (RNN) models for classifying the emotions of Reddit comments. Methods: We use a small dataset of 4,922 comments labeled with four emotions: approval, disapproval, love, and annoyance. We also use pre-trained Glove.840B.300d embeddings as the input representation for all models. The models we compare are SimpleRNN, Long ShortTerm Memory (LSTM), bidirectional LSTM, and Gated Recurrent Unit (GRU). We experiment with different text preprocessing steps, such as removing stopwords and applying stemming, removing negation from stopwords, and the effect of setting the embedding layer as trainable on the models. Results: We find that GRU outperforms all other models, achieving an accuracy of 74%. Bidirectional LSTM and LSTM are close behind, while SimpleRNN performs the worst. We observe that the low accuracy is likely due to the presence of sarcasm, irony, and complexity in the texts. We also notice that setting the embedding layer as trainable improves the performance of LSTM but increases the computational cost and training time significantly. We analyze some examples of misclassified texts by GRU and identify the challenges and limitations of the dataset and the models Conclusion: In our study GRU was found to be the best model for emotion classification of Reddit comments among the four RNN models we compared. We also discuss some future directions for research to improve the emotion recognition task on Reddit comments. Furthermore, we provide an extensive discussion of the applications and methods behind each technique in the context of the paper.