Performance Of Long Short-term Memory Research Articles

Runoff Simulation in Data-Scarce Alpine Regions: Comparative Analysis Based on LSTM and Physically Based Models

Runoff simulation is essential for effective water resource management and plays a pivotal role in hydrological forecasting. Improving the quality of runoff simulation and forecasting continues to be a highly relevant research area. The complexity of the terrain and the scarcity of long-term runoff observation data have significantly limited the application of Physically Based Models (PBMs) in the Qinghai–Tibet Plateau (QTP). Recently, the Long Short-Term Memory (LSTM) network has been found to be effective in learning the dynamic hydrological characteristics of watersheds and outperforming some traditional PBMs in runoff simulation. However, the extent to which the LSTM works in data-scarce alpine regions remains unclear. This study aims to evaluate the applicability of LSTM in alpine basins in QTP, as well as the simulation performance of transfer-based LSTM (T-LSTM) in data-scarce alpine regions. The Lhasa River Basin (LRB) and Nyang River Basin (NRB) were the study areas, and the performance of the LSTM model was compared to that of PBMs by relying solely on the meteorological inputs. The results show that the average values of Nash–Sutcliffe efficiency (NSE), Kling–Gupta efficiency (KGE), and Relative Bias (RBias) for B-LSTM were 0.80, 0.85, and 4.21%, respectively, while the corresponding values for G-LSTM were 0.81, 0.84, and 3.19%. In comparison to a PBM- the Block-Wise use of TOPMEDEL (BTOP), LSTM has an average enhancement of 0.23, 0.36, and −18.36%, respectively. In both basins, LSTM significantly outperforms the BTOP model. Furthermore, the transfer learning-based LSTM model (T-LSTM) at the multi-watershed scale demonstrates that, when the input data are somewhat representative, even if the amount of data are limited, T-LSTM can obtain more accurate results than hydrological models specifically calibrated for individual watersheds. This result indicates that LSTM can effectively improve the runoff simulation performance in alpine regions and can be applied to runoff simulation in data-scarce regions.

Generalized Performance of LSTM in Time-Series Forecasting

ABSTRACT Optimizing the time-series forecasting performance is a multi-objective problem which enables the comparison of general applicability of methods across multiple use cases such as finance and demographics. Libra, a time-series forecasting framework which shifts the problem of optimization from minimizing single to multiple evaluation measures and use cases, is used as a benchmark to evaluate the performance of the Long Short-Term Memory (LSTM) neural network. LSTMs with parameter tuning have been shown to perform well with time-series forecasting. This paper applies LSTMs (mostly with standard parameters and variations of some of them) to the Libra framework and concludes that due to data characteristic variance and without increased hardware and time constraints LSTMs do not outperform the median measures of Libra.

Comparative Study of Conventional Machine Learning versus Deep Learning-Based Approaches for Tool Condition Assessments in Milling Processes

This evaluation of deep learning and traditional machine learning methods for tool state recognition in milling processes aims to automate furniture manufacturing. It compares the performance of long short-term memory (LSTM) networks, support vector machines (SVMs), and boosting ensemble decision trees, utilizing sensor data from a CNC machining center. These methods focus on the challenges and importance of feature selection, data preprocessing, and the application of tailored machine learning models to specific industrial tasks. Results show that SVM, with an accuracy of 96%, excels in handling high-dimensional data and robust feature extraction. In contrast, LSTM, which is appropriate for sequential data, is constrained by limited training data and the absence of pre-trained networks. Boosting ensemble decision trees also demonstrate efficacy in reducing model bias and variance. Conclusively, selecting an optimal machine learning strategy is crucial, depending on task complexity and data characteristics, highlighting the need for further research into domain-specific models to improve performance in industrial settings.

Traffic crash prediction in Kano, Nigeria: multivariate long short-term memory method

Accurate traffic crash prediction is crucial for implementing effective road safety measures. In this study, the performance of long short-term memory (LSTM) and multivariate LSTM (MLSTM) models in forecasting total crash count data in Kano state, Nigeria were compared. Human and vehicle factors, including speed violations, tyre bursts, brake failures, sign/light violations and phone use while driving, were incorporated as covariates in the MLSTM model. An autoregressive integrated moving average with exogenous variables (Arimax) model was used to investigate the effects of the covariates. The MLSTM model outperformed both the basic LSTM model and individual covariate models, emphasising the synergistic effect of considering a broad range of factors. The Arimax model results revealed that speed violation is significantly positively correlated with total crashes; the other covariates showed positive correlations but did not reach the statistical significance. The findings underscore the importance of a multivariate approach in enhancing traffic crash prediction. The MLSTM model's superior performance highlights the value of considering a comprehensive range of factors that influence crash occurrence to achieve more accurate predictions. Practical applications of these models could involve leveraging them for proactive traffic safety measures, including increased enforcement of traffic rules, improvements to road infrastructure and targeted driver education and campaigns.

Emotion Recognition in Reddit Comments Using Recurrent Neural Networks

Background: Reddit comments are a valuable source of natural language data where emotion plays a key role in human communication. However, emotion recognition is a difficult task that requires understanding the context and sentiment of the texts. In this paper, we aim to compare the effectiveness of four recurrent neural network (RNN) models for classifying the emotions of Reddit comments. Methods: We use a small dataset of 4,922 comments labeled with four emotions: approval, disapproval, love, and annoyance. We also use pre-trained Glove.840B.300d embeddings as the input representation for all models. The models we compare are SimpleRNN, Long ShortTerm Memory (LSTM), bidirectional LSTM, and Gated Recurrent Unit (GRU). We experiment with different text preprocessing steps, such as removing stopwords and applying stemming, removing negation from stopwords, and the effect of setting the embedding layer as trainable on the models. Results: We find that GRU outperforms all other models, achieving an accuracy of 74%. Bidirectional LSTM and LSTM are close behind, while SimpleRNN performs the worst. We observe that the low accuracy is likely due to the presence of sarcasm, irony, and complexity in the texts. We also notice that setting the embedding layer as trainable improves the performance of LSTM but increases the computational cost and training time significantly. We analyze some examples of misclassified texts by GRU and identify the challenges and limitations of the dataset and the models Conclusion: In our study GRU was found to be the best model for emotion classification of Reddit comments among the four RNN models we compared. We also discuss some future directions for research to improve the emotion recognition task on Reddit comments. Furthermore, we provide an extensive discussion of the applications and methods behind each technique in the context of the paper.

Enhancing Arabic Dialect Detection on Social Media: A Hybrid Model with an Attention Mechanism

Recently, the widespread use of social media and easy access to the Internet have brought about a significant transformation in the type of textual data available on the Web. This change is particularly evident in Arabic language usage, as the growing number of users from diverse domains has led to a considerable influx of Arabic text in various dialects, each characterized by differences in morphology, syntax, vocabulary, and pronunciation. Consequently, researchers in language recognition and natural language processing have become increasingly interested in identifying Arabic dialects. Numerous methods have been proposed to recognize this informal data, owing to its crucial implications for several applications, such as sentiment analysis, topic modeling, text summarization, and machine translation. However, Arabic dialect identification is a significant challenge due to the vast diversity of the Arabic language in its dialects. This study introduces a novel hybrid machine and deep learning model, incorporating an attention mechanism for detecting and classifying Arabic dialects. Several experiments were conducted using a novel dataset that collected information from user-generated comments from Twitter of Arabic dialects, namely, Egyptian, Gulf, Jordanian, and Yemeni, to evaluate the effectiveness of the proposed model. The dataset comprises 34,905 rows extracted from Twitter, representing an unbalanced data distribution. The data annotation was performed by native speakers proficient in each dialect. The results demonstrate that the proposed model outperforms the performance of long short-term memory, bidirectional long short-term memory, and logistic regression models in dialect classification using different word representations as follows: term frequency-inverse document frequency, Word2Vec, and global vector for word representation.

Testing the Performance of LSTM and ARIMA Models for In-Season Forecasting of Canopy Cover (CC) in Cotton Crops

Cotton (Gossypium spp.), a crucial cash crop in the United States, requires the constant monitoring of growth parameters for informed decision-making. Recently, forecasting models have gained prominence for predicting canopy indicators, aiding in-season planning and management decisions to optimize cotton production. This study employed unmanned aerial system (UAS) technology to collect canopy cover (CC) data from a 40-hectare cotton field in Driscoll, Texas, in 2020 and 2021. Long short-term memory (LSTM) models, trained using 2020 data, were subsequently applied to forecast the CC values for 2021. These models were compared with real-time auto-regressive integrated moving average (ARIMA) models to assess their effectiveness in predicting the CC values up to 14 days in advance, starting from the 28th day after crop emergence. The results showed that multiple-input multi-step output LSTM models achieved higher accuracy in predicting the in-season CC values during the early growth stages (up to the 56th day), with an average testing RMSE of 3.86, significantly lower than other single-input LSTM models. Conversely, when sufficient testing data are available, single-input stacked-LSTM models demonstrated precision in CC predictions for later stages, achieving an average RMSE of 3.06. These findings highlight the potential of LSTM models for in-season CC forecasting, facilitating effective management strategies in cotton production.

Research on Electricity Load Forecasting Based on LSTM Networks and BP Networks

The aim of this study is to compare the performance of Long Short-Term Memory (LSTM) network with Back Propagation Neural Network (BP network) for electricity load forecasting. Two prediction methods are used in the study: the first one uses a sliding window approach to predict the load of the next hour using the electricity load data of the past 24 hours and external factors such as temperature and humidity; the second one uses the data of the same time point of the past 24 days to predict the load of the corresponding time point of the next day. The results show that in the first method, the BP network outperforms the LSTM in prediction; while in the second method, both networks perform poorly, but the LSTM network outperforms the BP network in prediction. This finding provides new insights into power load forecasting, which is important for power system efficiency and reliability improvement.

Temporal Fusion Transformers for streamflow Prediction: Value of combining attention with recurrence

Over the past few decades, the hydrology community has witnessed notable advancements in streamflow prediction, particularly with the introduction of cutting-edge machine-learning algorithms. Recurrent neural networks, especially Long Short-Term Memory (LSTM) networks, have become popular due to their capacity to create precise forecasts and realistically mimic the system dynamics. Attention-based models, such as Transformers, can learn from the entire data sequence concurrently, a feature that LSTM does not have. This work tests the hypothesis that combining recurrence with attention can improve streamflow prediction. We set up the Temporal Fusion Transformer (TFT) architecture, a model that combines both of these aspects and has never been applied in hydrology before. We compare the performance of LSTM, Transformers, and TFT over 2,610 globally distributed catchments from the recently available Caravan dataset. Our results demonstrate that TFT indeed exceeds the performance benchmark set by the LSTM and Transformers for streamflow prediction, when models are trained individually for each basin. Additionally, being an explainable AI method, TFT helps in gaining insights into the streamflow generation processes.

Toward better atmospheric polycyclic aromatic hydrocarbons pollution control in the Northern Hemisphere: Process analysis based on interpretable deep learning models

The accurate prediction of atmospheric polycyclic aromatic hydrocarbons (PAHs) concentrations in the Arctic is crucial in guiding the development of pollution control measures for atmospheric PAHs. However, the complex formation process and strong time-dependence of PAHs in the Arctic increase the difficulty of prediction. Herein, deep learning (DL) models were developed to predict monthly air concentrations of 16 priority PAHs at Alert monitoring station by incorporating the inherent periodicity of PAH concentrations and ancillary data including PAH emissions, meteorological information, fire emissions, and sea ice area. The prediction performance of long short-term memory (LSTM), gated recurrent unit (GRU) and bidirectional LSTM (Bi LSTM) algorithms were compared by multiple model evaluation metrics. The results showed that Bi LSTM models outperformed LSTM and GRU in predicting PAHs concentrations because it exhibited higher accuracy in terms of average R2ext (0.09) and errors (average MAPEext = 1.45). The DL interpretation based on SHAP values suggested that the main drivers of PAHs concentrations were PAHs emissions and sea ice area, with a more prominent contribution from meteorological conditions. The interpretable deep learning approach provides a potential shortcut for predicting time-delayed PAHs concentrations in the Arctic, promoting the management and control of global PAHs pollution.

Unveiling Emotional Stress: Analyzing EEG Signals With Deep Learning Insights

This study focuses on a comprehensive exploration of EEG-based emotion recognition, with a particular focus on leveraging recurrent neural network (RNN) architectures, including long short-term memory (LSTM) and gated recurrent unit (GRU). The investigation is rooted in a publicly available EEG Brainwave Dataset, meticulously curated to capture human responses to emotional stimuli. The dataset features signals obtained from frontal and temporal brain lobes, categorized into distinct emotional states encompassing positive, neutral, and negative emotions. The analysis spans four distinct scenarios, each representing different combinations of features and preprocessing strategies. Notably primary emphasis is on LSTM and GRU architectures due to their proven effectiveness in sequential data processing tasks. In addition to evaluating the performance of LSTM and GRU models a thorough examination of all the four cases to assess their efficacy in accurately predicting emotional states from EEG signals. The study also emphasis on the importance of meticulous feature selection and preprocessing in optimizing model accuracy and robustness, highlighting the intricate interplay between data representation, model architecture, and task-specific requirements. Ultimately, the research findings contribute to advanced understanding of EEG-based emotion recognition and pave the way for further research in this exciting interdisciplinary field. Key Words: Electroencephalography(EEG), Long short-term memory (LSTM), Gated recurrent unit (GRU), Feature extraction, Preprocessing techniques, Statistical metrics, Frequency domain analysis.

Enhancing CNN‐LSTM neural networks using jellyfish search algorithm for pandemic modeling

SummaryThis paper presents a comprehensive three‐step approach (CNN‐JSO‐LSTM) for predictive modeling using a pandemic such as COVID‐19 as a test case. Initially, a Convolutional Neural Network (CNN) is employed to extract crucial features pertinent to the pandemic. Subsequently, the Jellyfish Search Optimizer (JSO) algorithm is applied for feature selection, identifying the most relevant factors. These chosen features are then inputted into a Long Short‐Term Memory (LSTM) network, responsible for classifying samples into “healthy” and “diseased” categories. Our method enhances LSTM performance using the Jellyfish Search optimizer, resulting in exceptional prediction accuracy. Our experiments achieved remarkable metrics, with an accuracy of 95.32%, high sensitivity (94.87%), and precision (94.28%), surpassing alternative methods. In conclusion, our study presents a promising and highly accurate approach for pandemic prediction, harnessing deep learning and swarm intelligence techniques. These findings suggest a potential for more effective pandemic management and intervention strategies.

Impacts of spatiotemporal resolutions of precipitation on flood event simulation based on multimodel structures – a case study over the Xiang River basin in China

Abstract. Accurate flood event simulation and prediction, enabled by effective models and reliable data, are critical for mitigating the potential risk of flood disaster. This study aims to investigate the impacts of spatiotemporal resolutions of precipitation on flood event simulation in a large-scale catchment of China. We use high-spatiotemporal-resolution Integrated Multi-satellite Retrievals for Global Precipitation Measurement (IMERG) products and a gauge-based product as precipitation forcing for hydrologic simulation. Three hydrological models (HBV, SWAT and DHSVM) and a data-driven model (long short-term memory (LSTM) network) are utilized for flood event simulation. Two calibration strategies are carried out, one of which targets matching of the flood events, with peak discharge exceeding 8600 m3 s−1 between January 2015 and December 2017, and the other one is the conventional strategy for matching the entire streamflow time series. The results indicate that the event-based calibration strategy improves the performance of flood event simulation compared with a conventional calibration strategy, except for DHSVM. Both hydrological models and LSTM yield better flood event simulation at a finer temporal resolution, especially in flood peak simulation. Furthermore, SWAT and DHSVM are less sensitive to the spatial resolutions of IMERG, while the performance of LSTM obtains improvement when degrading the spatial resolution of IMERG-L. Generally, LSTM outperforms the hydrological models in most flood events, which implies the usefulness of the deep learning algorithms for flood event simulation.

Emergency sign language recognition from variant of convolutional neural network (CNN) and long short term memory (LSTM) models

Sign language is the primary communication tool used by the deaf community and people with speaking difficulties, especially during emergencies. Numerous deep learning models have been proposed to solve the sign language recognition problem. Recently. Bidirectional LSTM (BLSTM) has been proposed and used in replacement of Long Short-Term Memory (LSTM) as it may improve learning long-team dependencies as well as increase the accuracy of the model. However, there needs to be more comparison for the performance of LSTM and BLSTM in LRCN model architecture in sign language interpretation applications. Therefore, this study focused on the dense analysis of the LRCN model, including 1) training the CNN from scratch and 2) modeling with pre-trained CNN, VGG-19, and ResNet50. Other than that, the ConvLSTM model, a special variant of LSTM designed for video input, has also been modeled and compared with the LRCN in representing emergency sign language recognition. Within LRCN variants, the performance of a small CNN network was compared with pre-trained VGG-19 and ResNet50V2. A dataset of emergency Indian Sign Language with eight classes is used to train the models. The model with the best performance is the VGG-19 + LSTM model, with a testing accuracy of 96.39%. Small LRCN networks, which are 5 CNN subunits + LSTM and 4 CNN subunits + BLSTM, have 95.18% testing accuracy. This performance is on par with our best-proposed model, VGG + LSTM. By incorporating bidirectional LSTM (BLSTM) into deep learning models, the ability to understand long-term dependencies can be improved. This can enhance accuracy in reading sign language, leading to more effective communication during emergencies.

Denoising CSAMT signals in the time domain based on long short-term memory

Abstract Controlled-source audio frequency magnetotellurics (CSAMT) is an artificial-source electromagnetic technique that partially mitigates the limitations of weak natural-field signals. However, practical field surveys inevitably encounter strong interference, severely affecting signal quality. Traditional methods such as Fourier transformation, which directly computes apparent resistivity from frequency-domain information, are inadequate in this context, so we need alternative denoising approaches. However, research on CSAMT denoising is currently limited. Given the excellent performance of long short-term memory (LSTM) neural networks in the processing of magnetotelluric (MT) data, as demonstrated by previous studies, this paper proposes the use of LSTM to denoise CSAMT signals in the time domain. Unlike traditional denoising methods, we aim to directly extract the target frequency signal from the time-series data for denoising. For MT data, target frequency signals and noise are all mixed together, so noise suppression can only be achieved by identifying noise characteristics in the time series. However, unlike MT data, CSAMT data has an artificial transmitting source, and the frequency of the valid signal is fixed within a time interval. This allows for the direct extraction of target frequency signals without considering the complex characteristics of noise. In this study, we developed a neural network based on bidirectional LSTM to accomplish the task of noise suppression. After conducting both simulated and measured data tests, this method was able to, on average, improve the signal-to-noise ratio (SNR) of CSAMT data by approximately 20 dB and partially address the challenge of denoising when the data's SNR falls below 0 dB.

Short-term prediction of on-street parking occupancy using multivariate variable based on deep learning

Short-term prediction of on-street parking occupancy is essential to the ITS system, which can guide drivers in finding vacant parking spaces. And the spatial dependencies and exogenous dependencies need to be considered simultaneously, which makes short-term prediction of on-street parking occupancy challenging. Therefore, this paper proposes a deep learning model for predicting block-level parking occupancy. First, the importance of multiple points of interest (POI) in different buffers is sorted by Boruta, used for feature selection. The results show that different types of POI data should consider different buffer radii. Then based on the real on-street parking data, long short-term memory (LSTM) that can address the time dependencies is applied to predict the parking occupancy. The results demonstrate that LSTM considering POI data after Boruta selection (LSTM (+BORUTA)) outperforms other baseline methods, including LSTM, with an average testing MAPE of 11.78%. The selection process of POI data helps LSTM reduce training time and slightly improve the prediction performance, which indicates that complex correlations among the same type of POI data in different buffer zones will also affect the prediction accuracy of LSTM. When there are more restaurants on both sides of the street, the prediction performance of LSTM (+BORUTA) is significantly better than that of LSTM.

Research on Time Series-Based Pipeline Ground Penetrating Radar Calibration Angle Prediction Algorithm.

The pipeline ground-penetrating radar stands as an indispensable detection device for ensuring underground space security. A wheeled pipeline robot is deployed to traverse the interior of urban underground drainage pipelines along their central axis. It is subject to influences such as resistance, speed, and human factors, leading to deviations in its posture. A guiding wheel is employed to rectify its roll angle and ensure the precise spatial positioning of defects both inside and outside the pipeline, as detected by the radar antenna. By analyzing its deflection factors and correction trajectories, the intelligent correction control of the pipeline ground-penetrating radar falls into the realm of nonlinear multi-constraint optimization. Consequently, a time-series-based correction angle prediction algorithm is proposed. The application of the long short-term memory (LSTM) deep learning model facilitates the prediction of correction angles and torque for the guiding wheel. This study compares the performance of LSTM with an autoregressive integrated moving average model under identical dataset conditions. The subsequent findings reveal a reduction of 4.11° and 8.25 N·m in mean absolute error, and a decrease of 10.66% and 7.27% in mean squared error for the predicted correction angles and torques, respectively. These outcomes are achieved utilizing the three-channel drainage pipeline ground-penetrating radar device with top antenna operating at 1.2 GHz and left/right antennas at 750 MHz. The LSTM prediction model intelligently corrects its posture. Experimental results demonstrate an average correction speed of 5 s and an average angular error of ±1°. It is verified that the model can correct its attitude in real-time with small errors, thereby enhancing the accuracy of ground-penetrating radar antennas in locating pipeline defects.

Stratification of Depressed and Non-Depressed Texts from Social Media using LSTM and its Variants

This work examines the performance of various LSTM (long short-term memory) variants on social media text data. This study evaluates the performance of LSTM models with different architectures, namely, classic LSTM, Bidirectional LSTM, Stacked LSTM, gated recurrent unit (GRU), and bidirectional GRU, on a social network dataset comprising texts extracted from multiple social media platforms. We aim to identify the most effective LSTM variant of the five considered LSTM models for text analysis through a comparative study of the models’ precision, recall, F1-score, and accuracy. The research findings show that the Classic LSTM and the GRU model perform better than the other models in accuracy. In contrast, the bidirectional models (Bidirectional LSTM and Bidirectional GRU) provide better precision scores than their respective primitive models. This research has significant implications for developing more efficient models for natural language processing applications. It offers beneficial insights into the implications involving the scrutiny of depression on social media platforms through text data analysis.

Predictive Analytics for Production Line Downtime: A Comprehensive Study Using Advanced Machine Learning Models

This study delves into predictive analysis, employing advanced machine learning models to predict downtime in production lines. The research follows a detailed methodology, starting by exploring production lines and their conditions. The focal point is Overall Equipment Effectiveness (OEE), which includes availability, performance, and quality, with a special focus on downtime impacts. Through an OEE tracking system, downtime is categorized into planned and unplanned, offering insights into critical components affecting production line availability. The datasets cover three years, including runtime, unplanned downtime due to equipment failures, and idle times, emphasizing instances prone to frequent equipment failures. 
 
 Examining relationships through a heatmap reveals essential correlations crucial for accurate predictions. We outline the feature selection process and fundamental data preprocessing steps to maintain dataset integrity. Introducing machine learning models such as Xgboost, Prophet, Multi-Layer Perceptron (MLP), and Long Short-Term Memory (LSTM), each model is carefully configured with past window parameters and forecast horizons. To prevent overfitting, the study employs K-fold cross-validation and early stopping mechanisms, ensuring robust model performance. 
 
 Two datasets, comprising runtime, equipment-induced downtime, and idle time, are utilized for analysis. Data preprocessing involves a heatmap for characteristic-target correlation and basic steps like mean-based null value imputation. Results, presented in terms of Mean Absolute Error (MAE), underscore the superior performance of LSTM. The discussion interprets these findings, confirming the potential of LSTM for downtime prediction, supported by visualizations comparing actual and predicted downtimes. Future directions include comprehensive model comparisons, expanding metrics, and increasing datasets, building on the solid foundations established by this study.

Bitcoin price and return prediction based on LSTM

This paper focuses on the prediction of Bitcoin prices and returns based on the Long Short Term Memory (LSTM) neural network model, to better consider the impact of time factors. Since Bitcoin has long dominated the digital currency trading market, many researchers have completed many Bitcoin prediction results, including the screening of optimal features, comparison of prediction models and classification of prediction problems. Based on previous work, this article adds a Bitcoin revenue forecast section, presenting the results in the form of charts and data to provide more intuitive trends and more accurate performance. This paper uses LSTM as the experimental model, and uses the Bitcoin transaction history data set with timestamps as the original input. After a specific normalization method, the original model is trained, and then the subsequent transaction data is predicted. Compare it with the real value in the data set to get the final experimental results show that in this prediction problem, the performance of LSTM is slightly better than Autoregressive Integrated Moving Average (ARIMA) and eXtreme Gradient Boosting (XGBoost); on the other hand, compared with price prediction based on real values for prediction, the prediction fluctuations of return are more obvious and more realistic, providing better reference value.