Performance Of Long Short-term Memory Research Articles

Emergency sign language recognition from variant of convolutional neural network (CNN) and long short term memory (LSTM) models

Sign language is the primary communication tool used by the deaf community and people with speaking difficulties, especially during emergencies. Numerous deep learning models have been proposed to solve the sign language recognition problem. Recently. Bidirectional LSTM (BLSTM) has been proposed and used in replacement of Long Short-Term Memory (LSTM) as it may improve learning long-team dependencies as well as increase the accuracy of the model. However, there needs to be more comparison for the performance of LSTM and BLSTM in LRCN model architecture in sign language interpretation applications. Therefore, this study focused on the dense analysis of the LRCN model, including 1) training the CNN from scratch and 2) modeling with pre-trained CNN, VGG-19, and ResNet50. Other than that, the ConvLSTM model, a special variant of LSTM designed for video input, has also been modeled and compared with the LRCN in representing emergency sign language recognition. Within LRCN variants, the performance of a small CNN network was compared with pre-trained VGG-19 and ResNet50V2. A dataset of emergency Indian Sign Language with eight classes is used to train the models. The model with the best performance is the VGG-19 + LSTM model, with a testing accuracy of 96.39%. Small LRCN networks, which are 5 CNN subunits + LSTM and 4 CNN subunits + BLSTM, have 95.18% testing accuracy. This performance is on par with our best-proposed model, VGG + LSTM. By incorporating bidirectional LSTM (BLSTM) into deep learning models, the ability to understand long-term dependencies can be improved. This can enhance accuracy in reading sign language, leading to more effective communication during emergencies.

Denoising CSAMT signals in the time domain based on long short-term memory

Abstract Controlled-source audio frequency magnetotellurics (CSAMT) is an artificial-source electromagnetic technique that partially mitigates the limitations of weak natural-field signals. However, practical field surveys inevitably encounter strong interference, severely affecting signal quality. Traditional methods such as Fourier transformation, which directly computes apparent resistivity from frequency-domain information, are inadequate in this context, so we need alternative denoising approaches. However, research on CSAMT denoising is currently limited. Given the excellent performance of long short-term memory (LSTM) neural networks in the processing of magnetotelluric (MT) data, as demonstrated by previous studies, this paper proposes the use of LSTM to denoise CSAMT signals in the time domain. Unlike traditional denoising methods, we aim to directly extract the target frequency signal from the time-series data for denoising. For MT data, target frequency signals and noise are all mixed together, so noise suppression can only be achieved by identifying noise characteristics in the time series. However, unlike MT data, CSAMT data has an artificial transmitting source, and the frequency of the valid signal is fixed within a time interval. This allows for the direct extraction of target frequency signals without considering the complex characteristics of noise. In this study, we developed a neural network based on bidirectional LSTM to accomplish the task of noise suppression. After conducting both simulated and measured data tests, this method was able to, on average, improve the signal-to-noise ratio (SNR) of CSAMT data by approximately 20 dB and partially address the challenge of denoising when the data's SNR falls below 0 dB.

Short-term prediction of on-street parking occupancy using multivariate variable based on deep learning

Short-term prediction of on-street parking occupancy is essential to the ITS system, which can guide drivers in finding vacant parking spaces. And the spatial dependencies and exogenous dependencies need to be considered simultaneously, which makes short-term prediction of on-street parking occupancy challenging. Therefore, this paper proposes a deep learning model for predicting block-level parking occupancy. First, the importance of multiple points of interest (POI) in different buffers is sorted by Boruta, used for feature selection. The results show that different types of POI data should consider different buffer radii. Then based on the real on-street parking data, long short-term memory (LSTM) that can address the time dependencies is applied to predict the parking occupancy. The results demonstrate that LSTM considering POI data after Boruta selection (LSTM (+BORUTA)) outperforms other baseline methods, including LSTM, with an average testing MAPE of 11.78%. The selection process of POI data helps LSTM reduce training time and slightly improve the prediction performance, which indicates that complex correlations among the same type of POI data in different buffer zones will also affect the prediction accuracy of LSTM. When there are more restaurants on both sides of the street, the prediction performance of LSTM (+BORUTA) is significantly better than that of LSTM.

Research on Time Series-Based Pipeline Ground Penetrating Radar Calibration Angle Prediction Algorithm.

The pipeline ground-penetrating radar stands as an indispensable detection device for ensuring underground space security. A wheeled pipeline robot is deployed to traverse the interior of urban underground drainage pipelines along their central axis. It is subject to influences such as resistance, speed, and human factors, leading to deviations in its posture. A guiding wheel is employed to rectify its roll angle and ensure the precise spatial positioning of defects both inside and outside the pipeline, as detected by the radar antenna. By analyzing its deflection factors and correction trajectories, the intelligent correction control of the pipeline ground-penetrating radar falls into the realm of nonlinear multi-constraint optimization. Consequently, a time-series-based correction angle prediction algorithm is proposed. The application of the long short-term memory (LSTM) deep learning model facilitates the prediction of correction angles and torque for the guiding wheel. This study compares the performance of LSTM with an autoregressive integrated moving average model under identical dataset conditions. The subsequent findings reveal a reduction of 4.11° and 8.25 N·m in mean absolute error, and a decrease of 10.66% and 7.27% in mean squared error for the predicted correction angles and torques, respectively. These outcomes are achieved utilizing the three-channel drainage pipeline ground-penetrating radar device with top antenna operating at 1.2 GHz and left/right antennas at 750 MHz. The LSTM prediction model intelligently corrects its posture. Experimental results demonstrate an average correction speed of 5 s and an average angular error of ±1°. It is verified that the model can correct its attitude in real-time with small errors, thereby enhancing the accuracy of ground-penetrating radar antennas in locating pipeline defects.

Recurrent Neural Network (RNN)–Based Approach to Uncover the Relationship Between Block Size and Blockchain Performance

Blockchain is a distributed system where transactions are recorded on blocks. The blocks are linked with previous blocks creating a chain of blocks that ensures the integrity of the blockchain. Throughput operates as a pivotal performance indicator in blockchain contributing to its popularity among organizations and individual users. To achieve optimized use of blockchain for an increasing number of users, accurate prediction of performance at different timestamps is necessary. In this paper, two recurrent neural network (RNN)–based predictive models, i.e., long short‐term memory (LSTM) and gated recurrent unit (GRU) are proposed that predict the performance (throughput) of blockchain in different scenarios. Moreover, the study analyzes blockchain data to find optimal block size, impact of block size on performance, and uncover the relationship between performance and block size. The RNN models are evaluated using a dataset collected from Hyperledger Caliper on the HF framework 2.3. This study found that the performance of LSTM is approximately close to GRU regarding evaluation metrics MAE and RMSE. However, GRU is more suitable for practical applications and performs better overall. This study also shows a Spearman correlation value of 0.032, which is indicative of a weak association between the two factors.

Stratification of Depressed and Non-Depressed Texts from Social Media using LSTM and its Variants

This work examines the performance of various LSTM (long short-term memory) variants on social media text data. This study evaluates the performance of LSTM models with different architectures, namely, classic LSTM, Bidirectional LSTM, Stacked LSTM, gated recurrent unit (GRU), and bidirectional GRU, on a social network dataset comprising texts extracted from multiple social media platforms. We aim to identify the most effective LSTM variant of the five considered LSTM models for text analysis through a comparative study of the models’ precision, recall, F1-score, and accuracy. The research findings show that the Classic LSTM and the GRU model perform better than the other models in accuracy. In contrast, the bidirectional models (Bidirectional LSTM and Bidirectional GRU) provide better precision scores than their respective primitive models. This research has significant implications for developing more efficient models for natural language processing applications. It offers beneficial insights into the implications involving the scrutiny of depression on social media platforms through text data analysis.

Predicting Chlorophyll-a Concentration and Harmful Algal Blooms in Lake Okeechobee Using Time-Series MODIS Satellite Imagery and Long Short-Term Memory

Highlights A 10-year dataset of time-series MODIS imagery and in situ Chl-a concentration were curated for Lake Okeechobee. LSTM significantly outperformed KNN, SVR, and RF for Chl-a prediction and subsequent HAB detection. The optimal window length was found to be 13 days with a 4-day temporal resolution for the LSTM model. KNN, SVR, and RF models were not effective at utilizing the temporal dynamics of the input features. Abstract. Harmful algal blooms (HABs) in inland waterbodies are a global concern due to their negative impact on human, animal, and ecosystem health. Chlorophyll-a (Chl-a) concentration is an important water quality parameter for monitoring HABs. While statistical and machine learning (ML) models have been widely studied to predict Chl-a concentration and HABs based on single-time-point satellite data, this work assessed whether long short-term memory (LSTM) can improve both tasks by leveraging temporal features in time-series MODIS satellite images compared to three classical ML models, including k-nearest neighbor (KNN), support vector regression (SVR), and random forest (RF). A dataset of daily MODIS images and monthly in situ Chl-a concentration measurements from 2011 to 2020 was curated for Lake Okeechobee, Florida. A window size of 13 days with a temporal resolution of four days was found to produce the optimal performance for LSTM, which significantly outperformed KNN, SVR, and RF for Chl-a prediction with a root mean square error of 11.95 µg/L, a mean absolute error of 8.55 µg/L, and a R2 value of 0.43. The superior performance of LSTM for Chl-a prediction was likely due to its ability to leverage the temporal dynamics in the features associated with HAB development. The Chl-a predictions were further used to determine HAB events, showing better accuracy and a significantly higher F1 score for LSTM over the other models. The study suggested that combining LSTM with high-temporal-resolution time-series data should be preferred over applying common ML models on time-series or single-time-point remote sensing data for Chl-a and HAB monitoring. Keywords: Cyanobacteria, LSTM, Machine Learning, Remote Sensing, Water Quality.

Predictive Analytics for Production Line Downtime: A Comprehensive Study Using Advanced Machine Learning Models

This study delves into predictive analysis, employing advanced machine learning models to predict downtime in production lines. The research follows a detailed methodology, starting by exploring production lines and their conditions. The focal point is Overall Equipment Effectiveness (OEE), which includes availability, performance, and quality, with a special focus on downtime impacts. Through an OEE tracking system, downtime is categorized into planned and unplanned, offering insights into critical components affecting production line availability. The datasets cover three years, including runtime, unplanned downtime due to equipment failures, and idle times, emphasizing instances prone to frequent equipment failures. 
 
 Examining relationships through a heatmap reveals essential correlations crucial for accurate predictions. We outline the feature selection process and fundamental data preprocessing steps to maintain dataset integrity. Introducing machine learning models such as Xgboost, Prophet, Multi-Layer Perceptron (MLP), and Long Short-Term Memory (LSTM), each model is carefully configured with past window parameters and forecast horizons. To prevent overfitting, the study employs K-fold cross-validation and early stopping mechanisms, ensuring robust model performance. 
 
 Two datasets, comprising runtime, equipment-induced downtime, and idle time, are utilized for analysis. Data preprocessing involves a heatmap for characteristic-target correlation and basic steps like mean-based null value imputation. Results, presented in terms of Mean Absolute Error (MAE), underscore the superior performance of LSTM. The discussion interprets these findings, confirming the potential of LSTM for downtime prediction, supported by visualizations comparing actual and predicted downtimes. Future directions include comprehensive model comparisons, expanding metrics, and increasing datasets, building on the solid foundations established by this study.

Bitcoin price and return prediction based on LSTM

This paper focuses on the prediction of Bitcoin prices and returns based on the Long Short Term Memory (LSTM) neural network model, to better consider the impact of time factors. Since Bitcoin has long dominated the digital currency trading market, many researchers have completed many Bitcoin prediction results, including the screening of optimal features, comparison of prediction models and classification of prediction problems. Based on previous work, this article adds a Bitcoin revenue forecast section, presenting the results in the form of charts and data to provide more intuitive trends and more accurate performance. This paper uses LSTM as the experimental model, and uses the Bitcoin transaction history data set with timestamps as the original input. After a specific normalization method, the original model is trained, and then the subsequent transaction data is predicted. Compare it with the real value in the data set to get the final experimental results show that in this prediction problem, the performance of LSTM is slightly better than Autoregressive Integrated Moving Average (ARIMA) and eXtreme Gradient Boosting (XGBoost); on the other hand, compared with price prediction based on real values for prediction, the prediction fluctuations of return are more obvious and more realistic, providing better reference value.

Performance of LSTM over SWAT in Rainfall-Runoff Modeling in a Small, Forested Watershed: A Case Study of Cork Brook, RI

The general practice of rainfall-runoff model development towards physically based and spatially explicit representations of hydrological processes is data-intensive and computationally expensive. Physically based models such as the Soil Water Assessment tool (SWAT) demand spatio-temporal data and expert knowledge. Also, the difficulty and complexity is compounded in the smaller watershed due to data constraint and models’ inability to generalize hydrologic processes. Data-driven models can bridge this gap with their mathematical formulation. Long Short-Term Memory (LSTM) is a data-driven model with Recurrent Neural Network (RNN) architecture, which is better suited to solve time series problems. Studies have shown that LSTM models have competitive performance in watershed hydrology studies. In this study, a comparative analysis of SWAT and LSTM models in the Cork Brook watershed shows that results from LSTM were competitive to SWAT in flow prediction with NSE of 0.6 against 0.63, respectively, given the limited availability of data. LSTM models do not overestimate the high flows like SWAT. However, both these models struggle with low values estimation. Although interpretability, explainability, and use of models across different datasets or events outside of the training data may be challenging, LSTM models are robust and efficient.

Study on the Applicability of LSTM for Predicting Stock Price when Facing Extreme Events

The analysis of stock price fluctuations holds considerable significance in the field of economics, particularly given the present environment characterized by unpredictability and rapid changes. Previously, the long short-term memory (LSTM) model has been employed effectively in addressing time series problems, including stock market forecasting. However, in the current dynamic landscape, the ability of LSTM to adapt to volatile conditions and provide accurate predictions is an area that merits further investigation. This study gathers stock data from prominent and representative companies, namely Apple, Google, Amazon, and Microsoft, spanning from January 2012 to March 2023. Specifically, two significant events are examined: the impact of the Covid-19 outbreak on the US stock market on February 26, 2020, and the Russia-Ukraine conflict occurring on February 26, 2022. By dividing the stock data surrounding these events into training and test sets, this research aims to evaluate the differential performance of LSTM in scenarios where it possesses no prior knowledge of these events versus situations where it has already assimilated the influence exerted by them.

Temporal data-driven short-term traffic prediction: Application and analysis of LSTM model

With the development of the economy, the number of private cars has increased, and traffic congestion is becoming increasingly common. In order to prevent traffic congestion, short-term prediction of traffic flow is urgent. This article is based on the data of a Hangzhou elevated bridge from October 4th to 18th, 2015, with a timestamp of 5 minutes and a total of over 4000 pieces of data, to have the Long Short-Term Memory (LSTM) model to be trained and the following days traffic flow to be predicted. The outcomes show that the model has high predictive capability and can reliably forecast future short-term transportation passenger flow, accurately reflecting the trend of data changes. Compared with the Auto-Regressive Moving Average (ARIMA) method with a Mean-Square Error (MSE) of 36, the LSTM model has a MSE of 22, which indicates a smaller MSE, indicating a more accurate prediction performance of LSTM. The LSTM models prediction is beneficial for alleviating traffic congestion, providing great convenience for peoples daily travel, and greatly reducing their travel time.

Unraveling COVID-19 Progression: A Comprehensive Com-parison of Advanced Deep Learning Methods for Precise Pre-dictions

The COVID-19 pandemic has significantly impacted the United Arab Emirates (UAE), necessitating effective and accurate forecasting tools to inform public health policies and strategies. This study presents a comparative analysis of advanced deep-learning models for predicting COVID-19 cases in the UAE. We investigate the performance of Long Short-Term Memory (LSTM), Bi-directional LSTM, Convolutional Neural Networks (CNN), CNN-LSTM, Multilayer perceptron, and Recurrent Neural Networks (RNN). The models are trained and evaluated using a comprehensive dataset of confirmed cases, demographic information, and relevant socio-economic indicators. The models are further optimized using a Bayesian optimizer and comparison is performed before and after the optimization of models. We have used predictive and perspective analytics on the COVID-19 dataset. Our research goal is to identify the most accurate and reliable model for forecasting COVID-19 cases in the region.
 The results demonstrate the effectiveness of these deep learning techniques in predicting COVID-19 cases, with each model exhibiting varying levels of accuracy and precision. A thorough and rigorous evaluation of the models' performances reveals the most suitable architecture for the UAE's specific context. This study contributes to the ongoing efforts to combat the pandemic by providing valuable insights into the application of advanced deep-learning models for accurate and timely COVID-19 case predictions. It was found that the RNN model performed the best without any optimization. The findings have significant implications for public health decision-making, enabling authorities to develop targeted and data-driven interventions to curb the spread of the virus and mitigate its impact on the UAE's population. This demonstrates the potential of deep learning algorithms in handling complex datasets and making accurate predictions which is a valuable capability to enhance accuracy in professional and healthcare environments.

Spatio-Temporal Crime Predictions

Abstract: Crime and transgressions pose a formidable challenge to the principles of justice and require vigilant control. Precise crime prediction and forecasting of future trends hold the potential to significantly bolster urban safety through computational means. The inherent limitations of human capacity to process intricate information from vast datasets impede our ability to achieve early and precise crime prognostication. The accurate estimation of crime rates, categories, and focal points based on historical patterns presents a plethora of computational challenges and opportunities. Notwithstanding substantial research endeavors, there persists a pressing need for an enhanced predictive algorithm that can effectively guide law enforcement patrols in response to criminal activities. Previous scholarly investigations have fallen short in attaining the desired precision in crime forecasting and prediction by utilizing various machine learning algorithms, including logistic regression, support vector machine (SVM), Naïve Bayes, knearest neighbors (KNN), decision trees, multilayer perceptrons (MLP), random forests, and extreme Gradient Boosting (xgBoost). In addition to these machine learning methodologies, this study also harnessed time series analysis, particularly the long-short term memory (LSTM) and autoregressive integrated moving average (ARIMA) models, to more aptly model crime data. The performance of LSTM in time series analysis exhibited a reasonably satisfactory level of accuracy, as evident from the magnitude of root mean square error (RMSE) and mean absolute error (MAE) on both datasets. A comprehensive exploration of the data unveiled the presence of over 35 distinct crime types and indicated an annual decline in the crime rate in Chicago, coupled with a marginal upturn in the crime rate in Los Angeles. Significantly, there was a lower incidence of reported crimes in February compared to other months. Projections suggest a moderate future increase in Chicago's overall crime rate, with a subsequent probable decline in the years ahead. In contrast, the ARIMA model implies a sharp decrease in the crime rate in Los Angeles. Furthermore, the study's crime forecasting results pinpointed specific high-crime regions in both cities.

Improving microalgae growth modeling of outdoor cultivation with light history data using machine learning models: A comparative study

Accurate prediction of microalgae growth is crucial for understanding the impacts of light dynamics and optimizing production. Although various mathematical models have been proposed, only a few of them have been validated in outdoor cultivation. This study aims to investigate the use of machine learning algorithms in microalgae growth modeling. Outdoor cultivation data of Phaeodactylum tricornutum in flat-panel airlift photobioreactors for 50 days were used to compare the performance of Long Short-Term Memory (LSTM) and Support Vector Regression (SVR) with traditional models, namely Monod and Haldane. The results indicate that the machine learning models outperform the traditional models due to their ability to utilize light history as input. Moreover, the LSTM model shows an excellent ability to describe the light acclimation effect. Last, two potential applications of these models are demonstrated: 1) use as a biomass soft sensor and 2) development of an optimal harvest strategy for outdoor cultivation.

Comparing Long Short-Term Memory (LSTM) and bidirectional LSTM deep neural networks for power consumption prediction

Electric consumption prediction methods are investigated for many reasons, such as decision-making related to energy efficiency as well as for anticipating demand and the dynamics of the energy market. The objective of the present work is to compare two Deep Learning models, namely the Long Short-Term Memory (LSTM) model, and the Bi-directional LSTM (BLSTM) for univariate electric consumption Time Series (TS) short-term forecast model. The Data Sets (DSs) were selected for their different contexts and scales, with the goal of assessing the robustness of the models. Four DSs were used, related to the power consumption of: (a) a household in France; (b) a university building in Santarém, Brazil; (c) the Tétouan city zones, in Morocco; and (d) the aggregated electric demand of Singapore. The metrics RMSE, MAE, MAPE and R2 were calculated in a TS cross-validation scheme. Friedman’s test was applied to normalized RMSE (NRMSE) results, showing that BLSTM outperforms LSTM with statistically significant difference (p = 0.0455), corroborating the fact that bidirectional weight updating significantly improves the LSTM performance with respect to different scales of electric power consumption. The present work provides statistical evidence supporting the conclusion that BLSTM outperforms LSTM models according to the tests performed, based on a complete methodology for TS prediction, and also establishes a baseline for future investigation of electric consumption TS prediction.

Stock Price Prediction using LSTM and KNN Algorithms

Abstract: The aim of this study is to enhance and evaluate the performance of Long Short-Term Memory (LSTM) and K Nearest Neighbor (KNN) algorithms when used to forecast stock prices. In this study, previously recorded data on price as well as other crucial factors like transaction volume and market sentiment must be collected from reliable sources. The information gathered is then cleaned, preprocessed and refined in order to make sure it is compatible with the training model. Then, we developed two forecasting models, one using (LSTM) and (KNN). LSTM models are trained using historical data, and the performance of these models is evaluated using a variety of metrics, such as mean squared error and mean error. The performance evaluation and comparison of the above two models are conducted saying LSTM model outperforms the KNN model in accuracy.

Text classification in fair competition law violations using deep learning

IntroductionEnsuring fair competition through manual review is a complex undertaking. This paper introduces the utilization of Long Short-Term Memory (LSTM) neural networks and TextCNN to establish a text classifier for classifying and reviewing normative documents.MethodsThe experimental dataset used consists of policy measure samples provided by the antitrust division of the Guangdong Market Supervision Administration. We conduct a comparative analysis of the performance of LSTM and TextCNN classification models.ResultsIn three classification experiments conducted without an enhanced experimental dataset, the LSTM classifier achieved an accuracy of 95.74%, while the TextCNN classifier achieved an accuracy of 92.7% on the test set. Conversely, in three classification experiments utilizing an enhanced experimental dataset, the LSTM classifier demonstrated an accuracy of 96.36%, and the TextCNN classifier achieved an accuracy of 96.19% on the test set.DiscussionThe experimental results highlight the effectiveness of LSTM and TextCNN in classifying and reviewing normative documents. The superior accuracy achieved with the enhanced experimental dataset underscores the potential of these models in real-world applications, particularly in tasks involving fair competition review.

Vibration prediction of offshore wind turbines based on long short-term memory network

ABSTRACT Offshore wind turbines (OWTs) have complex operating conditions and are prone to accidents. Timely and accurate vibration prediction can reduce the probability of failure of OWTs. To address this issue, this study proposed a vibration prediction model based on long short-term memory (LSTM) network. The proposed prediction model was applied to an OWT in an offshore wind farm in China, and three evaluation indicators, i.e. mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean square error (RMSE), were used to verify the vibration prediction performance of LSTM. The results demonstrated that the prediction accuracy of LSTM was better than that of back propagation (BP) neural network, and LSTM had optimum engineering practicability. The prediction accuracy of LSTM and BP neural networks decreased with an increase in prediction durations. However, the prediction accuracy of LSTM was higher. This study improves the online safety monitoring of OWTs.

A Hybrid Optimization Approach for Neural Machine Translation Using LSTM+RNN with MFO for Under Resource Language (Telugu)

NMT (Neural Machine Translation) is an innovative approach in the field of machine translation, in contrast to SMT (statistical machine translation) and Rule-based techniques which has resulted annotable improvements. This is because NMT is able to overcome many of the shortcomings that are inherent in the traditional approaches. The Development of NMT has grown tremendously in the recent years but NMT performance remain under optimal when applied to low resource language pairs like Telugu, Tamil and Hindi. In this work a proposedmethod fortranslating pairs (Telugu to English) is attempted, an optimal approach which enhancesthe accuracy and execution time period.A hybrid method approach utilizing Long short-term memory (LSTM) and traditional Recurrent Neural Network (RNN) are used for testing and training of the dataset. In the event of long-range dependencies, LSTM will generate more accurate results than a standard RNN would endure and the hybrid technique enhances the performance of LSTM. LSTM is used during the encoding and RNN is used in decoding phases of NMT. Moth Flame Optimization (MFO) is utilized in the proposed system for the purpose of providing the encoder and decoder model with the best ideal points for training the data.