Traditional Recurrent Neural Networks Research Articles

Development of Robot Feature for Stunting Analysis Using Long-Short Term Memory (LSTM) Algorithm

Stunting prevalence in Indonesia persists as a significant challenge necessitating concerted efforts from all stakeholders. We developed robot for stunting analysis using deep learning algorithm. It aligns with the Sustainable Development Goal (SDG) agenda, specifically targeting SDG 3, which focuses on ensuring good health and well-being for all. Long Short-Term Memory (LSTM) is a type of Recurrent Neural Network (RNN) designed to overcome the vanishing gradient problem in traditional RNNs. In general, either LSTM can be used in analysis. This study aims to classify stunting based on age and height using LSTM. The LSTM model was trained with 50 epochs using datasets collected from the health office and robots. The evaluation results show training accuracy of 96.65% and training validation of 96.61%, with precision, recall and f1-score varying with relevance to the f1-score and support value. This research illustrates the potential for using data classification methods in stunting diagnosis. However, it is necessary to adjust parameters and increase the amount of training data to improve model performance. With good convergence at epoch 50, these results show the model's ability to classify stunting based on age and height. However, further validation and testing on larger datasets is needed to thoroughly test the reliability and generalization of the model. This research can contribute to the development of deep learning regarding robots as a means of testing stunting. This research provides initial evidence of the potential of stunting classification methods using robots. However, parameter adjustments and increasing the amount of training data need to be done to improve the overall model performance.

RPEPL: Tibetan Sentiment Analysis Based on Relative Position Encoding and Prompt Learning

Sentiment analysis is a critical task for natural language processing. Much research has been done for high-resource languages such as English and Chinese. However, Tibetan is an extremely low-resource language with less reference information. According to the practical demands, this paper proposes a Tibetan sentiment analysis method based on relative position encoding and prompt learning, the method is abbreviated as RPEPL. First, Word information is introduced to syllable sequences by converting the directed acyclic lattice into a squashed structure. Second, a relative position encoding is used to encode the position information of syllables and words. Then the association relations and semantic information of tokens are identified by leveraging the multi-attention. Finally, the sentiment category of Tibetan sentence is obtained through a prompt learning framework. Experimental results demonstrate that RPEPL significantly outperforms the baseline methods on the TUSA dataset and TNEC dataset. Additionally, Traditional recurrent neural networks cannot perform large-scale parallel computation and convolutional neural networks have difficulty modeling long-distance dependencies in Tibetan text both of which are resolved using RPEPL. Furthermore, the use of a multi-attention not only enriches the association relations between syllables and words, but also enhances the understanding of sentence semantic and syntactic structure information, and improves the performance of Tibetan sentiment analysis.

Predicting temporal evacuation travel time in staircases between adjacent floors of super high-rise buildings by artificial neural networks

Predicting temporal evacuation travel time in staircases between adjacent floors is crucial for dynamic evacuation guidance to improve vertical evacuation efficiency during emergencies in super high-rise buildings. However, the prediction methods used in previous studies mainly focus on the total evacuation time in buildings and cannot predict the temporal evacuation travel time in every single part of the total evacuation route. This study proposes a novel method to predict temporal evacuation travel time in staircases between adjacent floors of super high-rise buildings by using artificial neural networks. Firstly, evacuation simulation data of a 71-storey office building are analyzed for feature engineering and dataset building. Secondly, three types of artificial neural networks, including convolutional neural networks, RNN-based model (the traditional recurrent neural networks, long-short term memory networks, and bidirectional long-short term memory networks), and Attention-based model (Attention model and Transformer model) are extended to predict the temporal evacuation travel time in staircases between adjacent floors based on the simulation data. Finally, the prediction results by different artificial neural networks are compared. The results show that the artificial neural networks can provide accurate prediction values for the temporal evacuation travel time in staircases between adjacent floors of super high-rise buildings.

Deep learning approaches in predicting tropical cyclone tracks: An analysis focused on the Northwest Pacific Region

In this study, we conducted a comprehensive and integrated test of tropical cyclone track prediction using deep learning technologies, aiming to enhance the efficiency and accuracy of the prediction methods. We employed the Best Track dataset from the China Meteorological Administration's Tropical Cyclone Data Center, which covers the Northwest Pacific region from 1949 to 2023. This dataset provides comprehensive coverage, encompassing critical tropical cyclone details like time, latitude, longitude, and wind speed. Our focus was on evaluating and comparing different deep learning models, including Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), and Gated Recurrent Units (GRU), for their effectiveness in handling complex time series data. Through detailed analysis of various model configurations, including factors such as input-output lengths, hidden size, the number of layers, the implementation of bi-directional networks, and attention mechanisms, we discovered that LSTM and GRU models significantly outperform traditional RNN models in dealing with long-term dependencies and enhancing prediction accuracy. Moreover, the LSTM model, used to forecast key tropical cyclones during the 2023 Pacific tropical cyclone season, achieved mean errors of 21.84 km, 37.56 km, and 26.12 km for Typhoons Mawar, Doksuri, and Saola, respectively. This method also demonstrated high efficiency in rapid response to extreme weather changes, processing each tropical cyclone's forecast in just about 8 s. The results not only illustrate the practical utility of deep learning in tropical cyclone track prediction but also provide new, effective tools for disaster prevention and mitigation efforts.

Gated recurrent unit and temporal convolutional network with soft thresholding and attention mechanism for tool wear prediction

Predicting tool wear significantly improves cutting efficiency and quality in intelligent manufacturing. Traditional recurrent neural networks suffer from vanishing or exploding gradients and need to be improved prediction accuracy. Therefore, this paper proposes a gated recurrent unit and temporal convolutional network with soft thresholding and attention mechanism (TCN-BiGRU-SA). Monotonicity, robustness, and trendability indicators are used to assess time and frequency domain features. Then, evaluation results of these three indicators are weighted to identify the sensitive features with the degradation trend. The attention mechanism is proposed to adaptively adjust the soft thresholding, and the SA is added to TCN-BiGRU to remain helpful features. Finally, the proposed model is validated using an experimental dataset. Compared to the TCN-BiGRU model without the SA mechanism, the predicted results show that the average mean absolute error and root mean square error are reduced by 48.29 % and 36.39 %, respectively.

LSTM-Based Gasoline Price Prediction - An Example of Gasoline 95# in Beijing

Gasoline, as one of the indispensable energy sources in the global economy, the stability of its price has an important impact on national energy security and economic development. This study aims to improve the accuracy of gasoline price prediction through the use of Long Short-Term Memory Network (LSTM). By collecting and analyzing the historical price data of No. 95 gasoline in Beijing, the author constructed a prediction model based on LSTM, which is particularly suitable for dealing with long-term dependence in time series data. By comparing their performance to the traditional Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN), LSTM demonstrates its advantages in processing complicated time series data. The validity of the model is verified by several evaluation metrics, including mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). The results of the study indicate that the LSTM model shows high accuracy and reliability in predicting gasoline prices. In addition, future work will explore the introduction of multivariate analysis and hybrid network models to further the accuracy of the predictions and its generalization ability.

Channel estimation for fiber-terahertz communication based on conditional generative adversarial networks

Fiber-terahertz communication is regarded as one of the best candidates for future 6G mobile communication systems. Using fiber optic-assisted terahertz communication can solve the problem of large-capacity terahertz coverage. Still, it also leads to more complex and variable channel environments. This makes channel estimation for fiber terahertz communication challenging. In this paper, we use the conditional generative adversarial networks (CGAN) to solve channel estimation problems with their excellent mapping ability to complex relationships. Meanwhile, we further improved the performance of the network by using a residual structure and an unrolled structure. Compared with the traditional recurrent neural networks (RNN) and convolutional neural networks (CNN), the normalized mean squared error (NMSE) of CGAN-based fiber terahertz channel estimation is reduced by up to 1.93 dB and 1.02 dB, respectively.

Enhancing stock market prediction accuracy with recurrent deep learning models: A case study on the CAC40 index

This paper explores the application of Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks for stock price prediction over a 10-day horizon. The study aims to compare the predictive performance of these two deep learning architectures within the context of financial forecasting. Utilizing historical stock data from the CAC40 dataset, which represents a capitalization-weighted measure of the 40 most significant stocks on the Euronext Paris, we train and evaluate RNN and LSTM models to forecast future stock prices. Our results demonstrate the superior performance of LSTM networks in capturing the intricate temporal dependencies inherent in stock price data. Compared to standard RNNs, LSTM models exhibit higher accuracy and provide more reliable forecasts over the 10-day prediction period. The specialized memory cells and gating mechanisms in LSTM networks enable them to effectively identify both short-term changes and long-term patterns in stock prices, thus outperforming traditional RNN architectures. This enhanced ability to model the complex dynamics of stock market data underscores the potential of LSTM networks to improve investment decision-making, risk management, and the overall efficiency of financial markets. The insights gained from this study contribute to the growing body of knowledge on the application of deep learning in finance and investment, offering valuable guidance for practitioners and researchers seeking to harness the power of advanced algorithms for stock market prediction and optimization.

Exploring Anomaly Detection and Risk Assessment in Financial Markets Using Deep Neural Networks

In this paper, deep learning technology, along with a Gated Recurrent Unit (GRU) combined with an attention mechanism, is used to enhance the recognition ability and risk assessment accuracy of abnormal trading behavior in financial markets. The GRU effectively solves the problem of gradient vanishing in traditional recurrent neural networks through its unique gated structure, allowing the model to learn more stable and effective feature representations in long sequence data. On this basis, the contextual attention (CA) module in the attention mechanism is introduced, enabling the model to automatically learn and assign different weights to various parts of the input sequence. Combined with bidirectional GRU and the attention mechanism, the model can not only capture temporal dependencies in the sequence, but also highlight the key features that affect market anomalies, thus improving the model's ability to understand complex market dynamics.

SMARTSeiz: Deep Learning With Attention Mechanism for Accurate Seizure Recognition in IoT Healthcare Devices.

The Internet of Things (IoT) is capable of controlling the healthcare monitoring system for remote-based patients. Epilepsy, a chronic brain syndrome characterized by recurrent, unpredictable attacks, affects individuals of all ages. IoT-based seizure monitoring can greatly enhance seizure patients' quality of life. IoT device acquires patient data and transmits it to a computer program so that doctors can examine it. Currently, doctors invest significant manual effort in inspecting Electroencephalograph (EEG) signals to identify seizure activity. However, EEG-based seizure detection algorithms face challenges in real-world scenarios due to non-stationary EEG data and variable seizure patterns among patients and recording sessions. Therefore, a sophisticated computer-based approach is necessary to analyze complex EEG records. In this work, the authors proposed a hybrid approach by combining traditional convolution neural (CN) and recurrent neural networks (RNN) along with an attention mechanism for the automatic recognition of epileptic seizures through EEG signal analysis. This attention mechanism focuses on significant subsets of EEG data for class recognition, resulting in improved model performance. The proposed methods are evaluated using a publicly available UCI epileptic seizure recognition dataset, which consists of five classes: four normal conditions and one abnormal seizure condition. Experimental results demonstrate that the suggested approach achieves an overall accuracy of 97.05% for the five-class EEG recognition data, with an accuracy of 99.52% for binary classification distinguishing seizure cases from normal instances. Furthermore, the proposed intelligent seizure recognition model is compatible with an IoMT (Internet of Medical Things) cloud-based smart healthcare framework.

Prediksi Harga Bitcoin Menggunakan Algoritma Long Short Term Memory (LSTM)

Cryptocurrency is a digital currency made from sequence code or called blockchain, one of the cryptocurrencies is bitcoin. Prediction is a process that projects or imagines what might happen in the future based on data in the past or factors that influence the current situation. Bitcoin price prediction uses a deep learning approach with the Long Short Term Memory method. Long Short-Term Memory is a type of model from the Recurrent Neural Network (RNN) algorithm, a method designed to process data and can overcome the problem of price movements that have long-term dependencies that cannot be handled by traditional RNN models, the LSTM model has the ability “Remembers” information over long periods of time, so as to recognize patterns and trends. In this study, the prediction period used a dataset from 12 December 2020 to 14 April 2024. The evaluation results for the RMSE method for train data were 17318.40 and for test data were 27921.84 and for the MAPE method for train data it was 3.24% and for test data it was 3.24%. 5.36%. This shows that the RMSE and MAPE values in the data train are relatively small because they are vulnerable to the bitcoin price being too wide.

Time-Dependent Deep Learning Prediction of Multiple Sclerosis Disability.

The majority of deep learning models in medical image analysis concentrate on single snapshot timepoint circumstances, such as the identification of current pathology on a given image or volume. This is often in contrast to the diagnostic methodology in radiology where presumed pathologic findings are correlated to prior studies and subsequent changes over time. For multiple sclerosis (MS), the current body of literature describes various forms of lesion segmentation with few studies analyzing disability progression over time. For the purpose of longitudinal time-dependent analysis, we propose a combinatorial analysis of a video vision transformer (ViViT) benchmarked against traditional recurrent neural network of Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) architectures and a hybrid Vision Transformer-LSTM (ViT-LSTM) to predict long-term disability based upon the Extended Disability Severity Score (EDSS). The patient cohort was procured from a two-site institution with 703 patients' multisequence, contrast-enhanced MRIs of the cervical spine between the years 2002 and 2023. Following a competitive performance analysis, a VGG-16-based CNN-LSTM was compared to ViViT with an ablation analysis to determine time-dependency of the models. The VGG16-LSTM predicted trinary classification of EDSS score in 6years with 0.74 AUC versus the ViViT with 0.84 AUC (p-value < 0.001 per 5 × 2 cross-validation F-test) on an 80:20 hold-out testing split. However, the VGG16-LSTM outperformed ViViT when patients with only 2years of MRIs (n = 94) (0.75 AUC versus 0.72 AUC, respectively). Exact EDSS classification was investigated for both models using both classification and regression strategies but showed collectively worse performance. Our experimental results demonstrate the ability of time-dependent deep learning models to predict disability in MS using trinary stratification of disability, mimicking clinical practice. Further work includes external validation and subsequent observational clinical trials.

An efficient transformer-based surrogate model with end-to-end training strategies for automatic history matching

In recent years, surrogate models have gained popularity as a tool to tackle the challenges presented by time-consuming numerical simulations in automatic history matching (AHM). Although there are many different surrogate models designed to alleviate this issue, it is still challenging for most of them to handle the high dimensionality and strong non-linearity in reservoir models and dynamic production data. Inspired by the rapid development of Transformer, we propose a novel hybrid hierarchical Vision Transformer (HHVT) approach for history matching, which utilizes a unified architecture to predict production data for specific physical fields with an end-to-end strategy. For predicting the production data of wells, the Transformer supports parallelism computation among multiple timesteps, which shows more superiority than traditional recurrent neural networks. Specifically, our approach constructs a novel encoder-decoder Transformer architecture to learn the implicit features of high-level spatial parameters to match the features of time-series production data. With this architecture, HHVT achieves fast training and inference, which is suitable for large-scale datasets and high-dimensional features. The proposed HHVT model is integrated with a multimodal optimization algorithm to find history-matching solutions. We first validated the effect of hyperparameters of HHVT on a simple 2D reservoir. Moreover, the proposed method was verified on the complex 3D Brugge model. The results demonstrate that the training speed of the Transformer-based model is approximately twice as fast as the surrogates based on convolutional and recurrent neural networks. The proposed HHVT also shows better prediction accuracy in two cases, compared with other surrogate models, which enhances the applicability of surrogate-based history-matching methods in large-scale complex reservoir scenarios.

Corrector LSTM: built-in training data correction for improved time-series forecasting

Traditional recurrent neural networks (RNNs) are essential for processing time-series data. However, they function as read-only models, lacking the ability to directly modify the data they learn from. In this study, we introduce the corrector long short-term memory (cLSTM), a Read & Write LSTM architecture that not only learns from the data but also dynamically adjusts it when necessary. The cLSTM model leverages two key components: (a) predicting LSTM’s cell states using Seasonal Autoregressive Integrated Moving Average (SARIMA) and (b) refining the training data based on discrepancies between actual and forecasted cell states. Our empirical validation demonstrates that cLSTM surpasses read-only LSTM models in forecasting accuracy across the Numenta Anomaly Benchmark (NAB) and M4 Competition datasets. Additionally, cLSTM exhibits superior performance in anomaly detection compared to hierarchical temporal memory (HTM) models.

Hierarchical Predictions of Fine-to-Coarse Time Span and Atmospheric Field Reconstruction for Typhoon Track Prediction

The prediction of typhoon tracks in the Northwest Pacific is key to reducing human casualties and property damage. Traditional numerical forecasting models often require substantial computational resources, are high-cost, and have significant limitations in prediction speed. This research is dedicated to using deep learning methods to address the shortcomings of traditional methods. Our method (AFR-SimVP) is based on a large-kernel convolutional spatio-temporal prediction network combined with multi-feature fusion for forecasting typhoon tracks in the Northwest Pacific. In order to more effectively suppress the effect of noise in the dataset to enhance the generalization ability of the model, we use a multi-branch structure, incorporate an atmospheric reconstruction subtask, and propose a second-order smoothing loss to further improve the prediction ability of the model. More importantly, we innovatively propose a multi-time-step typhoon prediction network (HTAFR-SimVP) that does not use the traditional recurrent neural network family of models at all. Instead, through fine-to-coarse hierarchical temporal feature extraction and dynamic self-distillation, multi-time-step prediction is achieved using only a single regression network. In addition, combined with atmospheric field reconstruction, the network achieves integrated prediction for multiple tasks, which greatly enhances the model’s range of applications. Experiments show that our proposed network achieves optimal performance in the 24 h typhoon track prediction task. Our regression network outperforms previous recurrent network-based typhoon prediction models in the multi-time-step prediction task and also performs well in multiple integration tasks.

Multi-scene application of intelligent inspection robot based on computer vision in power plant

As industries develop, the automation and intelligence level of power plants is constantly improving, and the application of patrol robots is also increasingly widespread. This research combines computer vision technology and particle swarm optimization algorithm to build an obstacle recognition model and obstacle avoidance model of an intelligent patrol robot in a power plant respectively. Firstly, the traditional convolutional recurrent neural network is optimized, and the obstacle recognition model of an intelligent patrol robot is built by combining the connection timing classification algorithm. Then, the artificial potential field method optimizes the traditional particle swarm optimization algorithm, and an obstacle avoidance model of an intelligent patrol robot is built. The performance of the two models was tested, and it was found that the highest precision, recall, and F1 values of the identification model were 0.978, 0.974, and 0.975. The highest precision, recall, and F1 values of the obstacle avoidance model were 0.97, 0.96, and 0.96 respectively. The two optimization models designed in this research have better performance. In conclusion, the two models in this study are superior to the traditional methods in recognition effect and obstacle avoidance efficiency, providing an effective technical scheme for intelligent patrol inspection of power plants.

Deterministic convergence analysis for regularized long short-term memory and its application to regression and multi-classification problems

Long short-term memory (LSTM) is a recurrent neural network (RNN) framework designed to solve the gradient disappearance and explosion problems of traditional RNNs. In recent years, LSTM has become a state-of-the-art model for solving various machine-learning problems. This paper propose a novel regularized LSTM based on the batch gradient method. Specifically, the L2 regularization is appended to the objective function as a systematic external force, effectively controlling the excessive growth of weights in the network and preventing the overfitting phenomenon. In addition, a rigorous convergence analysis of the proposed method is carried out, i.e., monotonicity, weak convergence, and strong convergence results are obtained. Finally, comparative simulations are conducted on the benchmark data set for regression and classification problems, and the simulation results verify the effectiveness of the method.

Sensor-Based Indoor Fire Forecasting Using Transformer Encoder.

Indoor fires may cause casualties and property damage, so it is important to develop a system that predicts fires in advance. There have been studies to predict potential fires using sensor values, and they mostly exploited machine learning models or recurrent neural networks. In this paper, we propose a stack of Transformer encoders for fire prediction using multiple sensors. Our model takes the time-series values collected from the sensors as input, and predicts the potential fire based on the sequential patterns underlying the time-series data. We compared our model with traditional machine learning models and recurrent neural networks on two datasets. For a simple dataset, we found that the machine learning models are better than ours, whereas our model gave better performance for a complex dataset. This implies that our model has a greater potential for real-world applications that probably have complex patterns and scenarios.

Early fire detection technology based on improved transformers in aircraft cargo compartments

The implementation of early and accurate detection of aircraft cargo compartment fire is of great significance to ensure flight safety. The current airborne fire detection technology mostly relies on single-parameter smoke detection using infrared light. This often results in a high false alarm rate in complex air transportation environments. The traditional deep learning model struggles to effectively address the issue of long-term dependency in multivariate fire information. This paper proposes a multi-technology collaborative fire detection method based on an improved transformers model. Dual-wavelength optical sensors, flue gas analyzers, and other equipment are used to carry out multi-technology collaborative detection methods and characterize various feature dimensions of fire to improve detection accuracy. The improved Transformer model which integrates the self-attention mechanism and position encoding mechanism is applied to the problem of long-time series modeling of fire information from a global perspective, which effectively solves the problem of gradient disappearance and gradient explosion in traditional RNN (recurrent neural network) and CNN (convolutional neural network). Two different multi-head self-attention mechanisms are used to classify and model multivariate fire information, respectively, which solves the problem of confusing time series modeling and classification modeling in dealing with multivariate classification tasks by a single attention mechanism. Finally, the output results of the two models are fused through the gate mechanism. The research results show that, compared with the traditional single-feature detection technology, the multi-technology collaborative fire detection method can better capture fire information. Compared with the traditional deep learning model, the multivariate fire prediction model constructed by the improved Transformer can better detect fires, and the accuracy rate is 0.995.

Short-term wind power forecasting through stacked and bi directional LSTM techniques.

Computational intelligence (CI) based prediction models increase the efficient and effective utilization of resources for wind prediction. However, the traditional recurrent neural networks (RNN) are difficult to train on data having long-term temporal dependencies, thus susceptible to an inherent problem of vanishing gradient. This work proposed a method based on an advanced version of RNN known as long short-term memory (LSTM) architecture, which updates recurrent weights to overcome the vanishing gradient problem. This, in turn, improves training performance. The RNN model is developed based on stack LSTM and bidirectional LSTM. The parameters like mean absolute error (MAE), standard deviation error (SDE), and root mean squared error (RMSE) are utilized as performance measures for comparison with recent state-of-the-art techniques. Results showed that the proposed technique outperformed the existing techniques in terms of RMSE and MAE against all the used wind farm datasets. Whereas, a reduction in SDE is observed for larger wind farm datasets. The proposed RNN approach performed better than the existing models despite fewer parameters. In addition, the approach requires minimum processing power to achieve compatible results.