Recurrent Neural Network Model Research Articles

Competition of systematically optimized deep neural networks for the estimation of tensile behavior of aluminum-magnesium alloy

Aluminum-magnesium (Al-Mg) alloys are prevalently employed within the aerospace sector. This research engaged a suite of deep learning approaches, encompassing the Artificial Neural Network (ANN), Gated Recurrent Unit (GRU) networks, Long-Short Term Memory (LSTM), and simple Recurrent Neural Network (RNN) to evaluate their predictive efficacy regarding the tensile strength and stiffness of Al-Mg alloys obtained from molecular dynamics simulation. The Taguchi method was initially applied to refine the architecture of each deep neural network (DNN), followed by a comparative analysis of their optimized configurations. The findings of this investigation revealed that the refined simple RNN and LSTM models exhibited superior predictive accuracy for estimating the strength and stiffness of the alloy, respectively. Moreover, the study elucidated that DNNs equipped with memory capabilities outstripped traditional ANNs in forecasting the tensile properties of Al-Mg alloys.

Nonlinear dynamical system approximation and adaptive control based on hybrid‐feed‐forward recurrent neural network: Simulation and stability analysis

AbstractWe proposed an online identification and adaptive control framework for the nonlinear dynamical systems using a novel hybrid‐feed‐forward recurrent neural network (HFRNN) model. The HFRNN is a combination of a feed‐forward neural network (FFNN) and a local recurrent neural network (LRNN). We aim to leverage the simplicity of FFNN and the effectiveness of RNN to capture changing dynamics accurately and design an indirect adaptive control scheme. To derive the weights update equations, we have applied the gradient‐descent‐based Back‐Propagation (BP) technique, and the stability of the proposed learning strategy is proven using the Lyapunov stability principles. We also compared the proposed method's results with those of the Jordan network‐based controller (JNC) and the local recurrent network‐based controller (LRNC) in the simulation examples. The results demonstrate that our approach performs satisfactorily, even in the presence of disturbance signals.

Educational Resource Material Translation System

The proposed "Educational Resource Material Translation System" represents a significant leap in bridging language barriers, particularly for individuals who lack proficiency in English. By leveraging advanced technologies such as Python, Flask, Recurrent Neural Network (RNN) and Large Language Models (LLM), the system offers a robust solution for translating educational content from English to prominent Indian regional languages like Kannada, Telugu and Hindi. Its user- friendly web interface, developed with Flask, ensures a seamless and intuitive translation experience for users of varying technical backgrounds. Beyond mere language translation, this project facilitates cross- cultural communication and enhances accessibility to educational resources, thereby empowering individuals and businesses alike. The utilization of Machine Learning further underscores the system's adaptability and potential for delivering precise and efficient translations. In essence, the "Language Translator" not only serves as a valuable tool for linguistic inclusivity but also fosters knowledge dissemination and cultural exchange on a broader scale.

A Deep Learning Model with Axial Attention for Radar Echo Extrapolation

ABSTRACT Radar echo extrapolation is an important approach in precipitation nowcasting which utilizes historical radar echo images to predict future echo images. In this paper, we introduce the self-attention mechanism into Trajectory Gated Recurrent Unit (TrajGRU) model. Under the sequence-to-sequence framework, we have developed a novel convolutional recurrent neural network called Self-attention Trajectory Gated Recurrent Unit (SA-TrajGRU), which incorporates the self-attention mechanism. The SA-TrajGRU model which combines spatiotemporal variant structure in TrajGRU and self-attention module is simple and effective. We evaluate our approach on the Moving MNIST-2 dataset and the CIKM AnalytiCup 2017 radar echo dataset. The experimental results show that the performance of the proposed SA-TrajGRU model is comparable to other convolutional recurrent neural network models. HSS and CSI scores of the SA-TrajGRU model are higher than scores of other models under the radar echo threshold of 25 dBZ, indicating that the SA-TrajGRU model has the most accurate prediction results under this threshold.

Dual-blend insight recommendation system for e-commerce recommendations and enhance personalization

E-commerce, short for electronic commerce, refers to the buying and selling of goods and services over the internet. This digital transaction model has revolutionized the way businesses operate and consumers shop. In response to the burgeoning complexity of e-commerce datasets, this work addresses the need for advanced recommendation systems. This work introduces the dual-blend insight recommendation system (DIRS) model for personalized e-commerce recommendation system. The DIRS model involves dataset loading, preprocessing, and feature extraction, enabling training with recurrent neural network (RNN) and Bayesian personalized ranking (BPR) models. Recommendations are generated based on user-defined functions, i.e., location and session, and evaluation metrics such as hit rate (HR) and mean reciprocal rate (MRR) highlight DIRS’s superior performance. The model is evaluated using the Tmall dataset. Results reveal DIRS consistently outperforms alternative algorithms, showcasing its effectiveness in 10k and 20k recommendation sets. This study provides valuable insights into optimizing e-commerce recommendations, emphasizing DIRS as a powerful model for enhancing user experience and engagement.

Prediction of emission characteristics of diesel/n-hexanol/graphene oxide blended fuels based on fast outlier detection-sparrow search algorithm-bidirectional recurrent neural network

This study aims to utilize deep learning (DL) technology to address the issue of carbon soot emission in traditional internal combustion engines. Spraying and combustion experiments were conducted in a constant volume combustion (CVC) chamber using n-hexanol and graphene oxide nanoparticles as partial substitutes for diesel, with the goal of predicting their emission characteristics based on experimental data. Initially, spray and combustion data under different operating conditions were collected using the CVC experimental setup, and outliers were removed using the fast outlier detection (FOD) algorithm. Subsequently, the learning rate and the number of hidden layer neurons in the Bidirectional Recurrent Neural Network (BiRNN) model were optimized using the Sparrow Search Algorithm (SSA) to enhance the model's accuracy and generalization ability. Unlike previous research methods, the novel FOD-coupled SSA optimization method used in this study ensures the optimal parameter configuration of the speed prediction model while accelerating the convergence speed, which ultimately improves the stability of the model. In addition, the optimal output variables were determined based on the Spearman and Pearson correlation coefficient rules. Finally, the effectiveness of the predictive model was validated using evaluation metrics such as root mean square error (RMSE) and R-squared (R2). The results indicate that the KL coefficient for D80H20GO20 fuel in the CVC device is minimal. The FOD-SSA-BiRNN model achieved R2 values of 0.99409, 0.99529, and 0.99692 for D100, D80H20, and D80H20GO20 fuels, respectively. Therefore, this study provides an effective method for the online accurate prediction of carbon soot emissions in mixed fuels.

Enhancing the exploitation of natural resources for green energy: An application of LSTM-based meta-model for aluminum prices forecasting

Efficient resource allocation for electric car production can be achieved by anticipating aluminum future pricing. For the electric car sector to maintain a steady supply chain, efficient production, and long-term growth, accurate forecasts are essential for policymakers to plan resource policies. The accuracy of ensemble learning models in predicting Aluminum prices was analyzed using Gene Expression programming and long short-term memory (LSTM) method. The analysis was based on monthly frequency spot settlement price data of Aluminum, copper, silver, and crude oil prices, as well as currency inflation rates of USA, China, and Peru. The study period spanned from January 1, 1994 to December 31, 2022, and also included the gross domestic product (GDP) of USA, China, and other metal minerals. Initially, we assess the level of multicollinearity between each chosen input parameter and Aluminum price volatility using Variance inflation factor-based multicollinearity. The primary innovation of this study lies in the creation of a highly precise model that utilizes the LSTM algorithm to estimate monthly Aluminum prices. This model serves as a revolutionary forecasting method. An extensive evaluation of various ensemble learning models reveals that XGBoost is the optimal model for accurately forecasting monthly Aluminum prices. The LSTM-based recurrent neural network model exhibited the lowest error, with a Mean Relative Error (MRE) ranging from 0.001 to 0.008, RMSE ranging from 54.527 to 136.044, and R2 ranging from 0.95 to 0.983. The findings demonstrated that the LSTM-based super learner models had superior performance compared to the CatBoost, LightBoost, Random Forest, XGBoost, and AdaBoost models, as evidenced by higher values of the determination coefficient (R2), value account for (VAF), and other 10 error analysis evaluators. Therefore, the utilization of an LSTM-based super learner model can serve as a dependable approach for forecasting future monthly Aluminum prices. In addition, this study introduces a logical mathematical model utilizing gene expression programming (GEP) to anticipate future Aluminum prices, which can be utilized by other researchers. Thus, we conclude that both the GEP and ensemble learning models, particularly the LSTM-based Super learner model, are appropriate for precise metal price predictions, which can benefit policymakers and assist in resource policy planning.

Short-term office temperature forecasting through a data-driven approach integrated with bidirectional gated recurrent neural network

Accurate indoor temperature forecasting can facilitate the energy savings, thus improving the building energy efficiency. Based on the discontinuous sensor data collection in this research, a novel gated recurrent neural network model, bidirectional Gated Recurrent Unit (Bi-GRU) is developed, achieving superior performances and optimizing the forecasting accuracy, compared to other state-of-the-art models. The developed data-driven approach based on integrating data pre-processing, feature engineering, Kalman time-series smoothing, and neural network models is applied for indoor temperature forecasting, calibrated on two test scenarios. The prediction results are examined by error performance metrics, and the relative prediction residuals are visually compared in time-series plots and quantile–quantile (Q-Q) plots. Kalman smoothing has effectively improved the forecasting accuracy of regression models by reducing their prediction errors and upgrading the prediction accuracies with approximately 4 ∼ 9 % and 6 ∼ 19 % in single-step- and multi-step-ahead forecasting tasks, respectively. Bi-GRU with Kalman smoothing is verified to be the most efficacious forecasting solution by yielding the accuracy of 94.28 % and 91.58 % in Test scenario I, and 91.58 % and 83.82 % in Test scenario II for single-step- and multi-step-ahead predictions, respectively.

EVALUATING HYBRID NEURAL NETWORK ARCHITECTURES FOR PREDICTING SLEEP DISORDERS FROM STRUCTURED DATA

The accurate diagnosis of sleep disorders is crucial for effective treatment and management, yet current methods often rely on subjective assessments and are not always reliable. This research examines the efficacy of various neural network architectures, including dense networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and innovative hybrid models, in predicting sleep disorders from structured health data. Our study focuses on comparing the performance of these models using metrics such as accuracy, precision, recall, and F1 score across a dataset comprising 400 individuals with detailed sleep and lifestyle data. Our findings demonstrate that while traditional models like dense networks and CNNs for structured data yield robust results, hybrid models, particularly the CNN-Transformer, significantly outperform others. This model effectively integrates convolutional layers with Transformer’s attention mechanisms, excelling in handling complex data interactions and providing superior predictive accuracy with an F1 score and accuracy reaching as high as 0.91. Conversely, RNN models, designed to capture temporal data dependencies, showed less efficacy, underscoring the importance of model selection aligned with data characteristics. This suggests that for datasets not exhibiting strong temporal features, models leveraging spatial relationships or advanced attention mechanisms are more suitable. This study not only advances our understanding of neural network applications in medical diagnostics but also highlights the potential of hybrid models in enhancing diagnostic accuracy. These insights could lead to significant improvements in the early detection and treatment of sleep disorders, thereby enhancing patient outcomes and contributing to the broader field of medical informatics.

A Hybrid Feature Selection Gradient Recurrent Neural Network (HFSGRNN) Model for Rainfall Prediction in India Regions

A Hybrid Feature Selection Gradient Recurrent Neural Network (HFSGRNN) Model for Rainfall Prediction in India Regions

Research on a Real-Time Prediction Method of Hull Girder Loads Based on Different Recurrent Neural Network Models

Real-time prediction of hull girder loads is of great significance for the safety of ship structures. Some scholars have used neural network technology to investigate hull girder load real-time prediction methods based on motion monitoring data. With the development of deep learning technology, a variety of recurrent neural networks have been proposed; however, there is still a lack of systematic comparative analysis on the prediction performance of different networks. In addition, the real motion monitoring data inevitably contains noise, and the effect of data noise has not been fully considered in previous studies. In this paper, four different recurrent neural network models are comparatively investigated, and the effect of different levels of noise on the prediction accuracy of various load components is systematically analyzed. It is found that the GRU network is suitable for predicting the torsional moment and horizontal bending moment, and the LSTM network is suitable for predicting the vertical bending moment. Although filtering has been applied to the original noise data, the prediction accuracy still decreased as the noise level increased. The prediction accuracy of the vertical bending moment and horizontal bending moment is higher than that of the torsional moment.

A Novel Method for Remaining Useful Life Prediction of RF Circuits Based on the Gated Recurrent Unit-Convolutional Neural Network Model.

The remaining useful life (RUL) prediction of RF circuits is an important tool for circuit reliability. Data-driven-based approaches do not require knowledge of the failure mechanism and reduce the dependence on knowledge of complex circuits, and thus can effectively realize RUL prediction. This manuscript proposes a novel RUL prediction method based on a gated recurrent unit-convolutional neural network (GRU-CNN). Firstly, the data are normalized to improve the efficiency of the algorithm; secondly, the degradation of the circuit is evaluated using the hybrid health score based on the Euclidean and Manhattan distances; then, the life cycle of the RF circuits is segmented based on the hybrid health scores; and finally, an RUL prediction is carried out for the circuits at each stage using the GRU-CNN model. The results show that the RMSE of the GRU-CNN model in the normal operation stage is only 3/5 of that of the GRU and CNN models, while the prediction uncertainty is minimized.

Analyzing Players' Tactical Positions and Movement Trajectories in Soccer Matches Using Machine Vision Algorithms

Tactical positions and movement trajectories in soccer matches using machine vision algorithms involves leveraging computer vision techniques to track players and the ball throughout the game. By processing video feeds from multiple cameras, machine vision algorithms can identify players, recognize their positions on the field, and map out their movement trajectories over time. This analysis provides valuable insights into team formations, player positioning, and tactical strategies employed during matches. Coaches and analysts can use this information to assess team performance, identify patterns in gameplay, and make data-driven decisions to optimize tactics and training regimes. Machine vision algorithms enhance the understanding of soccer dynamics, facilitating more strategic and effective approaches to coaching and gameplay.. This paper presents a comprehensive analysis of player movement, tactical positions, and detection in soccer matches using machine learning algorithms. Leveraging a simulation environment built upon a Recurrent Neural Network (RNN) model trained on historical match data, we investigate the efficacy of the model in accurately predicting future player positions based on contextual features such as player trajectories and ball movement. Furthermore, we explore the model's ability to estimate tactical movements during specific time intervals, providing valuable insights for coaches and analysts in understanding team strategies and adapting to match dynamics. Additionally, we evaluate player detection systems to assess their capabilities and limitations, laying the groundwork for future improvements in player tracking technologies. Our findings underscore the potential of machine learning in soccer analytics, offering actionable insights for enhancing performance analysis, strategic decision-making, and overall understanding of the game. Our results show a high prediction accuracy, with an average Mean Squared Error (MSE) of 0.05. Furthermore, we explore the model's ability to estimate tactical movements during specific time intervals, achieving an overall prediction accuracy of 85%. Additionally, we evaluate player detection systems to assess their capabilities and limitations, achieving a detection accuracy of 90%.

Recurrent neural network and long short-term memory models for audio copy-move forgery detection: a comprehensive study

AbstractOne of the most pressing challenges in audio forgery detection—a major topic of signal analysis and digital forensics research—is detecting copy-move forgery in audio data. Because audio data are used in numerous sectors, including security, but increasingly tampered with and manipulated, studies dedicated to detecting forgery and verifying voice data have intensified in recent years. In our study, 2189 fake audio files were produced from 2189 audio recordings on the TIMIT corpus, for a total of 4378 audio files. After the 4378 files were preprocessed to detect silent and unsilent regions in the signals, a Mel-frequency-based hybrid feature data set was obtained from the 4378 files. Next, RNN and LSTM deep learning models were applied to detect audio forgery in the data set in four experimental setups—two with RNN and two with LSTM—using the AdaGrad and AdaDelta optimizer algorithms to identify the optimum solution in the unlinear systems and minimize the loss rate. When the experimental results were compared, the accuracy rate of detecting forgery in the hybrid feature data was 76.03%, and the hybrid model, in which the features are used together, demonstrated high accuracy even with small batch sizes. This article thus reports the first-ever use of RNN and LSTM deep learning models to detect audio copy-move forgery. Moreover, because the proposed method does not require adjusting threshold values, the resulting system is more robust than other systems described in the literature.

Hybrid RNN-LSTM Networks for Enhanced Intrusion Detection in Vehicle CAN Systems

Electric vehicles (EVs) use electric motors for propulsion, relying on electric energy stored in batteries or other energy storage devices. The standard communication protocol used in EVs is the Control Area Network (CAN), a communication protocol widely used in the automotive industry for networking and communication between various components within a vehicle. CAN protocol, designed without any care about protection, as automotive systems become more connected, the vulnerability to cyber threats, including intrusion attacks. The most common intrusion attacks on EVs are Denial of Service (DoS), Fuzzy, and Impersonation Attacks. These become a significant challenge due to the imperative need for robust Intrusion Detection Systems (IDS) in CAN networks. This paper explores the application of advanced deep learning techniques, specifically Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) networks, to enhance the effectiveness of intrusion detection in the EV domain. We will use a hybrid Deep-learning model to improve the analysis. First, we will apply the RNN model, and the output will come as input for the second model, LSTM. Our proposed hybrid model achieved an accuracy of 93%. The outcomes of this research contribute to advancing cybersecurity measures in vehicular networks, ensuring the integrity and safety of connected vehicles. The applicability of RNN and LSTM techniques in the context of CAN networks demonstrates their potential to evolve as integral components of next-generation intrusion detection systems, fostering a secure and resilient automotive ecosystem.

Stock Market Prediction Using Machine Learning Techniques

The prediction of stock market prices holds significant importance within the contemporary economic landscape. Consequently, there has been a notable surge in scholarly interest directed towards exploring novel avenues for enhancing stock market prediction capabilities. Recent research endeavors have illuminated the potential predictive value inherent in various data streams, including historical stock data and user-generated content sourced from platforms such as Twitter and web news. These investigations have revealed a discernible relationship between social mood, as reflected in online discourse, and future stock price movements. However, prior studies have often overlooked the incorporation of such sentiment-derived information, thus presenting an information gap. In the present study, we address this gap by proposing an effective methodology for the integration and analysis of multiple information sources to facilitate more accurate stock price predictions. Leveraging Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) models, we conduct a comprehensive analysis of disparate data sources. Furthermore, we employ an ensemble approach, incorporating Weighted Average and Differential Evolution techniques, to refine the predictive accuracy of our models. Our findings demonstrate the efficacy of the proposed methodology in generating highly accurate stock price predictions across varying future time horizons, including one-day, seven-day, 15-day, and 30-day intervals. These predictions offer valuable insights for investors seeking to make informed decisions regarding their investment strategies and enable companies to gauge their anticipated performance within the stock market landscape.

WIFI based human activity recognition using multi-head adaptive attention mechanism

Human Activity Recognition (HAR) utilizing Channel State Information (CSI) extracted from WiFi signals has garnered substantial interest across various domains and applications. This field’s potential paths and applications extend beyond CSI-based HAR and include smart homes, assisted living, security, gaming, surveillance, and context-aware computing. The ability of deep learning algorithms to effectively process and interpret CSI data opens up new possibilities for accurate and robust human activity recognition in real-world scenarios. However, traditional Recurrent Neural Networks (RNN) models, such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), rely solely on their internal memory cells to maintain information over time. Important details might be diluted or lost within the memory cells in complex CSI sequences. To address this limitation, we propose a lightweight approach that incorporates a multi-head adaptive attention weight mechanism MHAAM into the HAR framework. The multi-head attention mechanism allows the model to attend to different informative patterns within the CSI data simultaneously, capturing fine-grained temporal dependencies and improving the model’s ability to recognize complex activities. The implemented models effectively filter out noise and irrelevant information by assigning higher weights to informative CSI features, further enhancing activity classification accuracy. Experimental evaluations and comparative analyses of HAR for seven activities demonstrate that attention-based RNN models with multi-head attention consistently outperform traditional RNN models. The multi-head attention mechanism achieves improved generalization and testing for seven common human activities and environments, leading to a higher complex human activity classification accuracy of up to 98.5%.

Recurrent neural network plasticity models: Unveiling their common core through multi-task learning

Recurrent neural network models are known to be particularly suitable for data-driven constitutive modeling due to their built-in memory variables. The main challenge preventing their widespread application to engineering materials lies in their excessive need of data for training. Here, we postulate that RNN models of elasto-plastic solids feature a large common core that is shared by all materials of the same class. The common core is complemented by material-specific layers with parameters that vary from material-to-material. After training RNN models for 25 different von Mises materials, we identify the common core of a multi-task model. Furthermore, we demonstrate through ensemble transfer learning that adding a new material to the multi-task model requires datasets that are two to three orders of magnitude smaller than the datasets needed for training an RNN from scratch. In addition, to multi-task learning, we introduce probabilistic ensembles of RNN plasticity models to quantify the epistemic uncertainty. A deep drawing simulation is performed to demonstrate the superior generalization capabilities of RNNs identified via multi-task learning as compared to those obtained through single task training.

An Evaluation of Bidirectional Long Short-Term Memory Model for Estimating Monthly Rainfall in India

Objectives: Predicting the amount of rainfall is difficult due to its complexity and non-linearity. The objective of this study is to predict the average rainfall one month ahead using the all-India monthly average rainfall dataset from 1871 to 2016. Methods: This study proposed a Bidirectional Long Short-Term Memory (LSTM) model to predict the average monthly rainfall in India. The parameters of the models are determined using the grid search method. This study utilized the average monthly rainfall as an input, and the dataset consists of 1752 months of rainfall data prepared from thirty (30) meteorological sub-divisions in India. The model was compiled using the Mean Square Error (MSE) loss function and Adam optimizer. The models' performances were evaluated using statistical metrics such as Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). Findings: This study discovered that the proposed Bidirectional LSTM model achieved an RMSE of 240.79 and outperformed an existing Recurrent Neural Network (RNN), Vanilla LSTM and Stacked LSTM by 8%, 4% and 2% respectively. The study also finds that increasing the input time step and increasing the number of cells in the hidden layer enhanced the prediction performance of the proposed model, and the Bidirectional LSTM converges at a lower epoch compared to RNN and LSTM models. Novelty: This study applied the Bidirectional LSTM for the first time in predicting all-India monthly average rainfall and provides a new benchmark for this dataset. Keywords: Deep Learning, LSTM, Rainfall prediction, Stacked LSTM, Bidirectional LSTM

Using an interpretable deep learning model for the prediction of riverine suspended sediment load.

The prediction of suspended sediment load (SSL) within riverine systems is critical to understanding the watershed's hydrology. Therefore, the novelty of our research is developing an interpretable (explainable) model based on deep learning (DL) and Shapley Additive ExPlanations (SHAP) interpretation technique for prediction of SSL in the riverine systems. This paper investigates the abilities of four DL models, including dense deep neural networks (DDNN), long short-term memory (LSTM), gated recurrent unit (GRU), and simple recurrent neural network (RNN) models for the prediction of daily SSL using river discharge and rainfall data at a daily time scale in the Taleghan River watershed, northwestern Tehran, Iran. The performance of models was evaluated by using several quantitative and graphical criteria. The effect of parameter settings on the performance of deep models on SSL prediction was also investigated. The optimal optimization algorithms, maximum iteration (MI), and batch size (BC) were obtained for modeling daily SSL, and structure of the model impact on prediction remarkably. The comparison of prediction accuracy of the models illustrated that DDNN (with R2 = 0.96, RMSE = 333.46) outperformed LSTM (R2 = 0.75, RMSE = 786.20), GRU (R2 = 0.73, RMSE = 825.67), and simple RNN (R2 = 0.78, RMSE = 741.45). Furthermore, the Taylor diagram confirmed that DDNN has the highest performance among other models. Interpretation techniques can address the black-box nature of models, and here, SHAP was applied to develop an interpretable DL model to interpret of DL model's output. The results of SHAP showed that river discharge has the strongest impact on the model's output in estimating SSL. Overall, we conclude that DL models have great potential in watersheds to predict SSL. Therefore, different interpretation techniques as tools to interpret DL model's output (DL model is as black-box model) are recommended in future research.