Gated Recurrent Unit Neural Network Research Articles

Using an interpretable deep learning model for the prediction of riverine suspended sediment load.

The prediction of suspended sediment load (SSL) within riverine systems is critical to understanding the watershed's hydrology. Therefore, the novelty of our research is developing an interpretable (explainable) model based on deep learning (DL) and Shapley Additive ExPlanations (SHAP) interpretation technique for prediction of SSL in the riverine systems. This paper investigates the abilities of four DL models, including dense deep neural networks (DDNN), long short-term memory (LSTM), gated recurrent unit (GRU), and simple recurrent neural network (RNN) models for the prediction of daily SSL using river discharge and rainfall data at a daily time scale in the Taleghan River watershed, northwestern Tehran, Iran. The performance of models was evaluated by using several quantitative and graphical criteria. The effect of parameter settings on the performance of deep models on SSL prediction was also investigated. The optimal optimization algorithms, maximum iteration (MI), and batch size (BC) were obtained for modeling daily SSL, and structure of the model impact on prediction remarkably. The comparison of prediction accuracy of the models illustrated that DDNN (with R2 = 0.96, RMSE = 333.46) outperformed LSTM (R2 = 0.75, RMSE = 786.20), GRU (R2 = 0.73, RMSE = 825.67), and simple RNN (R2 = 0.78, RMSE = 741.45). Furthermore, the Taylor diagram confirmed that DDNN has the highest performance among other models. Interpretation techniques can address the black-box nature of models, and here, SHAP was applied to develop an interpretable DL model to interpret of DL model's output. The results of SHAP showed that river discharge has the strongest impact on the model's output in estimating SSL. Overall, we conclude that DL models have great potential in watersheds to predict SSL. Therefore, different interpretation techniques as tools to interpret DL model's output (DL model is as black-box model) are recommended in future research.

A Health Management Technology Based on PHM for Diagnosis, Prediction of Machine Tool Servo System Failures

The computer numerically controlled (CNC) system is the key functional component of CNC machine tool control systems, and the servo drive system is an important part of CNC systems. The complex working environment will lead to frequent failure of servo drive systems. Taking effective health management measures is the key to ensure the normal operation of CNC machine tools. In this paper, the comprehensive effect of fault prediction and fault diagnosis is considered for the first time, and a health management system for machine tool servo drive systems is proposed and applied to operation and maintenance management. According to the data collected by the system and related indicators, the technology can predict the state trend of equipment operation, identify the hidden fault characteristics in the data, and further diagnose the fault types. A health management system mainly includes fault prediction and fault diagnosis. The core of fault prediction is the gated recurrent unit (GRU). The attention mechanism is introduced into a GRU neural network, which can solve the long-term dependence problem and improve the model performance. At the same time, the Nadam optimizer is used to update the model parameters, which improves the convergence speed and generalization ability of the model and makes it suitable for solving the prediction problem of large-scale data. The core of fault diagnosis is the self-organizing mapping (SOM) neural network, which performs cluster analysis on data with different characteristics, to realize fault diagnosis. In addition, feature standardization and principal component analysis (PCA) are introduced to balance the influence of different feature scales, enhance the feature of fault data, and achieve data dimensionality reduction. Compared with the other two algorithms and their improved versions, the superiority of the health management system with high-dimensional data and the enhancement effect of fault identification are verified. The relative relationship between fault prediction and diagnosis is further revealed, and the adjustment idea of the production plan is provided for decision makers. The rationality and effectiveness of the system in practical application are verified by a series of tests of fault data sets.

Effective Forecasting of Key Features in Hospital Emergency Department

Efficiently forecasting the demands within a hospital’s Emergency Department (ED) is critical for optimal resource allocation and patient care management. This study focuses on leveraging deep learning techniques to predict various types of ED patient flows, facilitating informed decision-making by ED managers. The rising success of deep learning networks in modeling timeseries data makes them a compelling choice for patient flow forecasting. In this context, we investigate and compare seven deep learning models-Deep Belief Network (DBN), Restricted Boltzmann Machines (RBM), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), combined GRU and Convolutional Neural Networks (CNN-GRU), LSTM-CNN, and Generative Adversarial Network based on Recurrent Neural Networks (GAN-RNN)—to accurately forecast patient flow within a hospital’s emergency department. To enable traffic flow forecasting, a forecaster layer is introduced for each model. Real-world patient flow data spanning different ED services (biology, radiology, scanner, and echography) at Lille regional hospital in France serve as a case study to evaluate these models. Four effectiveness metrics are employed to assess and compare the forecasting methods. The outcomes demonstrate the superior performance of deep learning models in predicting ED patient flows compared to conventional shallow approaches like ridge regression and support vector regression. Significantly, the Deep Belief Network (DBN) stands out, achieving an averaged mean absolute percentage error of approximately 4.097.

A Data-Driven Model for Predictive Modeling of Vortex-Induced Vibrations of a Long-Span Bridge

Vortex-induced vibration (VIV) of long-span bridges can be of large amplitude, which can influence serviceability. Therefore, it is important to predict the response of vortex-induced vibration to aid the management of long-span bridges. A novel data-driven model is proposed to predict the time history of the dynamic response of VIV events. Specifically, the proposed model consists of gated recurrent unit (GRU) neural networks and the Newmark-beta method. GRU neural networks can perform accurate sequential prediction, and the Newmark-beta method can complement the physical meaning of the middle output of the proposed model. To aid the accurate prediction of the amplitude of VIV events, the proposed model employs weighted mean square error as the loss function, which can put more emphasis on the amplitude. The proposed model is validated on measured VIV events of a long-span suspension bridge. The weighted mean absolute percentage error and Pearson correlation coefficient of the trained model indicate the effectiveness of the proposed model.

A GRU network framework towards fault-tolerant control for flight vehicles based on a gain-scheduled approach

This paper introduces a novel approach to enhance the fault-tolerant control of flight vehicles by incorporating a gated recurrent unit (GRU) neural network within a gain-scheduled framework. Gain scheduling, a well-established technique for achieving active fault-tolerant control, enables the selection of control gains from a predefined set based on specific faults. An improved terminal sliding mode controller is derived, and appropriate parameters are chosen to formulate a gain-scheduled fault-tolerant controller. Based on it, training data is generated by comprehensive simulations on a Winged-Cone configuration flight vehicle. The GRU neural network architecture is designed and trained to function as the flight controller. The effectiveness of the proposed GRU network controller is demonstrated through a series of simulations.

Improved SSA-Based GRU Neural Network for BDS-3 Satellite Clock Bias Forecasting.

Satellite clock error is a key factor affecting the positioning accuracy of a global navigation satellite system (GNSS). In this paper, we use a gated recurrent unit (GRU) neural network to construct a satellite clock bias forecasting model for the BDS-3 navigation system. In order to further improve the prediction accuracy and stability of the GRU, this paper proposes a satellite clock bias forecasting model, termed ITSSA-GRU, which combines the improved sparrow search algorithm (SSA) and the GRU, avoiding the problems of GRU's sensitivity to hyperparameters and its tendency to fall into local optimal solutions. The model improves the initialization population phase of the SSA by introducing iterative chaotic mapping and adopts an iterative update strategy based on t-step optimization to enhance the optimization ability of the SSA. Five models, namely, ITSSA-GRU, SSA-GRU, GRU, LSTM, and GM(1,1), are used to forecast the satellite clock bias data in three different types of orbits of the BDS-3 system: MEO, IGSO, and GEO. The experimental results show that, as compared with the other four models, the ITSSA-GRU model has a stronger generalization ability and forecasting effect in the clock bias forecasting of all three types of satellites. Therefore, the ITSSA-GRU model can provide a new means of improving the accuracy of navigation satellite clock bias forecasting to meet the needs of high-precision positioning.

Real-time prediction of ROP based on GRU-Informer

Accurate ROP (rate of penetration) prediction contributes to better production task planning, ensuring efficient production line operation, and reducing production costs. ROP prediction is influenced by multiple factors, making accurate prediction challenging. Current research primarily relies on historical data for training and modeling, lacking methods for real-time ROP prediction. This paper introduces a GRU-Informer model for real-time ROP prediction. The model employs GRU (Gated Recurrent Unit) neural networks at the lower level to capture short-term correlations in drilling parameters and uses the Informer model at the top to address long-term dependencies among drilling parameters. Thus, the GRU-Informer can capture both short-term and long-term time dependencies, providing better ROP predictions. This paper constructs a dataset using historical data from a southwestern Chinese oil field for experimentation. RMSE (Root Mean Square Error), MAE (mean absolute error) and R2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${R}^{2}$$\\end{document} (Coefficient of Determination) are employed as evaluation metrics for the model. Experimental results demonstrate that the GRU-Informer outperforms traditional recurrent neural networks like LSTM (Long Short-Term Memory), GRU neural networks and Informer in real-time ROP prediction, indicating its practical value.

Efficient Extraction and Automated Thyroid Prediction with an Optimized Gated Recurrent Unit in Recurrent Neural Networks

Computer-aided tools are becoming increasingly important in medical diagnostics. This paper introduces the Efficient Feature Extraction Based Recurrent Neural Network (FERNN) for computer-aided thyroid disease prediction. The FERNN model uses a Gated Recurrent Unit Recurrent Neural Network (GRU-RNN) optimized with the COOT Optimization Algorithm.The study begins by gathering data from an open-source system and preprocessing it using min-max normalization to address missing values. The preprocessed data undergoes a two-level feature extraction (TLFE) procedure. In the first level, a ranked filter feature set technique is used to prioritize features based on medical expert recommendations. In the second level, a variety of metrics, including information gain, gain ratio, chi-square, and relief, are used to rank and select features. A composite measure guided by fuzzy logic is then used to select a judicious subset of features. The FERNN model uses the GRU-RNN to classify thyroid diseases in the databases. To optimise, the COOT optimization method is employed. The model's weights. The FERNN model was put into practise in MATLAB and assessed with a variety of statistical metrics, including kappa, accuracy, precision, recall, sensitivity, specificity, and the F-measure. The proposed methodology was benchmarked against traditional techniques, including the deep belief neural network (DBN), artificial neural network (ANN), and support vector machine (SVM).

A novel state of health prediction method for battery system in real-world vehicles based on gated recurrent unit neural networks

State of health (SOH) is crucial to battery management system. However, SOH accuracy and reliability is difficult to be guaranteed owing to complex ageing mechanisms and driving conditions. This paper proposes a novel battery SOH prediction method based on gated recurrent unit (GRU) neural network for real-world vehicles. Firstly, real-vehicle operating data is extracted as well as cleaned and sliced to improve data reliability, then the Kalman filter and recursive least squares method are used to identify the ohmic internal resistance (OIR). The GRU neural network is established to realize the battery SOH prediction, and the average relative error of final validation set is less than 0.65 %. To further validate the reliability and applicability of the prediction method, four other vehicles data are set as input, and the SOH results indicate that all average relative errors are still within 4 %. This study demonstrates the feasibility of using OIR as a health factor for real-world vehicles, and also provides a new solution for accurately predicting battery SOH for real-world vehicles.

Brain tumor classification and detection via hybrid alexnet-gru based on deep learning

Brain tumors are among the most dangerous types of brain cancer due to their aggressiveness. The development of aberrant brain tissue leads to brain tumors, which pose a major threat to people's health and welfare. Early detection of these malignancies is still fairly challenging, particularly when attempting to distinguish between various kinds including gliomas, meningioma, and pituitary tumors. The complexity of the brain's structure further increases the need for a speedy and accurate identification of irregularities. An innovative approach using a hybrid model that combines AlexNet and GRU (Gated Recurrent Unit) neural networks is presented to identify and characterize brain cancers using MRI data. The MRI images should first be sharpened and denoised using a non-local means filter to ensure the best input data. To avoid overfitting and achieve optimal model performance, the AlexNet architecture employs layers to extract specificfeatures from images. Model complexity and potential for generalization are regulated through hyperparameter modification. The GRU component resolves gradient vanishing in deep networks and uses the softmax activation function to categorize braintumorsinto four distinct classes. According to the findings, the model can categorize and diagnose brain cancers with 97% accuracy, 97.63% precision, a robust 96.78% recall rate, and an impressive 97.25% F1-Score. These findings show that theresearch has the potential to improve patient outcomes and create a more favorable healthcare environment in medical imaging and brain tumor detection.

Research on the prediction of LOCA condition in nuclear power plants based on GRU recurrent neural network and its variants

AbstractConsidering the development of deep learning and the emergence of intelligent control demands in nuclear reactors, along with the presence of plant‐level real‐time information monitoring systems in most nuclear power plants, there is a considerable accumulation of sensor measurements from long‐term operation. This makes it feasible to conduct medium to long‐term predictions for various real‐time conditions in nuclear power plants. Therefore, this paper proposes the utilization of a gate‐based recurrent neural network called GRU (Gated Recurrent Unit) and its variants for parameter prediction of LOCA (Loss of Coolant Accident) scenarios. The main content of this paper consists of two parts: (1) Experimental verification is conducted to demonstrate that GRU has excellent capability in capturing long‐term sequential information and generalization ability, making it suitable for predicting accident conditions in nuclear power plants. Two accident trend prediction methods based on the GRU network are proposed for scenarios with limited data. The results show that these methods can effectively provide short‐term development trends for accident conditions. Additionally, by considering the feature extraction capacity of CNN, the fusion of CNN and GRU models is employed for parameter prediction under different sizes of broken area. The results indicate an improvement in the model's generalization ability. (2) In scenarios with limited and incomplete data, a more robust variant of GRU called GRU‐D model is utilized for both univariate and multivariate synchronous prediction of accident conditions with different missing values. Experimental results demonstrate that even with a data missing rate of 90%, the GRU‐D network exhibits excellent predictive accuracy and generalization ability in parameter prediction for the given conditions.

Prediction and Impact Analysis of Passenger Flow in Urban Rail Transit in the Postpandemic Era

In the postpandemic era, exploring the relationship between the daily new COVID-19 cases and passenger flow in urban rail transit can help effectively predict the impact of future pandemic situations on rail transit. In this study, based on a gated recurrent unit (GRU) neural network model, the daily passenger flow in urban rail transit in the postpandemic era was predicted, and the results were compared with those obtained using the long short-term memory (LSTM) neural network and other conventional time series analysis models such as SARIMA (seasonal autoregressive integrated moving average). Based on the trained GRU model, a partial dependence plot (PDP) was adopted to explore the quantitative relationship between the daily passenger flow and the daily new cases or weather attribute. The results showed that (1) the prediction accuracy of the GRU neural network model was 95.25%, which was the highest among the prediction models studied, indicating that the GRU could achieve the best performance. (2) The GRU model did not fluctuate significantly in the initial training stage, and its convergence rate was higher than that of the LSTM. (3) The number of daily new cases was negatively correlated with the daily passenger flow. For every new case on the previous day, the daily passenger flow fell by an average of 54,600 person-times. (4) Compared with no rain condition, the daily passenger flow decreased by 207,600 person-times on an average on rainy days. In summary, the neural network could achieve accurate prediction, while the PDP could compensate for the “black box” disadvantage of nonparametric models, owing to which the quantitative relationship between the number of new cases and daily passenger flow could be successfully explored. Our study can serve as a basis for demand prediction, operational organization, and policy implementation related to urban rail transit.

A Thermal Displacement Prediction System With an Automatic LRGTVAC-PSO Optimized Branch Structured Bidirectional GRU Neural Network

Considering technology’s rapid development, traditional manufacturing methods are insufficient to achieve the high accuracy demanded by aerospace, national defense, and numerous leading-edge engineering projects. Thermal displacement is a significant source of manufacturing errors, and accurately correcting such errors is difficult or even impossible using traditional machining methods. This paper proposes a machine learning method for high-accuracy error prediction that nonprofessionals can easily implement. An optimized automatic Logistic Random Generator Time Varying Acceleration Coefficient Particle Swarm Optimization (LRGTVAC-PSO) method is proposed to optimize a branch structured bidirectional Gated Recurrent Unit (GRU) neural network. The accuracy of the proposed method (with a three-axis average of 0.945) is superior to that of the other optimized algorithms evaluated in this study. The method serves as a means not only of accurately predicting thermal displacement but also of autotuning the hyperparameters of machine learning algorithms.

GRU Neural Network Based on CEEMDAN–Wavelet for Stock Price Prediction

Stock indices are considered to be an important indicator of financial market volatility in various countries. Therefore, the stock market forecast is one of the challenging issues to decrease the uncertainty of the future direction of financial markets. In recent years, many scholars attempted to use different conventional statistical and deep learning methods to predict stock indices. However, the non-linear financial noise data will usually cause stochastic deterioration and time lag in forecast results, resulting in existing neural networks that do not demonstrate good prediction results. For this reason, we propose a novel framework to combine the gated recurrent unit (GRU) neural network with the complete ensemble empirical mode decomposition of adaptive noise (CEEMDAN) to predict the stock indices with better accuracy, in which the wavelet threshold method is especially used to denoise high-frequency noises in the sub-signals to exclude noise interference for future data predictions. Firstly, we choose representative datasets collected from the closing prices of S&P500 and CSI 300 stock indices to evaluate the proposed GRU-CEEMDAN–wavelet model. Additionally, we compare the improved model to the traditional ARIMA and several modified neural network models using different gate structures. The result shows that the mean values of MSE and MAE for GRU based on CEEMDAN–wavelet are the smallest by significance analysis. Overall, we found that our model could improve prediction accuracy and alleviates the time lag problem.

Short-term wind power prediction model based on improved ACA-GRU neural network

To improve the accuracy of wind power forecasting, the improved ACA (Ant Colony Algorithm) is used to optimize the GRU (Gated Recurrent Unit) model. First, the original power generation data is normalized; Second, the GRU neural network model is established, and the ant colony algorithm is used to optimize it; Finally, the optimized GRU model and the non-optimized model are used to predict the short-term wind power output, and the prediction results are compared to verify that the improved ACA-GRU prediction model has higher prediction accuracy for short-term wind power output.

A hybrid CNN-GRU based probabilistic model for load forecasting from individual household to commercial building

In load forecasting, build load is relatively difficult to predict due to its variability and uncertainty. This study intends to develop a hybrid deep learning models and quantile regression loss function for building load prediction. Then, through the numerical study for the individual and commercial building, it is provided to demonstrate the practicality of proposed method which showed the accuracy with lower loss than existing models. In academic contribution, this study proves that Convolutional Neural Network (CNN) can extract useful information from high uncertainty power load, and Gated Recurrent Unit (GRU) has the benefit in the times-series forecasting. Furthermore, the proposed hybrid framework outperforms the conventional LSTM and GRU neural network. In practical contribution, the issues of reducing energy waste and demand side management can be improved through more accurate load forecasting. Also, RE100 company can realize the risk of the power purchasing planning and customized decision-making through prediction intervals.

Short-Term Electricity Price Forecasting Using EEMD and GRU-NN

A vital necessity for the reliable operation of the power system as well as for the financial well-being of the consumers is an accurate forecast of the price of electricity. The price of electricity is highly volatile, nonlinear, and subject to seasonal fluctuations. The price series is also affected by electrical market uncertainty and demand, which is strongly reliant on weather and electricity usage time. This study presents a short-term electricity price forecasting method that follows the similar days approach comprising Ensemble Empirical Mode Decomposition technique. The suggested method utilizes a feature selection methodology that combines the Random Forest Regressor and the Gradient Boosting Regressor to determine which features are most important for the machine learning model. To forecast the electricity price, a Gated Recurrent Unit Neural Network (GRU-NN) is employed as the machine learning model. The GRU-NN has the capability of accurately capturing complicated temporal relationships in electricity price time series, which enables it to make correct predictions. To evaluate the validity of the proposed method, data from the PJM electricity market have been used. The simulation results demonstrate that the suggested method is superior to the existing technique, with significantly improved values achieved for both the mean absolute percentage error (MAPE) and the root mean square error (RMSE).

Simulated annealing algorithm optimized GRU neural network for urban rainfall-inundation prediction

Abstract In the context of global climate change and the continuous development of urban areas, rainfall-inundation modeling is a common approach that provides critical support for the protection and early warning of urban waterlogging protection. The present study conducts a data-driven model for hourly urban rainfall-inundation depth prediction, which is based on a gated recurrent unit (GRU) neural network and uses the simulated annealing (SA) algorithm for the hyperparameter optimization of GRU, namely the SA-GRU model. To verify the performance of the proposed model, backpropagation, long short-term memory (LSTM), and bidirectional LSTM (BiLSTM) neural networks are set as benchmarks. Results show that the SA-GRU has high accuracy in the case of short-term inundation prediction, with the Nash–Sutcliffe efficiency from 0.999 to 0.596 for the 1-h-ahead to 8-h-ahead predictions. And further research reveals that the SA-GRU integrates the significant optimization of SA, with an average 20% reduction of the root mean square error within the first eight prediction periods, and the efficient training speed of GRU, with 23.7% faster than LSTM and 44.2% faster than BiLSTM. In conclusion, the SA-GRU excels in urban inundation prediction, demonstrating its value in flood management and decision-making.

Using Time-Series Generative Adversarial Networks to Synthesize Sensing Data for Pest Incidence Forecasting on Sustainable Agriculture

A sufficient amount of data is crucial for high-performance and accurate trend prediction. However, it is difficult and time-consuming to collect agricultural data over long periods of time; the consequence of such difficulty is datasets that are characterized by missing data. In this study we use a time-series generative adversarial network (TimeGAN) to synthesize multivariate agricultural sensing data and train RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), and GRU (Gated Recurrent Unit) neural network prediction models on the original and generated data to predict future pest populations. After our experiment, the data generated using TimeGAN and the original data have the smallest EC value in the GRU model, which is 9.86. The results show that the generative model effectively synthesizes multivariate agricultural sensing data and can be used to make up for the lack of actual data. The pest prediction model trained on synthetic data using time-series data generation yields results that are similar to that of the model trained on actual data. Accurate prediction of pest populations would represent a breakthrough in allowing for accurate and timely pest control.

Differential Settlement of Track Foundations Identification Based on GRU Neural Network

The timely identification of differential settlement of track foundations is of great significance for the safety of train operation and the maintenance of track structures. However, traditional monitoring techniques cannot meet the requirements of efficient, real-time, and automatic monitoring of track foundation settlement. In order to solve these problems, a real-time identification method based on a gated recurrent unit (GRU) neural network is proposed for the differential settlement of track foundations monitoring. According to parameter sensitivity analysis, the vertical acceleration of the vehicle is selected as the known data fed into the GRU network for differential settlement identification. Then the GRU network is employed to establish the nonlinear relationship between the vertical acceleration of the vehicle and the differential settlement of the track foundation. The results indicate that the longitudinal continuous differential settlement distribution curve of track foundations could be accurately identified with GRU neural network through the real-time vibration response of the vehicle–track. The current method may provide a new means for the real-time and efficient identification of the differential settlement of track foundations.