Recurrent Neural Network Prediction Research Articles

Remaining useful lifetime prediction for milling blades using a fused data prediction model (FDPM)

AbstractIn various industry sectors, predicting the real-life availability of milling applications poses a significant challenge. This challenge arises from the need to prevent inefficient blade resource utilization and the risk of machine breakdowns due to natural wear. To ensure timely and accurate adjustments to milling processes based on the machine's cutting blade condition without disrupting ongoing production, we introduce the Fused Data Prediction Model (FDPM), a novel temporal hybrid prediction model. The FDPM combines the static and dynamic features of the machines to generate simulated outputs, including average cutting force, material removal rate, and peripheral milling machine torque. These outputs are correlated with real blade wear measurements, creating a simulation model that provides insights into predicting the wear progression in the machine when associated with real machine operational parameters. The FDPM also considers data preprocessing, reducing the dimensional space to an advanced recurrent neural network prediction algorithm for forecasting blade wear levels in milling. The validation of the physics-based simulation model indicates the highest fidelity in replicating wear progression with the average cutting force variable, demonstrating an average relative error of 2.38% when compared to the measured mean of rake wear during the milling cycle. These findings illustrate the effectiveness of the FDPM approach, showcasing an impressive prediction accuracy exceeding 93% when the model is trained with only 50% of the available data. These results highlight the potential of the FDPM model as a robust and versatile method for assessing wear levels in milling operations precisely, without disrupting ongoing production.

Data-driven modeling for residual velocity of projectile penetrating reinforced concrete slabs

The penetration of projectiles into reinforced concrete (RC) structures is crucial for defense technology. However, accurately estimating the resistance of RC structures to projectile penetration is challenging due to complex parameter nonlinearities. This paper proposes a data-driven modeling methodology that combines validated finite element (FE) simulations with machine learning (ML) algorithms to investigate the penetration resistance of RC slabs against projectiles. The methodology employs an extreme learning machine (ELM) network to identify extreme conditions characterizing penetration resistance, reducing data requirements, and improving prediction efficiency. Predicted results from the ELM network are then used as input for a recurrent neural network (RNN) to predict residual velocity curves over time, a crucial metric for evaluating penetration resistance. To cover various loading scenarios, 64 sets of FE models with different incident velocities, caliber-radius-head (CRH), and projectile masses are developed based on available experimental studies. The Johnson-Holmquist II model and Johnson-Cook model are employed for concrete and steel, respectively, to describe material behavior at high strain rates. The FE prediction results form a training sample database, and ML models are established using CRH, incident velocity, and projectile mass as input layers and projectile residual velocity as the output layer. The results demonstrate high Pearson correlation coefficients of 0.984 and 0.991 for the ELM and RNN prediction results, respectively, with numerical simulation results. The proposed networks provide accurate and efficient alternatives to time-consuming FE simulations for predicting projectile penetration performance. This strategy offers a new ML-based method for rapid evaluation, particularly for in-situ applications.

Machine Learning-Based Lane-Changing Behavior Recognition and Information Credibility Discrimination

Intelligent Vehicle–Infrastructure Collaboration Systems (i-VICS) put forward higher requirements for the real-time security of dynamic traffic information interaction. It is difficult to ensure the safety of dynamic traffic information interaction by means of traditional static information security. In this study, a method was proposed through machine learning-based lane-changing (LC) behavior recognition and information credibility discrimination, based on the utilization and exploitation of traffic business characteristics. The method consisted of three stages: LC behavior recognition based on Support Vector Machine (SVM), LC speed prediction based on Recurrent Neural Network (RNN), and credibility discrimination of speed information under LC states. Firstly, the labeling rules of vehicle LC behavior and the input/output of each stage model were determined, and the raw NGSIM data were processed to obtain data sets for LC behavior identification and LC speed prediction. Both the SVM classification and RNN prediction models were trained and tested, respectively. Afterwards, a model of credibility discrimination speed information under an LC state was constructed, and the real vehicle speed data were processed for model verification. The results showed that the overall accuracy of vehicle status recognition by the SVM model was 99.18%, and the precision of the RNN model was on the order of magnitude of cm/s. Considering transverse and longitudinal abnormal velocity, the accuracy credibility discrimination of LC velocity was more than 97% in most experimental groups. The model can effectively identify the abnormal speed data of LC vehicles and provide support for the real-time identification of LC vehicle speed information under i-VICS.

Developing an optimized recurrent neural network model for air quality prediction using K-means clustering and PCA dimension reduction

Prediction is a means of forecasting a future value by using and analyzing historical or current data. A popular neural network architecture used as a prediction model is the Recurrent Neural Network (RNN) because of its wide application and very high generalization performance. This study aims to improve the RNN prediction model’s accuracy using k-means grouping and PCA dimension reduction methods by comparing the five distance functions. Data were processed using Python software and the results obtained from the PCA calculation yielded three new variables or principal components out of the five examined. This study used an optimized RNN prediction model with k-means clustering by comparing the Euclidean, Manhattan, Canberra, Average, and Chebyshev distance functions as a measure of data grouping similarity to avoid being trapped in the local optimal solution. In addition, PCA dimension reduction was also used in facilitating multivariate data analysis. The k-means grouping showed that the most optimal distance is the average function producing a DBI value of 0.60855 and converging at the 9th iteration. The RNN prediction model results evaluated based on the number of RMSE errors which was 0.83, while that of MAPE was 8.62%. Therefore, it was concluded that the K-means and PCA methods generated a more optimal prediction model for the RNN method.

Comparative Analysis of the Applicability of Feed-Forward and Recurrent Neural Networks for Prediction of Surface Quality of Laser Polished Surfaces

Comparative Analysis of the Applicability of Feed-Forward and Recurrent Neural Networks for Prediction of Surface Quality of Laser Polished Surfaces

Prediction of New Media Information Dissemination Speed and Scale Effect Based on Large-Scale Graph Neural Network

In recent years, because of the popularity of the internet and mobile devices, the dissemination of new media in social networks has attracted extensive attention from scholars and the industry. Scale prediction or propagation speed prediction is to use the initial data to predict the propagation scale of the network. In the complex and changeable social network, how to accurately predict the cascading scale of new media information is the biggest problem at present. In the process of new media information transmission, because of the role of new media information transmission in guiding public opinion, the current hierarchical model of new media information transmission lacks the overall and local models. To solve this problem, a global structure modeling method is proposed. In addition, because of the uncertainty of new media information dissemination, a method of bidirectional recurrent neural network prediction and algorithm complexity is used, and a new method based on large-scale graph neural network is constructed. A prediction method of new media information dissemination speed and scale based on large-scale graph neural network. Through comparative experiments with previous research models, it is found that the NWIDF model constructed in this paper has a good prediction effect.

An innovative partition method for predicting shallow landslides by combining the slope stability analysis with a dynamic neural network model

The objective of this study is to explore the relevance between the rainfall pattern, slope stability of the soil layer, and the occurrence of shallow landslides. A seepage flow model for subsurface flow simulation with a slope stability analysis approach was established to examine the temporal and spatial variation of unstable grids. To define the threshold conditions to trigger shallow landslides for different regions in the watershed, the spatial distribution of the safety factor at the initial state was derived and adopted as the reference basis to classify all grids of the watershed into multiple zones. The first novelty of this study is to apply such a partition, depending on the watershed topography and soil characteristics, to train the landslide prediction model separately. The second novelty is to execute a dynamic recurrent neural network (RNN) model to determine the possible duration and the start time of shallow landslides for each zone by considering the rainfall condition as well as the cumulative area of unstable grids, which can be obtained by performing the seepage flow model. In this way, the physical significance of the RNN prediction model can be reinforced. The analysis results showed that the proposed methodology could effectively track the respective period of occurring shallow landslides for each zone, additionally, only some specific zones in the watershed were necessary to be investigated because they were prone to cause a large number of grids to become unstable during rainstorms.

A strategy of improving indoor air temperature prediction in HVAC system based on multivariate transfer entropy

The prediction for indoor temperature as an essential reference for system tuning has attracted great concern in the HVAC field. The existing researches mostly focused on transplanting novel machine learning algorithms, especially recurrent neural network (RNN), from the neuro-linguistic programming (NLP) like text generation and speech recognition into HVAC systems. But unlike semantics, the large-scale data processing during the training phase requires the high-speed processor and large memory, which restricts their online application in HVAC systems. In this work, we develop a strategy of improving gated RNN prediction for indoor temperature by estimating time delays of HVAC systems. The suggested strategy can lighten the computational burden and decrease memory capacity to favor online prediction and control, where the traceback range of error is narrowed down to the nearest-neighbors delay periods and the parameter learning time is reduced to about 2% of the original. Among them, we introduce transfer entropy (TE) into the HVAC field to estimate its time delays by mining monitoring data. Compared with correlation coefficients, it can filter redundant information between variables and discern the nonlinearity. Subordinately, we reconstruct time series by embedding time delays as inputs of the prediction model, and clip gradients within backpropagation through time (BPTT) with multivariate TE index, which promote the correlation between inputs and output and absorb the most valuable information in the parameter learning. Lastly, the proposed methods are validated with simulated and real time series. This work is enlightening and instructive to improving time series prediction in HVAC systems.

P694. Effects of Life Events on Suicidal Ideation in EMA Data Using Recurrent Neural Network Prediction

Ecological Momentary Assessment (EMA) records individuals’ ratings of emotions, thoughts, behaviors and stressors during participants’ daily lives. Recurrent Neural Networks (RNNs) were used to model momentary Suicidal Ideation (SI) and assess the importance of individual input features.

Untangling the contribution of input parameters to an artificial intelligence PM2.5 forecast model using the layer-wise relevance propagation method

The recurrent neural network (RNN), an artificial intelligence algorithm, applied to the predictions based on the Community Multiscale Air Quality operational model has significantly improved the forecast accuracy of the concentrations of particulate matter with a diameter of ≤2.5 μm (PM2.5) in the Seoul metropolitan area of the Republic of Korea. It is challenging to interpret the prediction results and identify the related error sources because the decision-making process of the RNN model is inaccessible. This study evaluated the relevance score of the RNN input variables using the layer-wise relevance propagation (LRP) at 6-hourly forecasts over the winters of 2015–2021 (December through February). The relevance score magnitudes summed over the period from the target prediction time to 2–5 and 4–7 time-steps before it (i.e., the most recent 12–30 h and 24–42 h, respectively) show ∼80% of the total relevance score for one- and two-day forecasts, respectively. The input variables were originally selected by their correlation coefficients with the observed PM2.5 concentration; however, the order of input variable contributions measured by the LRP differs from the order of the correlation coefficients, implying inconsistency between the linear and nonlinear methods. Retraining the RNN model using a subset of variables of high relevance scores is found to yield prediction skills comparable to the original set of input variables. This study can contribute to the improvement of the RNN prediction model by decoding the black box of an artificial intelligence model using the LRP method.

Deep Learning Neural Network Prediction System Enhanced with Best Window Size in Sliding Window Algorithm for Predicting Domestic Power Consumption in a Residential Building.

Buildings are considered to be one of the world's largest consumers of energy. The productive utilization of energy will spare the accessible energy assets for the following ages. In this paper, we analyze and predict the domestic electric power consumption of a single residential building, implementing deep learning approach (LSTM and CNN). In these models, a novel feature is proposed, the “best N window size” that will focus on identifying the reliable time period in the past data, which yields an optimal prediction model for domestic energy consumption known as deep learning recurrent neural network prediction system with improved sliding window algorithm. The proposed prediction system is tuned to achieve high accuracy based on various hyperparameters. This work performs a comparative study of different variations of the deep learning model and records the best Root Mean Square Error value compared to other learning models for the benchmark energy consumption dataset.

Bayesian Regularization Algorithm Based Recurrent Neural Network Method and NSGA-II for the Optimal Design of the Reflector

The optical-mechanical system of a space camera is composed of several complex components, and the effects of several factors (weight, gravity, modal frequency, temperature, etc.) on its system performance need to be considered during ground tests, launch, and in-orbit operation. In order to meet the system specifications of the optical camera system, the dimensional parameters of the optical camera structure need to be optimized. There is a highly nonlinear functional relationship between the dimensional parameters of the optical machine structure and the design indexes. The traditional method takes a significant amount of time for finite element calculation and is less efficient. In order to improve the optimization efficiency, a recurrent neural network prediction model based on the Bayesian regularization algorithm is proposed in this paper, and the NSGA-II is used to globally optimize multiple prediction objectives of the prediction model. The reflector of the space camera is used as an example to predict the weight, first-order modal frequency, and gravitational mirror deformation root mean square of the reflector, and to complete the lightweight design. The results show that the prediction model established by BR-RNN-NSGA-II offers high prediction accuracy for the design indexes of the reflector, which all reach over 99.6%, and BR-RNN-NSGA-II can complete the multi-objective optimization search efficiently and accurately. This paper provides a new idea of optimization of optical machine structure, which enriches the theory of complex structure design.

Combining empirical mode decomposition and deep recurrent neural networks for predictive maintenance of lithium-ion battery

Predictive maintenance of lithium-ion batteries has been one of the popular research subjects in recent years. Lithium-ion batteries can be used as the energy supply for industrial equipment, such as automated guided vehicles and battery electric vehicles. Predictive maintenance plays an important role in the application of smart manufacturing. This mechanism can provide different levels of pre-diagnosis for machines or components. Remaining useful life (RUL) prediction is crucial for the implementation of predictive maintenance strategies. RUL refers to the estimated useful life remaining before the machine cannot operate after a certain period of operation. This study develops a hybrid data science model based on empirical mode decomposition (EMD), grey relational analysis (GRA), and deep recurrent neural networks (RNN) for the RUL prediction of lithium-ion batteries. The EMD and GRA methods are first adopted to extract the characteristics of time series data. Then, various deep RNNs, including vanilla RNN, gated recurrent unit, long short-term memory network (LSTM), and bidirectional LSTM, are established to forecast state of health (SOH) and the RUL of lithium-ion batteries. Bayesian optimization is also used to find the best hyperparameters of deep RNNs. Experimental results with the lithium-ion batteries data of NASA Ames Prognostics Data Repository show that the proposed hybrid data science model can accurately predict the SOH and RUL of lithium-ion batteries. The LSTM network has the optimal results. The proposed hybrid data science model with multiple artificial intelligence-based technologies also demonstrates critical digital-technology enablers for digital transformation of smart manufacturing and transportation.

Prediction of the motion of chest internal points using a recurrent neural network trained with real-time recurrent learning for latency compensation in lung cancer radiotherapy.

During the radiotherapy treatment of patients with lung cancer, the radiation delivered to healthy tissue around the tumor needs to be minimized, which is difficult because of respiratory motion and the latency of linear accelerator (LINAC) systems. In the proposed study, we first use the Lucas-Kanade pyramidal optical flow algorithm to perform deformable image registration (DIR) of chest computed tomography (CT) scan images of four patients with lung cancer. We then track three internal points close to the lung tumor based on the previously computed deformation field and predict their position with a recurrent neural network (RNN) trained using real-time recurrent learning (RTRL) and gradient clipping. The breathing data is quite regular, sampled at approximately 2.5 Hz, and includes artificially added drift in the spine direction. The amplitude of the motion of the tracked points ranged from 12.0 mm to 22.7 mm. Finally, we propose a simple method for recovering and predicting three-dimensional (3D) tumor images from the tracked points and the initial tumor image, based on a linear correspondence model and the Nadaraya-Watson non-linear regression. The root-mean-square (RMS) error, maximum error and jitter corresponding to the RNN prediction on the test set were smaller than the same performance measures obtained with linear prediction and least mean squares (LMS). In particular, the maximum prediction error associated with the RNN, equal to 1.51 mm, is respectively 16.1% and 5.0% lower than the error given by a linear predictor and LMS. The average prediction time per time step with RTRL is equal to 119 ms, which is less than the 400 ms marker position sampling time. The tumor position in the predicted images appears visually correct, which is confirmed by the high mean cross-correlation between the original and predicted images, equal to 0.955. The standard deviation of the Gaussian kernel and the number of layers in the optical flow algorithm were the parameters having the most significant impact on registration performance. Their optimization led respectively to a 31.3% and 36.2% decrease in the registration error. Using only a single layer proved to be detrimental to the registration quality because tissue motion in the lower part of the lung has a high amplitude relative to the resolution of the CT scan images. The random initialization of the hidden units and the number of these hidden units were found to be the most important factors affecting the performance of the RNN. Increasing the number of hidden units from 15 to 250 led to a 56.3% decrease in the prediction error on the cross-validation data. Similarly, optimizing the standard deviation of the initial Gaussian distribution of the synaptic weights σinitRNN led to a 28.4% decrease in the prediction error on the cross-validation data, with the error minimized for σinitRNN=0.02 with the four patients.

Rainfall Analysis and Forecasting Using Deep Learning Technique

Rainfall forecasting is very challenging due to its uncertain nature and dynamic climate change. It's always been a challenging task for meteorologists. In various papers for rainfall prediction, different Data Mining and Machine Learning (ML) techniques have been used. These techniques show better predictive accuracy. A deep learning approach has been used in this study to analyze the rainfall data of the Karnataka Subdivision. Three deep learning methods have been used for prediction such as Artificial Neural Network (ANN) - Feed Forward Neural Network, Simple Recurrent Neural Network (RNN), and the Long Short-Term Memory (LSTM) optimized RNN Technique. In this paper, a comparative study of these three techniques for monthly rainfall prediction has been given and the prediction performance of these three techniques has been evaluated using the Mean Absolute Percentage Error (MAPE%) and a Root Mean Squared Error (RMSE%). The results show that the LSTM Model shows better performance as compared to ANN and RNN for Prediction. The LSTM model shows better performance with mini-mum Mean Absolute Percentage Error (MAPE%) and Root Mean Squared Error (RMSE%).

A predictive replication for multi‐tenant databases using deep learning

AbstractThe service level agreement (SLA) is an agreement between clients and the service provider, which defines the minimum performance and availability requirements for software. The service provider should have effective strategies for placement, migration, and replication of the tenants to reduce their operational costs, maximize the utilization of their hardware and software resources and accordingly meet the SLA requirements with slight SLA violations. In this research, a clustered‐based multi‐tenant database management system (CB‐MT DBMS) is proposed. Additionally, a dynamic proactive provisioning technique is built using three different prediction models: The Recursive Window Forecasting Autoregressive Integrated Moving Average (ARIMA) model, Exponential Moving Average (EMA) model, and the proposed Recurrent Neural Network (RNN) with Long Short‐Term Memory (LSTM) cells model. Various experimental scenarios with different datasets have been conducted to prove that the RNN model with LSTM cells is a promising solution in multi‐tenant environments, where the tenants have irregular workload patterns. Experimental results firstly show that the RNN model accuracy is superior to their counterparts (i.e., ARIMA and EMA models) when applied to multi‐tenant database workloads generated using TPC benchmarks, as it reduces the prediction error value which is computed using the Mean Absolute Percentage Error (MAPE) and the Root Mean Square Error (RMSE) metrics. Secondly, Experimental results prove that the RNN prediction model accuracy is superior to their counterparts for detecting SLA violation values and windows using different SLA values.

Application of recurrent neural network for prediction of the time-varying storm surge

ABSTRACT This study investigates the overall performance of non-linear regression models based either on DNN or on RNN for the fast predictions of time-varying storm surge heights. We selected Tokyo-Bay as a case study site and conducted numerical simulations of storm surges induced by 151 recorded typhoons that passed around Japan from 1951 to 2017. Obtained time-series data of the storm surge heights at selected 10 target points were used as outcome variables of the non-linear regression model. The corresponding predictor variables are obtained from the time series of typhoon data and the recent time history of the storm surges height at the same target point. This study then performed a parametric study to explore the optimum model setups of: (i) types of optimizer; (ii) the number of hidden layers (iii) the number of nodes of each hidden layer; and (iv) the duration time and the lead time of the time-series data of the typhoon and the recent storm surge height. Besides the optimum conditions of the present regression model, it was found that RNN showed clearly better predictive skills than DNN, and the recent history of the storm surge did not significantly improve the predictive performance of the present regression model.

Predicting stock high price using forecast error with recurrent neural network

Abstract Stock price forecasting is an eye-catching research topic. In previous works, many researchers used a single method or combination of methods to make predictions. However, accurately predicting stock prices is very difficult. To improve the predicting precision, in this study, an innovative prediction approach was proposed by recurrent substitution of forecast error into the historical neural network model through three steps. According to the historical data, the initial predicted value of the next day is obtained through the neural network. Then, the prediction error of the next day is obtained through the neural network according to the historical prediction error. Finally, the initial predicted value and the prediction error are added to obtain the final predicted value of the next day. We use recurrent neural network prediction methods, such as Long Short-Term Memory Network Model and Gated Recurrent Unit, which are popular in the recent neural network study. In the simulations, the past stock prices of China from June 2010 to August 2017 are used as training data, and those from September 2017 to April 2018 are used as test data. The experimental findings demonstrate that the proposed method with forecast error gives a more accurate prediction result for the stock’s high price on the next day, which indicates that the performance of the proposed one is superior to that of the traditional models without forecast error.

Geometric and dosimetric verification of a recurrent neural network algorithm to compensate for respiratory motion using an articulated robotic couch

The purpose of this study is to evaluate the performance of a recurrent neural network (RNN)-based prediction algorithm to compensate for respiratory movement using an articulated robotic couch system. A prototype of a real-time respiratory motion compensation couch was built using an optical 3D motion tracking system and a six-degree-of-freedom-articulated robotic system. To compensate for the system latency from motion detection to re-positioning of the system, RNN and double exponential smoothing (ES2) prediction algorithms were applied. Three aspects of performance were evaluated, simulation and experiments for geometric and dosimetric evaluations, using data from three liver and three lung patients who underwent stereotactic body radiotherapy. Overall, the RNN algorithm showed better geometric and dosimetric results than the other approaches. In simulation tests, RNN showed 82% average improvement ratio, compared with non-predicted results. In the geometric evaluation, RNN only showed average FWHM broadening of 1.5 mm, compared with the static case. In the dosimetric evaluation, RNN showed average gamma passing rates of 97.4 ± 1.0%, 89.0 ± 2.4% under the 3%/3 mm, 2%/2 mm respectively. It may be technically feasible to use the RNN prediction algorithm to compensate for respiratory motion with an articulated robotic couch system. The RNN algorithm could be widely used for motion compensation in patients undergoing radiotherapy.

Short-term wind speed forecasting using recurrent neural networks with error correction

As a type of clean energy, wind energy has been effectively used in power systems. However, due to the influence of the atmospheric boundary layer, wind speed exhibits strong nonlinearity and nonstationarity. Therefore, the accurate and stable prediction of wind speed is highly important for the security of the power grid. To improve the forecasting accuracy, a novel hybrid forecasting system is proposed in this paper that includes effective data decomposition techniques, recurrent neural network prediction algorithms and error decomposition correction methods. In this system, a novel decomposition approach is used to first decompose the original wind speed series into a set of subseries, then it predicts the wind speed by recurrent neural network, and finally, it decomposes the error to correct the previously predicted wind speed. The effectiveness of the proposed model is verified using data from four different wind farms in China. The results show that the proposed hybrid system is superior to other single models and traditional models and realizes highly accurate prediction of wind speed. The proposed system may be a useful tool for smart grid operation and management.