Nonlinear Autoregressive Network Research Articles

A Review on Deep Learning with Focus on Deep Recurrent Neural Network for Electricity Forecasting in Residential Building

The rapid increase in urbanization has resulted in a significant rise in electricity consumption, which resulted in a wide gap between the amount of electricity generated and the consumer’s demand. Literature shows that 40% of the generated electricity is consumed by building sectors. To address the gap between demand and supply, there is a need to develop novel prediction models that adopt automated techniques, to dynamically predict buildings energy consumption. An efficient load forecast can assist in efficient power generation and distribution among users. Different Machine Learning techniques have been applied in future electricity consumption forecasts with the need for a more ideal solution. This paper reviews methods for building energy consumption forecasts that use Machine Learning algorithms like Artificial Neural Network, Deep Belief Network, Recurrent Neural Network, Elman Neural Network, Deep Recurrent Neural Network, Convolutional Neural Network and Nonlinear Autoregressive Network. The review explores existing research gaps and research directions for future work. Finally, a novel Deep Learning framework was suggested for future work on enhancing prediction performance and reliability using occupancy profile and distinct climatic scenarios based on Transfer Learning and LSTM algorithms (Trans-LSTM) for medium to long term electricity consumption forecast.

Predicting groundwater level fluctuations under climate change scenarios for Tasuj plain, Iran

Coastal aquifers world-wide are being confronted with several major challenges, such as overextraction of groundwater, climate change impacts, contamination by wastewater, and saltwater intrusion into water resources. Climate change induced alteration of the hydrological cycle is one of the main threats to future accessibility of water resources. Effective prediction of possible impacts of climate change on groundwater reserves, a crucial water resource, could be of great importance for sustainable water management. In a comparative study, artificial neural network (ANN), least square support vector machine (LSSVM), and nonlinear autoregressive network with exogenous inputs (NARX) models was applied to evaluate possible impacts of three representative concentration pathways (RCP) climate change scenarios (RCP2.6, RCP4.5, RCP8.5) on groundwater levels in Tasuj Plain, Iran. Four general circulation models (GCM) was used to predict temperature and precipitation values for the future period 2022–2050 and found that future temperature increased, while the amount of precipitation decreased. To improve the accuracy of three models in groundwater level prediction, db4 wavelet transform was applied. The results indicated that the Wavelet-NARX approach gave the best accuracy in forecasting groundwater level in the study area. In all cases, prediction indicated that groundwater level in all representative wells would decline in future.

State of Charge Estimation for Lithium-Ion Battery Based on NARX Recurrent Neural Network and Moving Window Method

An accurate state of charge (SOC) estimation depends on an accurate battery model. The influence of nonlinear and unstable interference factors makes the accurate SOC estimation difficult. To obtain an accurate battery model, a method based on the NARX (nonlinear autoregressive network with exogenous inputs) recurrent neural network and moving window method is proposed. This paper improves the accuracy, modelling speed and robustness of SOC estimation from the following three aspects. First, to overcome the excessive reliance on the amount of data in the model training process, the NARX recurrent neural network is used to establish the battery model. NARX (nonlinear autoregressive with external input) recurrent neural network with the delay and feedback functions can keep the input and output of a previous moment and add it to the calculation of the next moment. Therefore, better estimation results are achieved using a small amount of data; second, the moving window method is used against the gradient explosion and the gradient vanishing that may occur in the NARX model training process. Third, by comparing it with other methods under different working conditions and different temperatures, the validity of the proposed model is verified. The results indicate that the proposed model has a higher accuracy and speed of the SOC estimation. The RMSE performance of the proposed model is reduced by approximately 65%, and the execution time is shortened by approximately 50%.

A Hybrid Data Fusion Approach to AI-Assisted Indirect Centralized Integrated SINS/GNSS Navigation System During GNSS Outage

The main challenges for integration between low-cost strap-down inertial navigation system (SINS) and global navigation satellite system (GNSS) are to manage the non-Gaussian measurement noises and to access the reliable measurements during GNSS outages. Therefore, in this paper, some approaches are taken in the proposed integrated navigation system: (1) using an iterative empirical mode decomposition (EMD) interval thresholding de-noising method for low-cost inertial measurements to provide smoother and more accurate SINS measurements with less complexity, (2) designing an enhanced indirect centralized correntropy Kalman filter using a novel fuzzy parameter adjustment approach, which adaptively changes the estimation algorithm parameters to further improve the robustness of the proposed algorithm in the presence of dynamical maneuvers and non-Gaussian measurement noise, and (3) utilizing a pre-filtered nonlinear autoregressive network with exogenous inputs (PFNARX) designed to provide reliable interrupted positioning information during GNSS outages. The proposed SINS/GNSS system is experimentally evaluated through real-world flight tests. The obtained results reveal that the designed integration algorithm remarkably improves the estimation accuracy by adaptively tuning the parameters in an improved correntropy Kalman filter and an efficient and well-trained artificial neural network-based model in the absence of GNSS signal.

Data pre-processing and artificial neural networks for tidal level prediction at the Pearl River Estuary

Abstract Traditionally, tidal level is predicted by harmonic analysis (HA). In this paper, three hybrid models that couple varied pre-processing methods, which are empirical mode decomposition (EMD), ensemble empirical mode decomposition (EEMD), and empirical wavelet transform (EWT), with the nonlinear autoregressive networks with exogenous inputs (NARX) were applied to forecast tidal level. The models were, namely, EMD-NARX, EEMD-NARX, and EWT-NARX. The sub-series obtained by using EMD or EEMD or EWT were then used as the input vectors to the NARX with the original data as targets. Notably, the EWT-NARX model was employed to predict the tidal level for the first time. Simulations were based on the measurements from four tidal stations at the Pearl River Estuary, China. The results showed that the EWT-NARX, EEMD-NARX, and EMD-NARX outperformed the HA model. Specifically, EWT-NARX was optimal among the four. Moreover, from the Hilbert energy spectra we can see the EWT solved the mode-mixing problem that EMD and EEMD suffered from, thus enabling precise tidal level prediction. Simulations and experimental results confirmed that the EWT-NARX model can achieve prediction of the tidal level with high accuracy.

Predicting future cancer burden in the United States by artificial neural networks.

Aims: To capture the complex relationships between risk factors and cancer incidences in the US and predict future cancer burden. Materials & methods: Two artificial neural network (ANN) algorithms were adopted: a multilayer feed-forward network (MLFFNN) and a nonlinear autoregressive network with eXogenous inputs (NARX). Data on the incidence of the four most common tumors (breast, colorectal, lung and prostate) from 1992 to 2016 (available from National Cancer Institute online datasets) were used for training and validation, and data until 2050 were predicted. Results: The rapid decreasing trend of prostate cancer incidence started in 2010 will continue until 2018-2019;it will then slow down and reach a plateau after 2050, with several differences amongethnicities. The incidence of breast cancer will reach a plateau in 2030, whereascolorectal cancer incidence will reach a minimum value of 35 per100,000 in 2030. As for lung cancer, the incidence will decrease from 50 per100,000 (2017) to 31 per100,000 in 2030 and 26 per100,000 in 2050. Conclusion: This up-to-date prediction of cancer burden in the UScould be a crucial resource for planning and evaluation of cancer-control programs.

Dynamic modelling of the biomass gasification process in a fixed bed reactor by using the artificial neural network

Abstract Biomass gasification enables the transformation of biomass feedstock into syngas suitable for further energy conversions. Mathematical models of gasification are not only valuable tool for the design and optimization of the processes, but could be also employed for online prediction and process control. In this work, the potential of using nonlinear autoregressive networks with exogenous inputs (NARX) for predicting the gasification process when a lower amount of experimental data is available was studied. The analysis of using an open-loop NARX network for an online prediction of the syngas composition in a downdraft gasifier at different data recording frequency was performed. The predicted results showed that by decreasing the data recording frequency, the prediction error increases. Furthermore, the possibility of improving the NARX network at lower data recording frequency was analysed by expanding the training dataset to reduce discrepancies between predicted and measured results. Inclusion of four data sets made neural network more robust and flexible for operating with fewer data points. Practical significance of results can be seen in the application of open-loop network for online prediction and control of the gasification process at lower data recording frequency as a part of a model predictive control module.

Time series modelling, NARX neural network and hybrid KPCA–SVR approach to forecast the foreign exchange market in Mauritius

PurposeThis study constructs time series model, artificial neural networks (ANNs) and statistical topologies to examine the volatility and forecast foreign exchange rates. The Mauritian forex market has been utilized as a case study, and daily data for nominal spot rate (during a time period of five years spanning from 2014 to 2018) for EUR/MUR, GBP/MUR, CAD/MUR and AUD/MUR have been applied for the predictions.Design/methodology/approachAutoregressive integrated moving average (ARIMA) and generalized autoregressive conditional heteroskedasticity (GARCH) models are used as a basis for time series modelling for the analysis, along with the non-linear autoregressive network with exogenous inputs (NARX) neural network backpropagation algorithm utilizing different training functions, namely, Levenberg–Marquardt (LM), Bayesian regularization and scaled conjugate gradient (SCG) algorithms. The study also features a hybrid kernel principal component analysis (KPCA) using the support vector regression (SVR) algorithm as an additional statistical tool to conduct financial market forecasting modelling. Mean squared error (MSE) and root mean square error (RMSE) are employed as indicators for the performance of the models.FindingsThe results demonstrated that the GARCH model performed better in terms of volatility clustering and prediction compared to the ARIMA model. On the other hand, the NARX model indicated that LM and Bayesian regularization training algorithms are the most appropriate method of forecasting the different currency exchange rates as the MSE and RMSE seemed to be the lowest error compared to the other training functions. Meanwhile, the results reported that NARX and KPCA–SVR topologies outperformed the linear time series models due to the theory based on the structural risk minimization principle. Finally, the comparison between the NARX model and KPCA–SVR illustrated that the NARX model outperformed the statistical prediction model. Overall, the study deduced that the NARX topology achieves better prediction performance results compared to time series and statistical parameters.Research limitations/implicationsThe foreign exchange market is considered to be instable owing to uncertainties in the economic environment of any country and thus, accurate forecasting of foreign exchange rates is crucial for any foreign exchange activity. The study has an important economic implication as it will help researchers, investors, traders, speculators and financial analysts, users of financial news in banking and financial institutions, money changers, non-banking financial companies and stock exchange institutions in Mauritius to take investment decisions in terms of international portfolios. Moreover, currency rates instability might raise transaction costs and diminish the returns in terms of international trade. Exchange rate volatility raises the need to implement a highly organized risk management measures so as to disclose future trend and movement of the foreign currencies which could act as an essential guidance for foreign exchange participants. By this way, they will be more alert before conducting any forex transactions including hedging, asset pricing or any speculation activity, take corrective actions, thus preventing them from making any potential losses in the future and gain more profit.Originality/valueThis is one of the first studies applying artificial intelligence (AI) while making use of time series modelling, the NARX neural network backpropagation algorithm and hybrid KPCA–SVR to predict forex using multiple currencies in the foreign exchange market in Mauritius.

Improved Rainfall Prediction through Nonlinear Autoregressive Network with Exogenous Variables: A Case Study in Andes High Mountain Region

Precipitation is the most relevant element in the hydrological cycle and vital for the biosphere. However, when extreme precipitation events occur, the consequences could be devastating for humans (droughts or floods). An accurate prediction of precipitation helps decision-makers to develop adequate mitigation plans. In this study, linear and nonlinear models with lagged predictors and the implementation of a nonlinear autoregressive model with exogenous variables (NARX) network were used to predict monthly rainfall in Labrado and Chirimachay meteorological stations. To define a suitable model, ridge regression, lasso, random forest (RF), support vector machine (SVM), and NARX network were used. Although the results were “unsatisfactory” with the linear models, the specific direct influences of variables such as Niño 1 + 2, Sahel rainfall, hurricane activity, North Pacific Oscillation, and the same delayed rainfall signal were identified. RF and SVM also demonstrated poor performance. However, RF had a better fit than linear models, and SVM has a better fit than RF models. Instead, the NARX model was trained using several architectures to identify an optimal one for the best prediction twelve months ahead. As an overall evaluation, the NARX model showed “good” results for Labrado and “satisfactory” results for Chirimachay. The predictions yielded by NARX models, for the first six months ahead, were entirely accurate. This study highlighted the strengths of NARX networks in the prediction of chaotic and nonlinear signals such as rainfall in regions that obey complex processes. The results would serve to make short-term plans and give support to decision-makers in the management of water resources.

Gamma ray log generation from drilling parameters using deep learning

Lithology identification plays a vital role in defining the petroleum reservoir. Although well logging represents the traditional means of obtaining petrophysical data for lithology identification, there could be cases where logging while drilling instruments may fail during drilling. This paper presents an approach that utilizes drilling parameters obtained from mud logging and measurement while drilling (MWD) for real-time prediction of gamma ray log which is used as a lithology identifier. In this work, several machine learning methodologies, such as simple recurrent neural network (RNN), long short-term memory recurrent neural network (LSTM-RNN), temporal convolution network (TCN), gated recurrent unit (GRU) network, nonlinear autoregressive network with exogenous inputs (NARX) and simple artificial neural network (ANN) were tested for their ability to capture the relationship between hydro-mechanical specific energy (computed from drilling parameters) and gamma ray log. A recently drilled exploration gas well in the tertiary deltaic system of the Niger delta basin was used as the case study. Based on the field data, the results show the TCN and simple RNN performed best. The receptive field of the TCN played a significant role in its performance, and therefore, the LSTM-RNN can be made to perform comparable to that of TCN/simple RNN if the LSTM-RNN is manually made to work with an optimal window of input data points for each output data point. The size and nature of the data (volume, velocity, variety, and veracity) is likely a significant factor in the performance of the machine learning methodologies. Thus, it is recommended to focus on obtaining the best receptive field of the data during the machine learning development phase.

Comparative Study of Recurrent and Non Recurrent Neural Network Based Approach for Modeling of PEM Fuel Cell Powered Electric Vehicle

In this paper, a non recurrent Focused Time Delay Neural Network (FTDN) and a recurrent nonlinear autoregressive network with exogenous inputs (NARX) Neural Network are employed as a black box prediction model for substituting the complex conventional model of PEM 5kW Proton Exchange Membrane (PEM) Fuel Cell system. A comparative assessment is performed between the recurrent and non recurrent neural network on certain performance measures to identify an optimal network for modeling the PEM Fuel Cell System in PEM Fuel Cell powered electric vehicle application. From the simulation result, it is observed that the recurrent NARX network is showed excellent prediction ability in terms of minimizing the Mean Square Error (MSE) value with faster convergence. The optimized network is tested with intermittent data for examining its adaptability and validated with experimental benchmark data for proving its reliability. Hence the optimum network is integrated with converters and vehicle dynamic system to develop a fuel cell based electric vehicle system. The performance of the proposed vehicle is tested with US06 drive cycle pattern for justifying its reliability.

The Use of Neural Networks and Genetic Algorithms to Control Low Rigidity Shafts Machining.

The article presents an original machine-learning-based automated approach for controlling the process of machining of low-rigidity shafts using artificial intelligence methods. Three models of hybrid controllers based on different types of neural networks and genetic algorithms were developed. In this study, an objective function optimized by a genetic algorithm was replaced with a neural network trained on real-life data. The task of the genetic algorithm is to select the optimal values of the input parameters of a neural network to ensure minimum deviation. Both input vector values and the neural network’s output values are real numbers, which means the problem under consideration is regressive. The performance of three types of neural networks was analyzed: a classic multilayer perceptron network, a nonlinear autoregressive network with exogenous input (NARX) prediction network, and a deep recurrent long short-term memory (LSTM) network. Algorithmic machine learning methods were used to achieve a high level of automation of the control process. By training the network on data from real measurements, we were able to control the reliability of the turning process, taking into account many factors that are usually overlooked during mathematical modelling. Positive results of the experiments confirm the effectiveness of the proposed method for controlling low-rigidity shaft turning.

Wrapper-Enabled Feature Selection and CPLM-Based NARX Model for Stock Market Prediction

Abstract The prices in the stock market are dynamic in nature, thereby pretend as a hectic challenge to the sellers and buyers in predicting the trending stocks for the future. To ensure effective prediction of the stock market, the chronological penguin Levenberg–Marquardt-based nonlinear autoregressive network (CPLM-based NARX) is employed, and the prediction is devised on the basis of past and the recent rank of market. Initially, input data are subjected to the features extraction that is based on the technical indicators, such as WILLR, ROCR, MOM, RSI, CCI, ADX, TRIX, MACD, OBV, TSF, ATR and MFI. The technical indicator is adapted for predicting the stock market. The wrapper-enabled feature selection is employed for selecting the highly significant features that are generated using the technical indicators. The highly significant features of the data are fed to the prediction module, which is developed using the NARX model. The NARX model uses the CPLM algorithm that is formed using the integration of the chronological-based penguin search optimization algorithm and the Levenberg–Marquardt algorithm. The prediction using the proposed CPLM-based NARX shows the superior performance in terms of mean absolute percentage error and root mean square error with values of 0.96 and 0.805, respectively.

A comparative analysis of artificial neural networks and wavelet hybrid approaches to long-term toxic heavy metal prediction

The occurrence of toxic metals in the aquatic environment is as caused by a variety of contaminations which makes difficulty in the concentration prediction. In this study, conventional methods of back-propagation neural network (BPNN) and nonlinear autoregressive network with exogenous inputs (NARX) were applied as benchmark models. Explanatory variables of Fe, pH, electrical conductivity, water temperature, river flow, nitrate nitrogen, and dissolved oxygen were used as different input combinations to forecast the long-term concentrations of As, Pb, and Zn. The wavelet transformation was applied to decompose the time series data, and then was integrated with conventional methods (as WNN and WNARX). The modelling performances of the hybrid models of WNN and WNARX were compared with the conventional models. All the given models were trained, validated, and tested by an 18-year data set and demonstrated based on the simulation results of a 2-year data set. Results revealed that the given models showed general good performances for the long-term prediction of the toxic metals of As, Pb, and Zn. The wavelet transform could enhance the long-term concentration predictions. However, it is not necessarily useful for each metal prediction. Therefore, different models with different inputs should be used for different metals predictions to achieve the best predictions.

Air catalytic biomass (PKS) gasification in a fixed-bed downdraft gasifier using waste bottom ash as catalyst with NARX neural network modelling

The air gasification of Palm Kernel Shells (PKS) using coal bottom ash (CBA) as a catalyst has been performed in a fixed-bed gasifier. The impact of three process parameters, namely, temperature (575–775 °C), air flowrate (1.5–45 litter/min) and catalyst loading (0–30 wt.%) has been investigated on the product gas yield. The composition of the H2 product is computed to be a maximum of 28 vol.% at 875 °C. The air flowrate has a direct relation with H2 production. The catalysts used have demonstrated a positive impact on the carbon conversion efficiency, showing the increase in carbon-containing gases in the product gas due to the increases in gas yield. A Non-linear Autoregressive Network with exogenous inputs (NARX) neural network has been used to predict the gaseous flowrate dynamically in order to improve gasification performance. The predicted results from the NARX network demonstrate good agreement with the experimental study with R2 ≥ 0.99.

Improving Thermochemical Energy Storage Dynamics Forecast with Physics-Inspired Neural Network Architecture

Thermochemical Energy Storage (TCES), specifically the calcium oxide (CaO)/calcium hydroxide (Ca(OH)2) system is a promising energy storage technology with relatively high energy density and low cost. However, the existing models available to predict the system’s internal states are computationally expensive. An accurate and real-time capable model is therefore still required to improve its operational control. In this work, we implement a Physics-Informed Neural Network (PINN) to predict the dynamics of the TCES internal state. Our proposed framework addresses three physical aspects to build the PINN: (1) we choose a Nonlinear Autoregressive Network with Exogeneous Inputs (NARX) with deeper recurrence to address the nonlinear latency; (2) we train the network in closed-loop to capture the long-term dynamics; and (3) we incorporate physical regularisation during its training, calculated based on discretized mole and energy balance equations. To train the network, we perform numerical simulations on an ensemble of system parameters to obtain synthetic data. Even though the suggested approach provides results with the error of 3.96×10−4 which is in the same range as the result without physical regularisation, it is superior compared to conventional Artificial Neural Network (ANN) strategies because it ensures physical plausibility of the predictions, even in a highly dynamic and nonlinear problem. Consequently, the suggested PINN can be further developed for more complicated analysis of the TCES system.

A non-invasive continuous cuffless blood pressure estimation using dynamic Recurrent Neural Networks

Cardiovascular diseases (CVD) have become the most important health problem of our time. High blood pressure, which is cardiovascular disease, is a risk factor for death, stroke, and heart attack. Blood pressure measurement is commonly used to limit blood flow in the arm or wrist, with the cuff. Since blood pressure cannot be measured continuously in this method, the dynamics underlying blood pressure cannot be determined and are inefficient in capturing symptoms. This paper aims to perform blood pressure estimation using Photoplethysmography (PPG) and Electrocardiography (ECG) signals that do not obstruct the vascular access. These signals were filtered and segmented synchronously from the R interval of the ECG signal, and chaotic, time, and frequency domain features were subtracted, and estimation methods were applied. Different methods of machine learning in blood pressure estimation are compared. Dynamic learning methods such as Recurrent Neural Network (RNN), Nonlinear Autoregressive Network with Exogenous Inputs Neural Networks NARX-NN and Long-Short Term Memory Neural Network (LSTM-NN) used. Estimation results have been evaluated with performance criteria. Systolic Blood Pressure (SBP) error mean ± standard deviation = 0.0224 ± (2.211), Diastolic Blood Pressure (DBP) error mean ± standard deviation = 0.0417 ± (1.2193) values have been detected in NARX artificial neural network. The blood pressure estimation results are evaluated by the British Hypertension Society (BHS) and American National Standard for Medical Instrumentation ANSI/AAMI SP10: 2002. Finding the most accurate and easy method in blood pressure measurement will contribute to minimizing the errors.

The nonlinear autoregressive network with exogenous inputs (NARX) neural network to damp power system oscillations

The nonlinear autoregressive network with exogenous inputs (NARX) neural network to damp power system oscillations

Tree Search Fuzzy NARX Neural Network Fault Detection Technique for PV Systems with IoT Support

The photovoltaic (PV) panel’s output energy depends on many factors. As they are becoming the leading alternative energy source, it is essential to get the best out of them. Although the main factor for maximizing energy production is proportional to the amount of solar radiation reaching the photovoltaic panel surface, other factors, such as temperature and shading, influence them negatively. Moreover, being installed in a dynamic and frequently harsh environment causes a set of reasons for faults, defects, and irregular operations. Any irregular operation should be recognized and classified into faults that need attention and, therefore, maintenance or as being a regular operation due to changes in some surrounding factors, such as temperature or solar radiation. Besides, in case of faults, it would be helpful to identify the source and the cause of the problem. Hence, this study presented a novel methodology that modeled a PV system in a tree-like hierarchy, which allowed the use of a fuzzy nonlinear autoregressive network with exogenous inputs (NARX) to detect and classify faults in a PV system with customizable granularity. Moreover, the used methodology enabled the identification of the exact source of fault(s) in a fully automated way. The study was done on a string of eight PV panels; however, the paper discussed using the algorithm on a more extensive PV system. The used fuzzy NARX algorithm in this study was able to classify the faults that appeared in up to five out of the eight PV panels and to identify the faulty PV panels with high accuracy. The used hardware could be controlled and monitored through a Wi-Fi connection, which added support for Internet of Things applications.

A Model-Guided Neural Network for the Prediction of Creep Behavior Under in-Service Conditions

Abstract Data-driven neural network methods have been widely applied for the prediction of stress–strain behavior, but have proven ill-suited for the extrapolation of time-dependent creep behavior. To overcome this problem, we embedded a physics-based model into feedforward neural networks (FFNNs) to construct a model-guided neural network (MGNN). We proposed a new initialization method for the weights in the model, based on selecting the appropriate physics-based model and activation function, and the resulting MGNN was used for predicting the creep behavior of blade-grooves in a steam turbine rotor under in-service conditions. We compared the performance of the MGNN with baseline methods, namely MGNN0, a FFNN, and a nonlinear autoregressive network with exogenous inputs (network). The results showed that the physics-based model and the neural network in the MGNN complemented each other: the model provided physical relationships to guide the neural network, and the neural network provided stress-fluctuation-tracking for the model. This functionality enabled primary creep behavior to be used as training data for the MGNN, enabling it to predict secondary creep behavior.