Nonlinear Autoregressive Neural Network Research Articles

Sound Velocity Profiles Time Series Prediction Method Based on EMD-NARX Model

To solve the problem of SVP (Sound velocity profiles) representative error caused by the difficulty of obtaining continuous time series SVP in deep-sea operations, an SVP timing prediction method combining EMD (Empirical mode decomposition) and NARX (Nonlinear autoregressive neural network with external input) is proposed. To begin with, the time-series SVP are stratified according to different depths, and the time-series variation curves of sound velocity at different depths are obtained; Furthermore, the EMD is used to decompose the time series variation curve of sound velocity into multiple IMF (Intrinsic mode function) components, each component contains local characteristic signals of different time scales of the original signal. The NARX is used to establish a prediction model for each IMF components, and the prediction values of the sound velocity at different depths are obtained. The EMD-NARX, NARX and polynomial fitting model are analyzed and verified by Argo buoys data in the South China Sea, and the results of the experiment show that EMD-NARX improves the time series prediction accuracy of sound velocity by 32.24% and 65.15% compared with NARX and polynomial fitting, respectively, so EMD-NARX has a good prediction effect on the time series SVP of the deep sea.

A Comparison Study of Predictive Models for Electricity Demand in a Diverse Urban Environment

The increasing population migration to urban and peri-urban areas increases basic service needs for cities worldwide. Residential electricity demand increases with more customers and varies with novel uses, such as charging electric vehicles, which may add additional dynamics to the residential electricity demand profile. Widespread installation of smart electricity meters provides fine temporal resolution data reflecting current demands and supports predicting future demands. As part of a demand-side management program, understanding the main drivers of current electricity usage based on demand-driven and exogenous predictors represents a valuable tool for utilities facing new demand scenarios. This work presents the application of multiple models to forecast electricity demand based on the input data and the forecasting horizon. Models with exogenous variables as predictors are part of the input–output category, including a Feed Forward Neural Network, Random Forest, and a Linear Gaussian State Space model. The second category is demand-driven models, where predictors include only previous demand values. The demand-driven models in our analysis include a univariate Nonlinear Autoregressive Neural Network and a Linear Gaussian State Space. Using smart electricity meter data from the greater Chicago area, we compare the performance of the models on two different types of accounts: single- and multi-family residential users when forecasting one and multiple steps. Results show that the Linear Gaussian model reports an R2 of 0.99 compared to an average R2 of 0.92 from the Feed Forward Neural Networks and Random Forest when forecasting single- and multi-family electricity demand one step ahead. However, Nonlinear Autoregressive Neural Networks report an average R2 of 0.85 compared to the Linear Gaussian R2 of 0.58 when forecasting 48 steps. We also found that the most important predictors for single-family demand are temporal variables like weekdays, working and non-working days, and day hours. For multi-family demand, electricity demand at the same hour as the previous week replaces weekdays as a significant predictor. Different forecasting models can assist utilities and city planners to manage demand under different and novel residential electricity usage conditions.

Design of intelligent hybrid NAR-GRNN paradigm for fractional order VDP chaotic system in cardiac pacemaker with relaxation oscillator

Managing cardiac disease and abnormal heart rate variability is a challenging problem with its psychological impact on lifesaving intervention. The research presents a novel machine learning approach to paradigm dynamic pacemaker design based on fractional order modified Van der Pol oscillator system implemented to generate rich non-sinusoidal signals for cardiac intervention and treatment. The physical framework of the novel contribution comprises Sino-Atrial (SA) and Atrio-Ventricular (AV) nodes, designed to utilize fractional order excitation signals derived from the VDP dynamic system. Hybrid paradigm is designed by integrating the dynamical nonlinear autoregressive (NAR) neural networks and generalized regression neural networks (GRNNs) to analyze the parametric fractal impulse model for cardiac muscle. The chaotic pattern of the proposed nonlinear system is explored in terms of phase portraits and Lyapunov exponents. The exceptional performance of the cutting-edge fractional order multimodal computing paradigm is validated by achieving an RMSE of 0.1E-14. The dynamic chaotic and fractal trajectory of relaxation oscillator based on its computed control parameters can provide valuable insight to design robust and efficient electronic implantable cardioverter defibrillators (ICDs) as pacemakers to stabilize cardiac impulse variability and alleviate healthcare burden as compared to traditional cardiac rehabilitation procedures.

Nonlinear auto regressive Elman neural network combined with unscented Kalman filter for data-driven dynamic data reconciliation in dynamic systems

Measurement data plays an important role in the control system, but the data collected by sensors often has measurement noise, which makes the states of the system cannot be accurately revealed. Unscented Kalman filter (UKF) is a highly accurate and robust filtering algorithm, but its limitation is the requirement of prior knowledge of the exact dynamic mathematical model, which is a critical issue to be addressed in practice. In this work, a data-driven dynamic data reconciliation scheme called nonlinear auto regressive Elman neural network (ENN) with exogenous inputs combined with UKF (NARX-ENN-UKF) is proposed, where nonlinear auto regressive ENN with exogenous inputs is used for dynamic data-driven modeling, and then UKF is applied for dynamic data reconciliation of the measurements based on the trained model. The scheme is applied to a DC/AC inverter experimental system and a self-developed sliding electrical contact experimental system to verify the effectiveness of NARX-ENN-UKF.

Elevating Univariate Time Series Forecasting: Innovative SVR-Empowered Nonlinear Autoregressive Neural Networks

Efforts across diverse domains like economics, energy, and agronomy have focused on developing predictive models for time series data. A spectrum of techniques, spanning from elementary linear models to intricate neural networks and machine learning algorithms, has been explored to achieve accurate forecasts. The hybrid ARIMA-SVR model has garnered attention due to its fusion of a foundational linear model with error correction capabilities. However, its use is limited to stationary time series data, posing a significant challenge. To overcome these limitations and drive progress, we propose the innovative NAR–SVR hybrid method. Unlike its predecessor, this approach breaks free from stationarity and linearity constraints, leading to improved model performance solely through historical data exploitation. This advancement significantly reduces the time and computational resources needed for precise predictions, a critical factor in univariate economic time series forecasting. We apply the NAR–SVR hybrid model in three scenarios: Spanish berry daily yield data from 2018 to 2021, daily COVID-19 cases in three countries during 2020, and the daily Bitcoin price time series from 2015 to 2020. Through extensive comparative analyses with other time series prediction models, our results substantiate that our novel approach consistently outperforms its counterparts. By transcending stationarity and linearity limitations, our hybrid methodology establishes a new paradigm for univariate time series forecasting, revolutionizing the field and enhancing predictive capabilities across various domains as highlighted in this study.

Модель для прогнозирования поверхностной концентрации метана в арк­тическом регионе, основанная на искусственной нейронной сети с длинной цепью элементов краткосрочной памяти и вейвлет-преобразованием исходных данных

The study of the dynamics of greenhouse gases in the Arctic regions of the planet is becoming increasingly important. Such studies are especially relevant due to the climate change observed in this region. The paper propose a hybrid model that combines wavelet transformation of the original data and an artificial neural network with a long chain of short-term memory (LSTM) elements to predict changes in the surface methane concentration in the Arctic latitudes. The methane concentration time series via a discrete wavelet transform was decomposed into four components — one approximating and three detailing ones. These components were used to train the LSTM network. The forecast was calculated as the sum of forecasts for each of the components. Three predictive models were built. In the first, the LSTM network was trained in a non-linear autoregressive mode. The second one was a combination of discrete wavelet transform with LSTM neural network. An additional model based on a non-linear autoregressive neural network (NAR) was also used for comparison. The work is based on data from environmental monitoring of greenhouse gases on Bely Island, Yamalo-Nenets Autonomous Area of Russia. The initial data for building the proposed model were obtained within July-August 2017. The accuracy of the forecast was assessed using several indicators. The hybrid model based on LSTM showed the best accuracy.

Wholesale Food Price Index Forecasts with the Neural Network

Food price forecasts in the agricultural sector have always been a vital matter to a wide variety of market participants. In this work, we approach this forecast problem for the weekly wholesale food price index in the Chinese market during a 10-year period of January 1, 2010–January 3, 2020. To facilitate the analysis, we propose the use of the nonlinear auto-regressive neural network. Technically, we investigate forecast performance, based upon the relative root mean square error (RRMSE) as the evaluation metrics, corresponding to one hundred and twenty settings that cover different algorithms for model estimations, numbers of hidden neurons and delays, and ratios for splitting the data. Our experimental result suggests the construction of the neural network with three delays and 10 hidden neurons, which is trained through the Levenberg–Marquardt algorithm, as the forecast model. It leads to high accuracy and stabilities with the RRMSEs of 1.93% for the training phase, 2.16% for the validation phase, and 1.95% for the testing phase. Comparisons of forecast accuracy between the proposed model and some other machine learning models, as well as traditional time-series econometric models, suggest that our proposed model leads to statistically significant better performance. Our results could benefit different forecast users, such as policymakers and various market participants, in policy analysis and market assessments.

Deep learning framework for day-ahead optimal charging scheduling of electric vehicles in parking lot

The increase of electric vehicles (EVs) in the power system, in addition to environmental benefits, also has economic advantages. Charging parking lots can perform better economically through aggregators the benefits of participating in energy markets and developing different charging programs. The main challenge for participating in the electricity market for parking lots is the uncertainties in the behavior of EV owners and the day-ahead market (DAM) price. Therefore, this paper proposes a deep learning-based framework for day-ahead optimal charging scheduling, which is in three stages. In the first stage, the parameters of charging sessions are modeled using the proposed Copula generative adversarial network (CopulaGAN). In the second stage, the DAM price is forecasted using the time series multi-input nonlinear autoregressive neural network for 24 h ahead. In addition, in the proposed framework, the balancing energy market (BEM) is used to keep the charging power balance on the implementation day of the optimal charging program. In the third stage, optimal day-ahead EV scheduling is performed. The simulation is carried out using Grey Wolf Optimization (GWO) based on the Caltech charging sessions dataset and the Germany energy market dataset. The simulation results show that using the proposed deep learning framework reduces the charging cost for the parking lots compared to the fixed price approaches, the use of the BEM, the time of use (TOU) program, and using normal distribution modeling about 19.94 €, 159.97 €, 14.27 €, and 114.03 € respectively.

Performance of short-terms prediction methods of vertical total electron content using nonlinear autoregressive neuronal network and stochastic autoregressive model

In this contribution the performance of global short-term predictions methods of vertical total electron content (vTEC) is analyzed during high solar activity. Two kind of predicted global vTEC maps value every 1 h, one-day-ahead, are used. They are C1PG, produced by the Center for Orbit Determination in Europe (CODE), based on the extrapolation of Spherical Harmonic coefficient using Least-squares collocation and the M1PG, proposed in this work, based on multi-step Nonlinear Autoregressive Neural Network (NAR-NN).Global vTEC maps from CODE (CODG) along the year 2015, are used as reference data. The results are obtained for quiet and disturbed conditions, based on geomagnetic and ionospheric planetary indexes.The performance of the forecasting approach is extensively tested under different geospatial conditions. The testing results are very similar in terms of RMSE, as it has been found to range between 1.7 and 7 TECu. RMSE depend on the latitude sectors and geomagnetic conditions, in terms of Mean Forecast Error (MFE) the C1PG shows a clear systematic behavior being negative at Southern Hemisphere and positive at Northern Hemisphere. According to the Mean Absolute Percentage Error (MAPE) values, the relative behavior of vTEC prediction is better for M1PG than for C1PG specially during quiet days and at mid-high latitudes. In general, both models are less accuracy in the equatorial ionization anomaly region and the Southern Hemisphere.Other important contribution of this manuscript is the definition of a planetary ionospheric disturbance index, Wd, based on W-index. This parameter is useful to more complete definition in quiet and disturbed days selection, and it is shown that the dependence of the RMSE according to the latitudinal bands, it is strongly related with the respective value of Wd.

Retraction Note: A new prediction approach of the COVID-19 virus pandemic behavior with a hybrid ensemble modular nonlinear autoregressive neural network

Retraction Note: A new prediction approach of the COVID-19 virus pandemic behavior with a hybrid ensemble modular nonlinear autoregressive neural network

Active vibration control using nonlinear auto-regressive neural network to identify secondary channel

The power unit on board the ship generates periodic low-frequency vibration that affects the normal operation of the equipment on board, and the adaptive feedforward control algorithm can effectively suppress such harmful vibration noise. But the adaptive feedforward control algorithm needs to obtain the identification model of the secondary channels, and the frequency domain least squares method based on the linear Extended auto-regressive model (ARX) is difficult to obtain the identification model with nonlinear characteristics. The nonlinear auto-regressive model (NARX) adds nonlinear mapping layers to the topology of the ARX model to enhance the identification capability of the NARX model for complex systems. In this paper, a block diagram of the Fx-LMS feedforward control algorithm based on the NARX model is proposed, then the initial parameters of the NARX neural network are optimized using the Quantum Particle Swarm Optimization (QPSO) algorithm and the secondary channel is identified, and the identification results show that the accuracy of identifying the secondary channel using the NARX neural network is higher than that of the ARX model. The simulation and experimental results show that the vibration damping effect of the proposed method is better than the traditional Fx-LMS method for both single-line spectrum and multi-line spectrum periodic low-frequency disturbances, which provides a new method for the suppression of periodic low-frequency disturbances.

Predicting global energy demand for the next decade: A time-series model using nonlinear autoregressive neural networks

Energy demand forecasting has been an indispensable research target for academics, which has led to creative solutions for energy utilities in terms of power system design, control, and planning. The usefulness of energy demand forecasting is confined to the power engineering industry but globally exceeds such outcomes to contribute to the environment and health sectors. Despite the large number of research projects published on this topic, the challenge of energy demand forecasting still exists, especially with the developments in modeling concepts via artificial intelligence, which motivates more attractive solutions for the variables involved in energy demand forecasting. Mathematical correlation or extrapolation-like methods cannot be effective in all situations; however, when a time series neural network is presented, most statistical, empirical, and theoretical problems can be easily handled. This paper presents a simple and easy-to-understand method for the next decade of energy demand forecasting based on a nonlinear autoregressive (NAR) neural network. From its time series past values, NAR structurally is an optimal predictor for a future variable. A publicly available data set for global energy consumption has been used to construct the network model with sufficiently accurate results. The evidence has appeared in precisely following the exponential trend of energy consumption as well as the regressions for training, testing, and validation, which ensures the model's robustness and avoids getting involved in overfitting. The proposed model concepts and results can be easily used in undergraduate engineering education, training graduates, and future research.

Data-driven nonlinear MIMO modeling for turbofan aeroengine control system of autonomous aircraft

The mathematical modeling problem of a general turbofan aeroengine control system used in autonomous aircraft is investigated in this paper. Unlike the thermodynamics or physical mechanism-based modeling methods in the existing literature, a pure data-driven modeling approach is presented with high accuracy via online and offline data only. This is achieved by using the nonlinear autoregressive neural network with exogenous inputs (NARX) neural network. In comparison with the existing NARX-based MIMO modeling methods, the series–parallel structure and the parallel structure are integrated for the neural network training. Faster convergence and stability as well as higher modeling accuracy are thus achieved. Another feature of this approach is that it is independent of all the components’ dynamics in the turbofan aeroengine. Simulation results are given to verify the effectiveness of the proposed modeling approach. This is further validated by experimental testing with the method applied to mini turbofan aeroengine testbed.

A Machine-Learning-Based Approach for Natural Gas Futures Curve Modeling

This work studies the term structure dynamics in the natural gas futures market, focusing on the Dutch Title Transfer Facility (TTF) daily futures prices. At first, using the whole dataset, we compared the in-sample fitting performance of three models: the four-factor dynamic Nelson–Siegel–Svensson (4F-DNSS) model, the five-factor dynamic De Rezende–Ferreira (5F-DRF) model, and the B-spline model. Our findings suggest that B-spline is the method that achieves the best in-line fitting results. Then, we turned our attention to forecasting, using data from 20 January 2011 to 13 May 2022 as the training set and the remaining data, from 16 May to 13 June 2022, for day-ahead predictions. In this second part of the work we combined the above mentioned models (4F-DNSS, 5F-DRF and B-spline) with a Nonlinear Autoregressive Neural Network (NAR-NN), asking the NAR-NN to provide parameter tuning. All the models provided accurate out-of-sample prediction; nevertheless, based on extensive statistical tests, we conclude that, as in the previous case, B-spline (combined with an NAR-NN) ensured the best out-of-sample prediction.

Scrap steel price forecasting with neural networks for east, north, south, central, northeast, and southwest China and at the national level

ABSTRACT There is little doubt about importance of forecasting commodity prices to policy makers and diverse varieties of market participants. In this present work, we analyse price forecasting problems for scrap steel for east, north, south, central, northeast, and southwest China and at the national level with daily data during 08/23/2013–04/15/2021. We focus on exploration of usefulness of non-linear auto-regressive neural networks and examine forecasting performance based upon a variety of settings over algorithms for model estimations, numbers of hidden neurons and delays, and ratios for splitting the data into training, validation, and testing phases. We arrive at relatively simple models for the seven price series, which lead to forecasts of high accuracy and stabilities with relative root mean square errors below 0.85%. Our results could serve as standalone technical forecasts. They could be combined with other forecasts for forming perspectives of price trends and carrying out policy analysis.

Short-Term Wind Speed Forecasting Using Nonlinear Autoregressive Neural Network: A Case Study in Kocaeli-Türkiye

Recently, wind energy has been utilized globally as a renewable, sustainable, and eco-friendly energy source. However, wind energy’s unpredictable and stochastic nature influences its entry into the national electrical grid. An effective wind speed prediction is required to meet these challenges. In this article, the Nonlinear Autoregressive Neural Network (NARNN) model is used and investigated for short-term wind speed forecasting by taking a dataset from the Kandıra wind farm in Kocaeli- Türkiye. The crux of the paper is to improve the actual application of the existing NARNN model with factual data using a different number of neurons of the hidden layer, delays, and training functions in the learning phase called the model’s hyperparameters. The mean squared error (MSE) and determination coefficient (R2) are used as performance measures. As a result, the hyperparameter optimization for wind speed prediction using the NARNN increased the forecasting performance. Suggested NARNN model is compared with its exogenous version (NARXNN) using three extra inputs. It is observed that NARNN is not falling behind NARXNN because they provide close results, and NARNN has been shorter to run. Likewise, the learning algorithms were also compared, and it turned out that Bayesian Regularization (BR) is the best learning algorithm. Still, Levenberg Marquardt (LM) algorithm is much faster to execute and provides close results to BR.

China mainland new energy index price forecasting with the neural network

For policymakers and investors, forecasting prices of energy indices has always been an important task. The present work focuses on the Chinese market and explores the daily price forecasting problem for the new energy index in the mainland during the period spanning January 4, 2016 – December 31, 2020. Our analysis is facilitated through the nonlinear autoregressive neural network model and one hundred and twenty model settings are tested in the fields of the algorithm for training the model, the number of hidden neurons utilized, the number of delays utilized, and the ratio utilized for segmenting the price series into different phases. Analysis here leads to the construction of a rather simple model based upon four delays and two hidden neurons with the Levenberg-Marquardt algorithm for training, which generates accurate and stable forecast results, with relative root mean square errors below 1.80% and mean absolute percentage errors below 1.30% across training, validation, and testing phases and for the overall sample. This constructed model also leads to statistically significantly better forecasting performance than a linear autoregressive model at the 1% significance level based upon the modified Diebold-Mariano test. The model built could be used as part of policy analysis for policymakers and decision making for investors. The forecasting results might also benefit design of similar energy indices by offering reference information in terms of price paths projected through the model.

Short-Term Prediction of PV Power Based on Combined Modal Decomposition and NARX-LSTM-LightGBM

Recently, solar energy has been gaining attention as one of the best promising renewable energy sources. Accurate PV power prediction models can solve the impact on the power system due to the non-linearity and randomness of PV power generation and play a crucial role in the operation and scheduling of power plants. This paper proposes a novel machine learning network framework to predict short-term PV power in a time-series manner. The combination of nonlinear auto-regressive neural networks with exogenous input (NARX), long short term memory (LSTM) neural network, and light gradient boosting machine (LightGBM) prediction model (NARX-LSTM-LightGBM) was constructed based on the combined modal decomposition. Specifically, this paper uses a dataset that includes ambient temperature, irradiance, inverter temperature, module temperature, etc. Firstly, the feature variables with high correlation effects on PV power were selected by Pearson correlation analysis. Furthermore, the PV power is decomposed into a new feature matrix by (EMD), (EEMD) and (CEEMDAN), i.e., the combination decomposition (CD), which deeply explores the intrinsic connection of PV power historical series information and reduces the non-smoothness of PV power. Finally, preliminary PV power prediction values and error correction vector are obtained by NARX prediction. Both are embedded into the NARX-LSTM-LightGBM model pair for PV power prediction, and then the error inverse method is used for weighted optimization to improve the accuracy of the PV power prediction. The experiments were conducted with the measured data from Andre Agassi College, USA, and the root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) of the model under different weather conditions were lower than 1.665 kw, 0.892 kw and 0.211, respectively, which are better than the prediction results of other models and proved the effectiveness of the model.

Predictions of solar activity cycles 25 and 26 using non-linear autoregressive exogenous neural networks

ABSTRACT This study presents new prediction models of the 11-yr solar activity cycles (SC) 25 and 26 based on multiple activity indicator parameters. The developed models are based on the use of non-linear autoregressive exogenous (NARX) neural network approach. The training period of the NARX model is from July 1749 to December 2019. The considered activity indicator parameters are the monthly sunspot number time series (SSN), the flare occurence frequency, the 10.7-cm solar radio flux, and the total solar irradiance (TSI). The neural network models are fed by these parameters independently and the prediction results are compared and verified. The obtained training, validation, and prediction results show that our models are accurate with an accuracy of about 90 per cent in the prediction of peak activity values. The current models produce the dual-peak maximum (Gnevyshev gap) very well. Based on the obtained results, the expected solar peaks in terms of SSN (monthly averaged smoothed) of the solar cycles 25 and 26 are RSSN = 116.6 (February 2025) and RSSN = 113.25 (October 2036), respectively. The expected time durations of SC 25 and SC 26 cycles are 9.2 and 11 yr, respectively. The activity levels of SC 25 and 26 are expected to be very close and similar to or weaker than SC 24. This suggests that these two cycles are at the minimum level of the Gleissberg cycle. A comparison with other reported studies shows that our results based on the NARX model are in good agreement.

A novel aging prediction method of fuel cell based on empirical mode decomposition and complexity threshold quantitative criterion

Data-driven methods have been widely applied to fault diagnosis and aging predictions to assist fuel cell Prognostic and Health Management (PHM) system, in order to achieve early maintenance management and corrective measures for fuel cell systems. This paper proposes a novel fuel cell aging prediction method considering the applicability of data and algorithm. This method first adopts empirical mode decomposition (EMD) to split the aging data into several intrinsic mode functions (IMFs), and each IMF represents a different characteristic. Then the sample entropy (SE) is used as the quantitative criterion for complexity threshold. Furthermore, the nonlinear autoregressive neural network (NARNN) and the Long Short-Term Memory (LSTM) recurrent neural network are combined to ensure the applicability of data and algorithm. The results show that EMD can split the various data types of the aging data and weaken or even eliminate the excessive mutation phenomenon that occurs at the beginning of each experimental fuel cell. In addition, the targeted selection of data-driven methods can ensure the applicability of the data and algorithm. Finally, by comparing different prediction methods, the proposed method shows higher accuracy in the prediction of each experimental dataset, and good generality for different fuel cell types.